Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

New project test organization

Our QA department is brand new to Test Rail and we’re wanting to ask around for the best way to organize our tests.
Here’s some details about our software:

  • Has 20+ separate programs that need to be tested
  • Has specific functionality that will need to be tested
  • Has a desktop component
  • Has a web component (SOAP and REST)
  • Has a mobile componenent
  • Will need the following test types (of course we could need more later):
  • Smoke
  • Functional
  • Regression
  • Performance

I believe those are the main bullet points.

We originally started creating folders for test cases based on test type, but we quickly realized that wasn’t ideal. Mainly because we had to make sure the folder contained the correct test types. We found that we could create a folder called Functional and put a test tagged as “Performance” in it without anything stopping us.

Our next thought is to create a test case folder structure based on programs or functionality. For example Inventory Manager (a program) as a folder and Farmouts (outsourcing functionality) as a folder. This would allow us to use the test case field “Section” as the program\functionality name and for the test type we’d obviously use the test case “Type” field.

Truthfully, we’d like to stay away from creating any folders at all and maybe have a custom test case field for “Program\Functionality” and have our programs and functionality listed in that.

Any thoughts or suggestions?

Thank you for you time. I know brevity isn’t my strong point.

-Rob

My primary tip would be to focus on what you want your reporting to show to stakeholders and then look at how you can organise your test rail structure to enable it.

We have a number of projects with separate release cycles and processes at different levels of maturity. Hence there was not a structure that fitted all projects, but we did reduce it to 2 main structures:

1 with the following top level sections to aid tracking of larger projects:
Sprint Planning (an area to design the tests, review them and execute them)
Low Priority Tests (tests that we don’t see important enough for regression, moved here after execution)
Non-Functional Tests (as it states, subsections separating the different types)
Regression (tests that we do see important enough for regression, moved here after execution)
Archive (tests no longer used)

The other with the following top level sections for simpler projects:
Functional Tests
Non-Functional Tests

For both of these, we’d add subsections in line with functional areas/epics. We make use of the fields such as test type to help with reporting and tracking too. In addition we have added some custom fields to add further clarity.

Importantly our structure has adapted over 6 months tweaking things that made tracking and reporting easier. Accept it won’t be right first time, start it simple. Hope that helps.

G