Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Better way to display test plans / configuration groupings


#1

Hi folks,

I am using TestRail to organize testing several product modules. In each of these modules, there are several hardware platforms, which we use Configurations to manage.

The challenge for us right now is that some of the tests we have for a particular suite don’t need to be run on all platforms, and some do. So here’s what we’ve done, which I’m trying to figure out if there is a more graceful way to get to:

Module 1 - Configuration A, B, C
Test 1-10

Module 1 - Configuration B,C
Test 11, 12, 20

Module 1 - Configuration C
Test 13, 15

So now you see Module 1 called out three times, with different configurations. When you look at this from the test plan perspective, all you see is:

Module 1 - Configuration A, B, C

Module 1 - Configuration B,C

Module 1 - Configuration C

You have no idea what tests are in each one, yet they are listed multiple times, which seems like a mistake of some sort.

If I could put a title on each of these, then it would make more sense:

Module 1 - Configuration A,B,C - Full Platforms with normal tests
Module 1 - Configuration B,C - Special Customer Tests with limited platforms
Module 1 - Configuration C - Manual one-off tests

One of the test plans we tried (which we’re splitting up now) has over 3000 tests, and 40+ of the above, 8 distinct modules with 4-5 combos per module.

When we try to go and review this with others, it’s pretty confusing - especially the printed report.

Any ideas for streamlining this?


#2

Hello Steve,

The approach you described is the way to do this at the moment, but you can change the name of the resulting runs in order to better identify the test cases that are included in the run. E.g you mentioned:

Can’t you just enter the names (e.g. “Module 1 - Manual one-off tests”) for the runs when you create the test plan? But we will also look into making the test plan grouping and categorization more flexible in the future.

Thanks,
Dennis


#3

Oh wow, I can’t believe I missed that - I never even looked at the name of the test suite you added to the test plan, and the fact that it can be changed!

I’ll give that a try for some basic organization and clearer understanding of what’s being tested.

Even so, having a more flexible solution here would be great. Part of our process involves reviewing the test plans with a number of internal customers, and showing them the list of Platforms and customized names for each test area is too high level. Clicking into each one is a pain, and the printed report is too detailed. Something in between that showed something like this:

[code]Module 1 - Custom Name Here
Includes: Configuration A,B,C

List of Tests Here[/code]

That way you’re not repeating the same test list for each configuration, since it’s identical (it has to be right?) yet it’s more detail than the main page of the test plan.


#4

Hello Steve,

Thanks for the additional information. I agree that an alternative view for test plans that makes it easier to view associated tests (+ results) would be very useful to have. This could either be added as an alternative view or by expanding test runs on the existing test plan page somehow. I believe we also had other requests for this before. We will look into this and consider it for a future release.

Regards,
Dennis


#5

Thanks! Much appreciated.