Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Individual configuration do not mark tests previously selected at Test Plan level?


#1

Hi,
I’m creating a Test Plan with a couple of test runs generated by using Configurations. Test cases are selected from various sections in a big suite and then assigned to the mentioned Test Plan. Some of these selected tests are not applicable to certain runs, so I’ll need to find those tests then remove them from the certain runs. The trouble is, the selected tests at the Test Plan level are not marked as such when modifying test case selection at runs level. Am I missing any step to show this info at runs level? or I’ll have to start selecting each needed test from the suite over again if I want to customize tests selection at runs level?

I’m hoping I’m not doing it the right way, otherwise I don’t understand the idea of selecting tests at TP level, and this info can not be used at the runs level to further filter out tests.

Anyone facing this before? appreciate if you could shed some light.

Thanks,

Vincent.


#2

Hello Vincent,

Thanks for your posting. The case selection at the test plan entry level is the default selection for all test runs in this group. You can override the selection per run/configuration as needed but this is independent of the group/entry-level selection. TestRail will fully respect the selection when rerunning test plans (via the Rerun button) so you can reuse the selection when creating new test plans based on previous plans.

Regards,
Tobias


#3

Hi,
Thanks Tobias for your quick reply; much appreciated.

This is not helpful for our use, and probably others. We need to be able to further fine tune the selection at run/config level within the test plan selection only. That is, start with the selection at the test plan level, then do further filtering (adding more to the selection, removing some of existing selection, etc.) at run/config level. Right now, looks like we cannot do that. Instead, we have to repeat the process of selecting tests out from the entire suite, which is big and has multiple sections and sub-sections. For each run. That’s a lot of manual effort.

Is there any other approach or method to accomplish this without having to select tests directly from suite for every run? It could be done a lot quicker if there is an option to use TP selection to start with.

Thanks,

Vincent.


#4

Just some more details what I’d like to do:

  1. create a test plan; assign a suite to it. Select tests from various sections in the suite and assign them to the test plan
  2. create a configuration with runs A, B, C, D. Link/assign the runs to the test plan
  3. Use the default selection from test plan to start with, further remove tests not applicable to certain runs. This is because:
    – a few % of the particular selected tests are not applicable to run A, thus will need to remove them from A
    – a few % of other particular selected tests are not applicable to run B, thus will need to remove them from B
    – similar for runs C, D
  4. Save the test plan, with fine tuned selection for each run.

Right now, we cannot do step 3 by using the selection from test plan to start with. Those selected at test plan level are not marked as selected while editing at run level. Instead, we’ll have to start with a blank sheet (ie. select tests from the suite again, for each run)


#5

Thanks for the additional details! If you use a selection filter for the case selection, the same filter is still active when you select the cases for other test runs. You can use the filter as a starting point for the other runs and you wouldn’t need to configure the same filter for each test runs. It’s also planned to make it easier to re-use previous case selections independent of the filters (e.g. from the group selection or from other configurations) and I’m happy to add another vote to this feature request.

Regards,
Tobias


#6

Hi,

Thanks for your reply Tobias.

Yeah actually while waiting I’ve tried a bit harder and realized group selection can be made available again, except that you need to click the ‘Set Selection’ button at bottom right to make it work.

Anyhow, yes definitely I’ll vote for an enhancement to re-use previous tests selections in dependent of filters. I mean, we work with test plan / configuration / test selection on a fairly frequent basis. It’d help a lot.

Regards,

Vincent.


#7

Thanks again for your feedback, Vincent!


#8

I’d like to add a vote for this enhancement too. My team has exactly the same need, for now it really is a pain to build a test plan on multiple configurations (we usually do it by opening 2 TestRail sessions on 2 different screens, one to see the group selection, another one too select the configuration test cases, not really handy…) and being able to see the default group selection to tune it on individual configurations would save us a lot of time.
Thanks

Sandra


#9

Hi Sandra,

Happy to look into it and just added your vote, thanks for the feedback :slight_smile:

Cheers,
Tobias