Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Delete specific Test Runs within a specific Test Plan


#1

I have a Test plan for which I have added at least one suite.
I’m using the configurations option to create test runs for a Windows and Mac configurations.
The problem is that while the majority of the test cases in the suite apply to both Windows and Mac, there are some test cases which are specific to only one of the configurations.

So when I generate the test runs, I end up having some number that can’t be executed.

I’d like the ability to easily remove these “invalid” test runs from the Test Plan… how can I do this?


#2

Hi Luis,

Thanks for your posting! You can just uncheck them and this looks as follows:

Cheers,
Tobias


#3

thanks Tobias,
but doing that removes all of the test cases for that specific config. Instead, I only want to remove some of them. Using the “select” option, next to the “As Above” link may help, except that it doesn’t show me that test cases that have been initially selected by using the “As Above” option so that I can easily remove the few that I don’t need.

We’ve tried to work around this apparent limitation by using a custom status, but as has been pointed out elsewhere in these Forums, doing that skews the statistics for the Test Plan… something which is highly undesirable.


#4

Hi Luis,

Yes, this would remove the full test run and I thought that was your intention (like in the screenshot above), sorry I got this wrong.

An elegant solution is to add a custom field of type dropdown/multi-select you use to manage the designated environment for your cases (Windows, Linux or both). You can then simply use the filter on the Select Cases dialog and this field to configure the case selection per configuration.

Cheers,
Tobias


#5

thanks… I’m considering doing exactly as you suggest, but that would require editing thousands of test cases to add the different environment definitions.

But it doesn’t address the need to remove individual test runs from a test plan.

Let me try to explain it better… say that I have 100 test cases in a suite. Of these 100 test cases, 80 apply to both Windows and Mac, whereas 15 test cases only apply to Windows, and 5 only apply to Mac.

I create a test plan, and select two configs (one Windows, and the other Mac). Test Rail will then generate 200 test runs… 100 for each platform, which is not exactly what’s needed. Instead, the Windows config should have 95 tests and Mac should have 85 tests. And while I could, in theory, be more selective when adding the individual test cases into each of the configs, doing so is far more tedious than adding all of them, and then removing the few that don’t apply to the config.

So the basic question is this: what’s the best way to remove a small number of tests which have been added to each of these configs? That is, in my example above, remove 5 not-applicable tests from the Windows config and remove 15 from the Mac config.


#6

Hello Luis,

The idea is to use the custom field on the case level to easily filter your cases on the Select Cases dialog. If you have a field on the case level, you can use the sidebar filter on the Select Cases dialog and the Set Selection/Add To Selection or Remove From Selection actions to set the selection. In your case, you could just select all cases, then configure the filter and deselect all cases that don’t apply (with Remove From Selection).

Cheers,
Tobias


#7

I get all of that… but you’re addressing the act of creating a test plan. And my question refers to a test plan that has already been created.

Should I assume from your comments that there is no way to remove a test case from a config once it has already been added to the test plan?


#8

Hi Luis,

The behavior is the same for creating a test plan or changing the plan later. You can change the case selection anytime (when you create it or later when you edit it) via the “Select cases” link per configuration.

Cheers,
Tobias


#9

doesn’t work… When I attempt to select cases for the entire test plan, I can see all of the test cases that have been selected, and while I can remove some, doing so removes them from ALL the configurations contained within the test plan.

yet, if I attempt to deselect a few test cases within the individual configuration, I find that the select cases contains no tests checked.

which basically gets me right back to my original/frustrating problem.


#10

Hi Luis,

The case selection can be different for every configuration and there’s the selection for the group of configurations (which comes into effect if a configuration does not override the selection). So, the idea is to override the selection per configuration.

For example, to configure the case selection for the Windows configuration, you would click “select cases” for this configuration, then select all cases, configure the Mac filter and choose Remove From Selection. You would end up with all cases that only apply to Windows. This assumes that the custom field would track only the environments it would apply for exclusively.

If you use a multi-select field for the cases and track all possible environments, you can also use a positive filter instead and start with an empty case selection. In this case, just click “select cases” again, choose Windows as the filter and then click “Update Selection”.

Cheers,
Tobias