Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Assigning Filter Options to Test Plan/Run Configurations


#1

Hi TestRail,

Is there currently any way to assign a set of filter options to Test Plan/Run configurations?

The use case here is that when I create a Test Plan and add a configuration set to one of the Test Plan Test Runs, it is very time consuming to go through each configuration ‘Select Cases’ and set the desired filter options, especially when my test plans can have 20+ Test Run configurations.

Current:

  1. Create Test Plan
  2. Add Test Suites
  3. Set configurations for each Test Run (e.g. “Chrome US”, “Chrome UK”, “Firefox US”, etc)
  4. Select Cases for first Test Run configuration
  5. Select filter, apply
  6. Select Cases for second Test Run configuration
  7. Select filter, apply
    etc

Feature Request:

Preconditions:
Test Configuration “Chrome US” will apply filter options:

  • Platform: Chrome
  • Region: US

Test Configuration “Chrome UK” will apply filter options:

  • Platform: Chrome
  • Region: UK
  1. Create Test Plan
  2. Add Test Run
  3. Set configurations for each Test Run (i.e. Chrome US, Chrome UK, Firefox US, etc)
  4. Test Suite configurations appear with Test Cases already filtered and selected according to configuration filter options

Additionally, if I add test cases to the Test Suite with these filter options, I have to manually go through all the configurations again and select cases, re-apply the filter, and save. It would be great if there was a feature/button within Test Plan edit mode where you could automatically re-apply all the configuration’s current filter options to the Test Plan Test Runs.

Thanks,
Justin


#2

Hi Justin,

Thanks for your posting. If all configurations share the same set of cases, you can simply set the case selection one per group and you wouldn’t need to apply the selection to each and every run/configuration in this group. Is this possible with your setup?

Cheers,
Tobias


#3

Hi Tobias,

All the configurations do share the same cases, but they are all using a different subset of the test cases.
Each test suite contains all the tests across all platforms, regions, priority level, etc.

I have the following setup:

Test Plan:

Test Suite 1

10 configurations

Test Suite 2

10 configurations

Test Suite 9

10 configurations

For example three simplistic configurations for each suite in our test plan structure might be:

Platform: Chrome
Region: US
Priority: 0 - Showstoppers, 1 - Must Test, 2 - Critical Tests

Platform: Chrome
Region: UK
Priority: 0 - Showstoppers, 1 - Must Test

Platform: IE
Region: US
Priority: 0 - Showstoppers

Each selection thus has a different set of cases based on it’s platform and other conditions - screen shot example of one typical suite configuration below:

The problem is:

  1. When setting up the test plan I have to spend a lot of time selecting the cases based on their filter set. This can be reduced by re-running pre-existing test plans where the configurations’ test selections are already set.

  2. When writing brand new cases into any of the suites, I have to go through each configuration in all open test plans and re-set the cases again based on the configuration parameters they were at. This is true even when re-running pre-existing test runs.

I hope that’s clear enough on my issue, let me know if you need any other details or if you have any advice on dealing with this admin overhead without the requirement of a new feature.

Thanks,
Justin


#4

Thanks for the update, Justin, that’s appreciated. It looks like some of the configuration share the same set of cases, it this correct? If so, and if the other configuration have the same subset, you would be able to use the group-level case selection and then just override the selection for select configurations. It’s also already planned to make the filters more flexible, for example by applying filters to multiple configurations or saving/reusing filters. I’m happy to add another vote to this feature request, thanks again for your feedback!

Cheers,
Tobias


#5

Perfect, thank you Tobias.

Justin


#6

Thanks again for your feedback, Justin, that’s appreciated!

Cheers,
Tobias


#7

+1 for reusable filters.

I would like to be able to set a filter in the “select cases” area of the test plan and then the selection dynamically change based on the filter criteria when new test cases are added (without having to go back and update the filtered selection).


#8

Thanks for your feedback, Chris, happy to add another vote!

Cheers,
Tobias


#9

Is there any update now?
Have the same problem as Justin

Thanks.

Raphael


#10

Hi Raphael,

Thank you for the follow up post. This is something that is still being looked at. I would however be happy to add a vote for you.