Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Feature Request: Disable some test runs, regarding with Configurations


#1

Hi, I am starting to use Testrail, so I am starting to setting up my test plans and cases.

Well, I found a little ‘problem’ when we use Configurations in the test plans. We have the following: TestPlan 1 --> TestRun 1 and we have defined the configuration OperatingSystem (Win7 and WinXP) and WebBrowers (IE8 and Firefox).

TestRail will create automatically the following test plan:

TestPlan 1:
- TestRun1 (Win7 IE8)
- TestRun1 (Win7 Firefox)
- TestRun1 (WinXP IE8)
- TestRun1 (WinXP Firefox)

This is a great feature from TestRail, but in our case we don’t want to test (for example) the TestRun1 (WinXP IE8), Is there a way to disable this test run?

In addtion, I would like to request other feather related with this. Exporting to CSV file, should be possible export the configuration to different fields? Now we will export to:

        - Configuration: Win7 IE8
        - Configuration: Win7 Firefox
        ........

I request the following:
- Configuration (OS): Win7 Configuration(WB): Firefox
- Configuration (OS): Win7 Configuration(WB): IE8

OK, that’s all :slight_smile:

Congratulations for work.

Best regards


#2

Hello Juan,

Thanks for your feedback. Yes, we are considering allowing you to select the exact configuration combination you want to test against in the future and we’ve received this request before. For now there are two options that could work for you:

  1. Instead of creating configurations like OS > Win 7 and Web Browser > Firefox, you could directly create the configuration combination you need and then select them, e.g.:

Combination

Win 7 & Firefox
Win 7 & IE 8
Win XP & IE 8

  1. If you want to test most of the combination, you could mark all tests of the few combination you don’t need as ‘blocked’. This way you would indicate that you don’t need to verify those tests and you would simply not assign those tests to any user.

I hope this helps. Just let me know in case you have any further questions.

Thanks,
Dennis


#3

I really like more your second option. The cases should be ‘blocked’ one by one, or is there a way to ‘blocked’ some cases at the same time?

Thanks for keep in mind my feature request.


#4

Hi Juan,

Sorry for the delayed response, the forum’s email notification didn’t get sent correctly. You can select multiple tests by checking the box next to the section titles. You can then scroll down to the bottom of the page and click the Add Result button to mark all selected tests as blocked. It’s also likely that we will add a Select All link to the page in the future.

I hope this helps.

Thanks,
Dennis


#5

Thanks Dennis!!


#6

Hello Dennis,
definetively we opted to block innecessary combinations. This is a good solution, but the TODO list is never completed because the blocked test cases.

Should be possible create a new state for theese cases? Something like ‘NOTNEEDED’, with this we can clean the TODO list.

Thanks.


#7

Hi Juan,

Thanks for your feedback. We would like to support choosing the configuration combination so we will hopefully have this eventually. You could also simply remove the assignments for these tests. While they would still appear in the overall statistics, you wouldn’t have those tests on your to-do page then. To do this you would select the tests (e.g. by selecting the entire sections), and click the Assign To button to remove the assignment (simply select the empty entry in the dialog).

I hope this helps.

Thanks,
Dennis


#8

Thanks for your quick and useful answer.


#9

I’d like to put in my vote for having the “Select Configurations” be, well more configurable. It’s a really handy option when for when you want to do the same tests across multiple configurations.

But, in our environment, we have more than 3,000 possible combinations of basic system configurations. We use all-pairs and other methods to whittle that number down for any given release.

I’ve read through some of the workarounds offered, and we have our own that we’ve come up with. But none are ideal. I don’t want to have to block 2900+ combinations to get at the ones I want. And I don’t want to have to put 3000+ combos in the selection list.

Thanks,

J Fincher


#10

Hello,

Thanks for the additional feedback. Yes, this is still on our to-do list and while I don’t have time frame for this, we definitely understand that this would be useful to have and we want to look into this. I appreciate the additional feedback and I added this to our feature entry.


#11

Yes, one vote for this feature too, more flexible for configuration selections.


#12

Hi!

This is available (and was added with TR 4 I believe) and you can flexibly select/deselect all possible configuration combinations:

I hope this helps!

Cheers,
Tobias


#13

awesome, thank you:+1:


#14

You are welcome, glad to hear that this helps :slight_smile:

Cheers,
Tobias