Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Question about Configurations


#1

I searched the forums but didn’t see anything pertaining to my specific scenario. If there is an applicable thread, sorry for the duplicate.

Up until recently, we’ve been focusing our tests on Windows configurations. As such, for Configurations, I had a “Windows” group and a handful of browsers within that group. When only those configurations are selected for a test suite I’ve added to a Test Plan, I see exactly what I expect. Example:

Selecting browsers from ONLY my Windows group like this,

Will result in a list of test passes for each of those browsers,

But if I add a configuration from the “Mac” group and one from the “Mobile” group,

I expect to get a list of tests to perform which has one for each configuration but rather I get a list of tests that seem to be combinations of the configurations I’ve selected

Is this something that I can configure? Or should I workaround it by just having a single group with configurations named such as:
Mac - Safari 5.1
Mac - Chrome 17
Mobile - iPhone
Windows - IE8
Windows - IE9
Windows - Firefox10

Thanks!


#2

Hello,

Thanks for your posting. Yes, the configurations from different configuration groups are currently “multiplied” so that you can start test runs for each configuration combination. Your solution is what we currently recommend if you just want to test against specific configuration combinations.

It is also planned to add support for just selecting specific combinations in a future version of TestRail.

I hope this helps.

Regards,
Dennis


#3

Thanks Dennis! Good to know that; A) I’m on the right track with my workaround, and B) that this type of configuration selection is slated for a future version.

Thanks again!
-Jason


#4

Hello,
I came across this issue this week and was glad to find this thread here to see I wasn’t doing something wrong, but I am curious if this is on the roadmap for a future version. I’d like to be able to choose configurations from various configuration groups without having them ‘multiplied’ as they are – one test run per configuration was what I was expecting.
Thanks!


#5

Thanks for your feedback on this. We currently recommend configuring the configuration groups manually if you don’t want to use all combinations or the multiply feature. It’s planned to make this more flexible (e.g. by choosing from a list of possible combinations).

Regards,
Tobias