Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Configurations and Client/Server architecture


#1

Hi,

After some time of evaluation we decided for Testrail as our Testmanagment Tool. So I am quite new to testrail. I am now in the progress of setting up the first tests. As our product has a client/server architecture, we need to test the client on different platforms against the server on different platforms. That’s where I thought that configurations may come in handy, but I am not quite satisfied with the results. At the moment the configurations look like this:

Architecture

  • 32bit
  • 64bit

Component

  • client
  • server

Operating System

  • Windows Server 2008
  • Windows Server 2012
  • Windows XP
  • Windows 7
  • Windows 8
  • Windows 10
  • CentOS 7

Processor family

  • x86

That gives me the e.g. the following test runs:

x86, 64bit, Server, Windows Server 2008
x86, 32bit, Client, Windows 10
x86, 32bit, Client windows 8

You get the idea.The test cases are mostly generic, as we usually don’t test on different platforms in everyday testing (Java Software). So, when running the test cases in one of the test runs I have no information that let’s say the test cases in x86,32bit,Client,Windows need to be run against x86,64bit,Server,Windows Server 2008.

I don’t want to duplicate test cases for every combination. I also don’t want to include every possible combination in the configurations which may not the intended use of configurations.

So I need some advice, how to best organize testing combinations of our client/server software.

Thanks a lot,

Ben


#2

Hi Ben,

Thanks for your posting. What’s often done is to specify the exact configurations inside a single group (or maybe two groups in your case: one for the client and one for the server) instead of using multiple groups and let TestRail calculate all possible combinations (which can be many with lots of groups and items). So, instead of letting TestRail calculate every possible combination, you could use a single group with just the configurations you would really like to use and that make sense in your case. Would this work for you?

Cheers,
Tobias


#3

Hi Tobias,

Yes, that’s what I have also thought of, as I have seen no other possibilty within Testrail. I really enjoy that Testrail calculates all possible combinations, as entering all combinations by hand is pretty awkward, although it is a one-time job if no other configurations are added later on. As far as I can see the only feature that is missing is a second layer of automatic combinations. But you are absolutely right, that this can become very hard to handle. So I will try as you suggested if this is considered best-practice.

Thanks for your support!

Ben


#4

Thanks, Ben, sounds good. Please let me know in case any questions come up about how to organize this, happy to help.

Have a good week!

Cheers,
Tobias