Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Having same test as part of multiple testsuites, multiple platforms


#1

I had a few questions while trying Test Rail out, and see if we can map it to the situation on hand. To explain more in detail…

Let us say we have the following scenario:
I have 3 platforms (platform1,platform2,platform3).
2 Testcases (tc1,tc2)
2 Test Suites (TestSuite1 and TestSuite2)

For a given test run,
TestSuite1 comprising tc1 on platform1 ; Testsuite 1 comprising tc1,tc2 and Testsuite 2 comprising tc2 only on platform2 ; TestSuite1 comprising tc1,tc2 ; TestSuite2 comprising tc1,tc2 on platform3
In this case,

I need to be clearly able to project the results as follows

TestRun1_Results

                            |               Platform1	            |                     Platform2	     |               Platform3

------------------------- |-----------------------------------|------------------------------------- |---------------------------------
testcases\testsuites | TestSuite1 | TestSuite2 | TestSuite1 | TestSuite2 | TestSuite1 | TestSuite2
------------------------ -|----------------|-----------------|------------------ |-------------------|-----------------|--------------

tc1 Pass Not Run Pass Not Run Fail Fail
tc2 Not Run Not Run Pass Fail Pass Pass
------------------------- ---------------- ------------------ ------------------- ------------------- ----------------- ------------------

Following are my questions :

  1. Is it possible to achieve the above without customization and without duplicating the same tests under multiple sections? For instance, I would not want to have separate sections for TestSuite1 and TestSuite2 and replicate the testcases tc1 and tc2 in both the sections.
    And similarly, I would not want to have separate sections for each platform.

  2. If possible, can you direct me how to do the same… Or else, if it can be done only through customization, can you tell me what is the best way to use customization option to achieve this.

  3. In terms of automation, do we have enough support to deal with this customization?

Looking forward for your support with regards to this.


#2


#3

Hi Rohith,

We would usually recommend using a single test suite in this case as this makes it easier to start test run and compare results. TestRail even comes with a single-suite mode for this and you would only see a single, combined case repository in this case:

You can organize and group test cases inside the case repository with sections and the hierarchy can be as complex or simple as you like.

For the platforms, we recommend using configurations and test plans and this automates a lot of things:

http://www.gurock.com/testrail/videos/test-plans-configurations/

Using this approach, you can then easily compare the different platforms using the Results > Comparison for Cases report and this looks as follows:

Just let me know in case anything is unclear, happy to help.

Cheers,
Tobias