Join 34,000+ subscribers and receive articles from our
blog about software quality, testing, QA and security.

One Case, multiple, parametrized test runs for test case


#1

Hello
In our organization’s automated test strategy we have very offen following use case:

  • One, generic test case defined for particular feature / requirement
  • Multiple tests executed based on above parametrized test case.

To identify particular test execution for particular parameter each of them have some unique identifier.

Now how can I implement it in TestRail?
TestRail provide unique id for test run and test (execution item) but it is not possible to include one test case many times inside the same Test run.

Perfectly to solve my issue would be to have one test run (RXXX) and mutiple tests (TXXX) of same test case (CXXX) additionally distinguished by additional parameter (label) but for Test instance (TXXX).
Today it is not possible to add any parameters to test instance (TXXX), only comments and test resuls

thanks
Tomasz


#2

I realized that I could implement this what by adding multiple test results to the same test instance (TXXX) and label every result with specific custome parameter value.

But…
The final test status for test is implemented as last test status.
In my case it should be “worst case” - e.g. one test results is FAILED than final status should be FAILED.

Is it possible to have control over final status calculation method in Test Rail??

What is more complex, I will also have “normal”, not parametrized test cases, executed only once per test run and the current method of final status calculation is proper for them…

similar topic I found in Aggregate Multiple Results
The generation of custom reports is not a solution for me, because depending on test case, I need to apply different method for “final test status” calculation


#3

Hi Tomas,

I also want to use single test cases that test over multiple values of data. What I would like is if the test results can query the user for multiple results. (e.g. test behavior with low battery at two different voltages.)

Did you resolve this question?

Thanks,
Howie


#4

Hi
Unfortunatelly I did not resolve this issue and this blocks implementation of Test Rail in my test organization.
I don’t see also any suggestion here from TestRail developers…

In case you will find any solution, please share with me.


#5

I would email Gurock support…


#6

Hi Tomasz,

Thank you for your response.

Talking with TestRail support, they suggested writing JavaScript/CSS
scripts to tie test steps together for an overall test result. There isn’t
a mechanism built into the tool.

We are still doing eval, and this issue is concerning.

Howie.

Howie.