Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.

Mark a test as failed if there is at least one failed result


If a test has multiple results, its status will be treated by the status of last result. I want my test status to be failed if there is at least one failed result in a test, doesn’t matter if it is last or not.

Is it possible to achieve right now? If not, then are there any plans for this feature in the future?


Let me understand this: assume you have a test in a test run, and have 3 results for it, say, Fail for the first result, and Pass for Results 2 and 3. For this set of test results, you want to say the test failed?

If this is so, could you explain the motivation for such behavior. I would think the Pass results of 2 and 3 are better and “out-vote” the Fail of result 1. Or perhaps these 3 results are due to running different test steps within the test?


Yes, your understanding is correct.

While I see that current behavior is the most proper one for the majority of cases, let me explain why I want it to be changed in my particular case.

I have a Test Suite. I create Test Runs based on this Test Suite. I need to run this Suite simultaneously on, say, 20 different mobile devices. For every Test Case in Test Suite I have my automated tests, which are integrated with TestRail with the help of TestRail API.

First solution for me was to create a new Test Run for each device (I use cloud devices and Test Runs are creating automatically). Then it would leave me with 20 Test Runs. It creates a lot of mess in TestRail, especially having in mind that I plan to increase the amount of devices in the future or/and the run my tests more frequently.

To get rid of the mess, I decided to create a single Test Run, where each Test Result in a Test corresponds to the device where Test Case was executed (I add device name to Test Result description to differentiate them).

That’s why I want my Test to be failed if there is at least one failed Test Result in it. So I will pay my attention to this Test, open it, and see on what device it failed.

I’m open to any criticism about my solution. Maybe I’s missing something about TestRail.