Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Using APIs to find unstable tests


#1

Hey,

I would like to use TestRail APIs to get a report similar to the one that is available via “Tests & Results” tab on a View Case page or “History & Context” tab on a View Test page. Idea is to identify test cases which have been intermittently failing over the last few runs.

I was thinking of doing using a bunch of different APIs to get Plans --> use plan ids to get run ids --> use run ids with case ids to get test results and then using this information. This would be a lot of back and forth calls via APIs.

Am I missing something? Any ideas?

Thanks!


#2

Hello Nullin,

Thanks for your posting! There’s currently no direct API for this and we would also recommend using a different approach. For example, one option are the XML exports and they include everything that’s needed to generate similar reports on your own. You can generate the XML exports on the milestone, plan or run level and they include all runs/tests and results.

An alternative is to use one of the built-in reports such as the Results > Comparison for Cases report from the Reports tab and you can also configure TestRail to automatically create a new report every day, for example:

Cheers,
Tobias


#3

Thanks Tobias, but I do want to do this via APIs. I have been able to implement something to get this working. It’s a bit heavy on the amount of data pulled from TestRail and takes a tad bit longer than I would like, but it gets the job done.


#4

Yep, that works as well of course and is easier to automate so good to hear that it works for you!

Cheers,
Tobias