Join 34,000+ subscribers and receive articles from our
blog about software quality, testing, QA and security.

Report on Configurations


#1

Hi there,

Hypothetically, let’s say I have a bunch of test cases that can be run on both Windows & Linux. So I create a test plan, and for my run I add a configuration for both platforms, and execute my tests.

At the end of the release I want to ask this question: ‘What is my pass rate on Windows compared to Linux’

The only thing I can do today is look at the comparison for cases report and create an algorithm to try and calculate the data displayed, but because its in an html format, this is very difficult to do.

Is there a way, using a custom report/api that I can actually query the configuration ID in a run and report on it?

This will help other questions like ‘What configurations do we run against often’ ‘Did we test on xxx configuration during xxx milestone’, etc…

I am trying to push my company to use configurations… and they are great to organize the test runs, but if I cannot report on them then it makes it difficult.


#2

Hey, just looking to see if anyone has a response?


#3

Hey!

Thanks for your posting. Yes, if you prefer a more structured approach than the HTML reports, you can either use TestRail’s API or the CSV/XML exports. Both options work well if you want to create custom reports, e.g. via Excel or custom scripts. The API and XML exports are the most structured way and also include everything regarding configurations/test plans. You can find a good overview of TestRail’s API here:

http://docs.gurock.com/testrail-api2/start
http://docs.gurock.com/testrail-api2/reference-plans

Exports in XML or CSV format can be downloaded on the test plan page in the toolbar next to the printer icon.

I hope this helps!

Cheers,
Tobias


#4

It does, thank you very much