Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Combined manual and automation test run reports

#1

We’re currently going through a huge initiative which partially involves providing more accurate reporting of our test runs. With the recent increase in our test automation coverage translated from our manual TestRail test cases, we need to identify just how many test cases have now been automated.

One of the exercises we have initiated, was to go through the manual test cases and set their ‘Type’ as ‘Automated’ from the dropdown field. This then allows the testers to avoid these test cases for execution.

Since we’ve also adopted the TestRail API within our automation framework, we can currently generate two separate reports, one on the manual test run, and the second for the automation test run.

What I’m seeking, is for there to be a single combined report which will provide a visual of the total amount of test cases which are automated and manual. A chart of some sort and if possible, to also state the execution remaining/pass/failed etc.

Would this be achievable in TestRail?

0 Likes

#2

Hi Pedro,

Thanks for the post! If you are wanting to obtain a report which shows the number of tests grouped by type, you may find the ‘Property Distribution’ report helpful (found under the ‘Results’ section), as this would provide overall statistics on each test by type, or other grouping you prefer. This report can also include the current status for each included test.

This report wouldn’t provide overall progress for each test run for effort remaining, but would provide the overall breakdown of automated, manual, and other test types.

Hope this helps,
Jon

0 Likes

#3

Do you need to keep the automated and manual runs separate? Could you add manual tests to the automated test run?

0 Likes

#4

Hi Pedro,

There’s isn’t a single report that will show both count for automated vs manual and provided statistics based on that test case filter, unfortunately.

As Jon has pointed out the Property Distribution would provide the count of manual vs automatedif you use “Group the test cases by: Automation Type”

The Summary for Runs is then the ideal report for finding state the execution remaining/pass/failed etc.

That said, it would be possible to manually combine the two reports by downloading the PDF separately then using a free tool such as PDF Creator, merge them together.

0 Likes