Join 34,000+ subscribers and receive articles from our
blog about software quality, testing, QA and security.

Distributed report by Test Suites


#1

Hello,

I have created a Test Plan with many Test Suites. Each Test Suites represent a feature within my overall release. For reporting, I would add each Test Run (each Test Suites above within the Test Plan).

The report gives me an overall results which is what I expect.

I need to include in the report a Pass/Fail summary for each individual Test Run as well within this report as well. Any help would be appreciated to have this.

I know I can create individual reports for each individual runs but I need them consolidated in the same report with a total summary as well. On average I have 50+ runs within a Test Plan, so breaking it into 50 different report daily or even weekly (at a time) is not feasible.

Michael


#2

Hi Michael,

Thank you for your post. In this case, you could use the ‘Runs (Summary)’ report. This would allow you to run the report against multiple test runs and see the resulting data in one report. If this is not what you are looking for exactly, we would normally recommend then exporting your specific test run data and running custom reports against that data locally.

Additionally, we normally do not recommend using test suites for separating your test cases. Starting with TestRail 4.0, we made test suites optional for projects. We found that for many projects, having a single test case repository and organizing your test cases via sections and sub sections works better than separating your test cases into different test suites. This makes it easier to start new test runs, filter your test cases and navigate between your tests. Because of this we added additional project options to use different suite modes and e.g. disable test suites for projects.

You can enable and use test suites by editing a project under Administration > Projects. However, we highly recommend using sections and sub sections in a single test case repository instead of separate test suites for most projects. You can also enable test suites for a project later in case you want to start using it then. You can also learn more about suite modes here (section “Suite Modes and Baselines”):

http://blog.gurock.com/postings/introducing-testrail-4-0/2757/


#3

Hi Marty,

For Test Case repository, I am using Sections to break down features within a Project release. That is working fine. The question at hand is creating Test Runs and then creating reports for the test run. I am using version 4.3. Here is what I have currently:

  • Milestone for my project.
  • Test Plan (under Test Run) for this milestone.
    • under this Test Plan, I added multiple Test Suites.
    • for each Test Suites, I select the test cases to run per the Sections defined under the Test Suites test cases repository.

This works well as now, I have an overall Project/Milestone for the release and a breakdown by features for execution.

The problem is reporting. All your reporting provides an overall summary (P/F/B/U/etc…) which is great. But I need a similar summary for each feature as well, all in a single report.

The option of creating a report for each feature is there but does not work for me as each release, I have over 50 feature/sections. So generating 50+ reports daily is not feasible.

Please let me know if there is a way I can achieve the above. Or if there is a way to create a Report Group. Basically a Report that contains 50+ reports. I do not need the test cases to be listed for these type of reporting.

thanks,
Michael