Join 34,000+ subscribers and receive articles from our
blog about software quality, testing, QA and security.

Bug In Summary Plan Report Filtering?


#1

On the activity tab I selected “The following statuses only:” and selected Passed and Failed only. When I generate the report the list of tests executed is filtered under activity but the graph and table include other statuses (which are also included in the % calculations).

I would expect the summary and list to use the filter so that the counts match:

Cheers

JG


#2

Hi John,

Thanks for your posting! The stats/chart always show the full statistics and the filter only applies to the list below the chart. The list would only include the results according to the filter you configured (Passed & Failed in your case).

Cheers,
Tobias


#3

Hi Tobias,

That is not very helpful as the intention (and expectation of the the user - me) of the entire report is to display stats and charts based on the filter (only test cases with Passed & Failed). It defeats the purpose of the filter if the chart is going to display all statuses. Are there any improvements in the works to improve this reporting based on filters?

Thanks,
Sriram


#4

Hi Sriram,

Thanks for your reply. There are currently no immediate plans to change this but this has been asked before and I understand that this may be a bit confusing and it’s planned to review this again for a future version. I know this sounds a bit vague but we are happy to look into this again, thanks again for your feedback!

Cheers,
Tobias


#5

Hello!

I would also like to add my vote to this feature request. I think this is the biggest miss in TestRail for my workflow.

My use case is the following:

I test a product which may be used by multiple customers. Some customer share the same needs an so the same test cases, but there are also some specific tests.
I would like to be able to generate a report per customer, by filtering the tests at the report generation. This is not currently relevant as the statistics/charts are not correct.
For the moment, the only way to do it is to create one test plan per customer. But it means entering multiple time the same test results into multiple test plans, although the test it self has been executed only one time in reality. :sweat:


#6

Hello,

Thanks for your feedback! Your use case definitely makes sense for this type of change to the report. I’ve added your feedback as well as another vote to this request on your behalf, and we’re happy to review this as well. If you’re using TestRail Server, you might also want to look into building a custom report plugin in the meantime if this is mission critical for your team:

http://docs.gurock.com/testrail-custom/reports-introduction

Hope this helps!

Regards,
Marco


#7

I would also like to add my vote to this feature request.

My use case is the following:
I need to split apart my reports, as I have too many test cases to list in one report. It is confusing that the top pie chart summary is the same for all reports, and makes it hard to find the details for the failed test cases.

Any word on changing this in TestRail?
Thanks,
Donna