Join 34,000+ subscribers and receive articles from our
blog about software quality, testing, QA and security.

% of passed test cases, and their basis (states)


#1

Let’s say I have a test run with 100 test cases.

for some reasons some of these test cases are wrong, not applicable, not executable etc.
lets say they I change their state to e.g. “not applicable” or any other (custom) state.

Is it possible to exclude certain states from the pie chart in the overview or from the bar chart in the test run overview ?
The reason for that: since certain test cases will never be executed (or even passed) because they are “not applicable” the pie charts will never reach 100% coverage (or even passed coverage).
I would like to exclude certain states from thr chart calculation.
Any option for that ?


#2

Hello,

Thanks for your posting. I would recommend deselecting the test cases in question from the test run or test plan in this case. You can do this when you edit a test run or test plan via the Select Cases links. This will remove the tests from the test run and also from the pie and bar charts. Would this work for you?

Regards,
Tobias


#3

Hi Tobias,

well yes it would work (I guess) concerning the pie charts.
On the other hand I’m losing information about planned test cases in this run, that are not executable for what ever reason. They simply are not listed in the plan anymore.

And even worse (for me): My total overall number of test cases in this run would be “a moving target”, as the total may increase from report to report.


#4

Thanks for the update. It’s not possible to exclude test cases from the statistics if they are still part of the test run or plan (e.g. hide them from the pie or bar charts). If the test cases do not apply to a certain test run (e.g., Not Applicable status), wouldn’t it make sense to remove those test cases from the test run? It’s normal that the number of tests in test runs change over time as you add new test cases to the related test suites and/or change your test case library.

Regards,
Tobias


#5

Hi Spacefrog,

I believe this is achievable with a UI script. I have a similar script that shows the percentage of total tests run next to the passed percentage on the “…/plans/view” page. It appears that the test result totals (number passed, number failed etc.) are stored as variables on this page and you can key on those. I’m sure with a little digging you could figure out a way to leverage those for your needs.


#6

Thanks for your feedback. It’s partially possible to do this with a UI script but this wouldn’t work in all cases so we don’t recommend this option. For example, if you are on the test run page, adding a new test result would refresh the stats and hooking into this (and modifying the stats again) wouldn’t be possible with a UI script currently. There are other situations where this wouldn’t work as well (e.g. the reports).

Regards,
Tobias


#7

Hello!
I have the same issue. Is there any possibility in TestRail 4.0 to remove status from the statistics?


#8

Hello,

If you just want to exclude select cases from the statistics, you can simply remove those cases from the test run/plan (by selecting them via the Select Cases dialog). We are happy to help if you could provide a few more details on what you are trying to accomplish.

Regards,
Tobias


#9

Hi
We have another issue relating to the statistics, we have added a status “Retest Passed” but the Passed % displayed on the far right hand side of the Piechart only reports the “Passed” status that is included in the original Testrail set up. Is there anyway that you can define which statuses should be included in this %?


#10

Hello,

Thanks for your posting. That’s currently not supported (but already on our feature request list) and we usually recommend assigning just the “Passed” status in this case. You can see the full history for tests, including the previous Retest state etc., when you view the test or also in various reports (Activity sections, for example), so it’s usually not needed to have this information in the final status as well. I’m happy to share more details if you could let me know what you are trying to implement exactly.

Thanks,
Tobias


#11

Hi Tobias,

+1 - this is also something our team could use.

We develop a product across a variety of platforms and so we often use 1 suite to create 30+ identical test runs (to be run across the various platforms). In this case, in every test run there are always going to be tests which are ‘Not Applicable’ or are ‘Not Tested’ (maybe because 5 of the platforms don’t support this feature, or the cases are low priority ‘test if time’ and we ran out of time, or perhaps a new feature was de-scoped from release).

This means we may get a Pie Chart which shows a PASS rate of 20% on a test run, which when the ‘Not Applicable’ and ‘Not Tested’ cases are removed would, would actually show a 90% pass rate.

While we could go through every test run and manually remove the test cases which are not applicable or ended up not being run, it is a large admin overhead in reporting. It would also be ideal to keep the results of the test cases in the runs, so we have a record of what features do and do not apply to each platform.

It would be extremely beneficial to our team to have a feature where we are able to customise the Pie Chart displayed at the top of the Test Run, Test Plan or Test Report, either across all Projects or only in specifically selected projects.

Thanks,
Justin


#12

Thanks for your feedback and the background information, Justin. I’ve just added your vote to this feature request. We are happy to look into this for a future version.

Thanks again!

Regards,
Tobias


#13

Hi,
I would like to add also a vote to this feature!


#14

Added, thanks :slight_smile:

Cheers,
Tobias


#15

Hi, I think update on test run result pie chart would be great. Is it possible to show only the statuses the team had used during execution other than showing all the statuses (I cannot see the value of displaying all the statuses which haven’t been used in a test run). I am more interested to know all the statuses the test are in at a given time.


#16

Hi N.R.,

Thanks for your feedback. That’s currently not possible but we are happy to consider this for a future version (and I’ve just added this to our feature request list). We haven’t received this request before but this definitely makes sense to keep the stats easier and fast to read, thanks again!

Cheers,
Tobias


#17

i think thats a nice idea - it supports also the view in the piechart.

So add a vote from me


#18

Added, thanks!

Cheers,
Tobias


#19

Hi,
Please add another vote to this topic.
At least it would be great to exclude the N.A. or Skipped tests in the reporting.
Best
Benjamin


#20

Added, thanks Benjamin :slight_smile:

Cheers,
Tobias