Join 34,000+ subscribers and receive articles from our
blog about software quality, testing, QA and security.

Reporting execution percentage rather than passed percentage


#1

Everywhere I go in TestRail, whether it’s the test run page or a report, TestRail always likes to show me the percentage passed in big bold numbers. However, as a QA Manager, I first and foremost want to know what percentage of test cases have been executed by my team. This always seems to be the first question from executives outside the group. How would I go about accomplishing this sort of reporting with the current version of TestRail? If not available, can you please add it to your list of feature requests?

Click for full image -> http://cksv.net/testrail.png

Thanks,

Chris


#2

Hello Chris,

Thanks for your posting. It’s currently not supported to change/influence this behavior but this this is already on our list of things to look into. This request has come up before and we understand that this isn’t the desired behavior in all cases. That’s used in a lot of places in TestRail so it won’t be part of a small minor update but it is planned to look into this.

Regards,
Tobias


#3

I would also love to see this. This is requested a lot from upper management.


#4

hi,

if this feature will be developed i hope it is matter of customization as i do use and like current behavior as well.

Thanks,
V


#5

Yes, this would be customizable and one option to implement this would be with an additional flag for the test statuses (which can be checked/unchecked in the administration area).

Regards,
Tobias


#6

I would also like to see this, as it does seem redundant to have the same metric in two places, and have to calculate the execution from other info.


#7

Hello Jim,

Thanks for your feedback on this! Yes, it is planned to revisit the numbers in the statistics and make this more flexible. I’ve just added another vote to this feature request.

Regards,
Tobias


#8

Yes, please, another vote for this one!

Not only the specific spot called out in the annotated screenshot – but also the test run statistics (in the screenshot here you see 67% and 33%) indicating percentage passed for that test run, where I’d like to see ‘percentage executed’ (so everything I’ve defined as final status).

I love working with TestRail and this is one of the few things that I continually wish were different. It would make things much easier for the way we work.


#9

Thanks for your feedback on this and I’ve just added another vote to this request. The final status is used for the progress/burndown feature currently and I believe adding another, separate status value for this would be more flexible (also to stay backwards compatible).

Regards,
Tobias


#10

Apologies for reviving an old topic, but did anything ever happen with this? Having an execution percentage would be incredibly useful and it’s something people in my team still ask me about regularly.


#11

Agree, even though I just started an eval. Great product :thumbsup:
We have the colors as indicators and the tooltips if we need exact number of passed, failed etc.
The execution percentage would give nice feedback if it might be time to
"Close this test plan/Close this test run" or perhaps check that all automated results were imported.
The number (passed percentage) given today is a bit confusing especially for the progress bars and I think this would be something my colleagues would comment on.
Also # Untested could be added to the pie chart legends.


#12

Hi all!

We currently don’t have an update but it’s still planned to address this / make this more flexible in a future version. The current plan is to make this configurable per status similar to the Final option for statuses that influences the progress behavior.

The number of untested tests is already displayed below the overall percentage (although it’s currently a bit small).

Regards,
Tobias


#13

I would also like this feature. At the moment we’re hiding the passed% with an API, but this is still a problem for reports. If there’s an easy way to remove the passed% count next to the pie chart, please let me know. Changing the passed% to execution% would work too and another thing that would help in my case, would be if you could make other statuses to be counted in the passed%. We use a status called “Passed (Remark)” with the normal Passed status and would like to have both counted towards passed%.

Regards,
JuhaH


#14

Hi Juha,

Thanks for your feedback! It’s planned to make this more flexible and I’m happy to add another vote to this feature request. It can make sense to use the same Passed status for tests with remarks and without (and maybe use an additional checkbox custom field to indicate remarks), but we also want to support your use case/customization better with a future version as well.

Cheers,
Tobias


#15

I want to add my vote to change (or make it configurable) the percentage to overall execution percentage vs % passed.

Also % passed shouldn’t count skipped test cases (e.g. - Test run based on 2 skipped + 8 passed out of total 10 test cases shows 80% pass). Pass percentage should only be calculated based on test cases marked as pass or fail. This sends the wrong message to my manager and other involved parties.


#16

Thanks, Frank! This feature request has become really popular and we will make sure to look into this again for one of the next versions.

Cheers,
Tobias


#17

+1 vote! This would be a great feature, especially in combination with the request listed here:

Thanks!


#18

Thanks for your feedback, Ryan!

Cheers,
Tobias


#19

Heya Tobias!

Any updates on the likelihood of this feature being implemented in upcoming releases?

Thanks!


#20

Hi George,

Thanks for your posting and feedback. It’s still on our todo list and we would like to add this as part of a larger feature for the statuses (so you can control this on a per status basis and how this affects the reports/statistics). We currently don’t have an update on the time frame or an estimate unfortunately but it’s still on our todo list.

Cheers,
Tobias