Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Test result tagging to filter test case results by cause


#1

We need to track how many times or what percentage of times a certain issue caused a test case to fail. Please consider providing ability to use tags on test results (similar to Pivotal Tracker labels in stories). A solution would be to chart the trend of test case success (without tag filtering) vs failure activity provided existence of a test result tag on the failures.

It would be identical to the current Test Results chart - but to filter the test failures based on the tag, symptom, or cause. The practicality of this: Sometimes issues are not 100% within engineering’s control, due to dependability and availability of interfaces with 3rd parties. Getting these trends help QA and product teams determine what to consider in improving reliability of product and service.


#2

Hello Matus,

Thanks for your posting. You can already add a new ‘Tags’ field to the test results via a multi-select custom field and this looks as follows:

(this example adds the field to the test cases but you can add this to test results as well)

To add this field, please go to Administration > Customizations and add a new field of type multi-select (under Test Result Field). Additional details about custom fields can be found on the following page and we are also happy to help in case you have any questions:

http://docs.gurock.com/testrail-userguide/howto-fields

Regards,
Tobias


#3

Thanks Tobias,

That should get us half way. The next step would be to filter the data in the Test Results chart by the test result tags. We wouldn’t want to have to drill down into every test run/result to count the number of times a specific tag appears in the failed tests.


#4

Hello Matus,

Thanks for the update. There’s currently no support for this unfortunately. There are several ways to display the new tags field in TestRail but it’s currently not supported to filter the stats based on this field.

What you can already do is to add a column with the tags to the test tables and this works via the small column icon in the top-right of the tables. For example:

http://blog.gurock.com/wp-content/uploads/2012/04/columns-large.png

Regards,
Tobias


#5

Tobias,

Can you insert a feature request to add a new custom field type for tags? It would behave different than multi-select, with the added feature of typing new options in on the fly. Currently the work aorund is for the administrator to go into customizations and add a new option and then the user can make the selection. The way it is we have a constant maintenance issue for managing failure reasons.


#6

Hello Matthew,

Sure, I’ve just added this to our feature request list, thanks! We use this approach in order to avoid that users use many similar, but slightly different tags for the same things (e.g. “mac-osx”, “mac-os”, “apple-mac”). This would make it very difficult for the user to filter on a multi-select field. That said, I understand that using a more flexible approach can be useful as well and I’ve added this to our list of things to look into.

Regards,
Tobias