Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Coverage for references only shows 100%


#1

Hi. My team uses references within Testrail to link test cases to work items in Visual Studio to show how many test cases cover which requirement. Which is great, but I’m wondering if there is any way to do this at a project level? My issue is that by logging the reference against the test case, my report is only telling me which items are covered. But for our test design I need to know which references we haven’t.

Any answers out there? TIA!

Matt


#2

Hi @mattandrews310,

It is possible to generate a report which only contains “Test cases without references”. To set that up start a new “Coverage for References” report. Under Report Options, select the Test Cases tab and uncheck “Test Cases with references” leaving the other aforementioned option. This will cover all test cases within the project and should provide what you need.


#3

Thanks. I know you can create a report which has test cases without references, but I want to know which references don’t have test cases. Is there any way to do this?


#4

Hey @mattandrews310

You’ll probably need to use the Visual Studio Team Services or server API to query project work items to then compare against existing TestRail test cases. I’m working on something similar with Jira and custom reports. Unfortunately there’s not much documentation on TestRail helper utilities, classes, etc to support custom reports.


#5

Thanks for the clarification and thank you @jsamsel for the additional info. I see what you mean, TestRail wouldn’t be able to do this as it wouldn’t be aware of any other references outside of the platform. What you could do instead is export the test cases as CSV or Excel. Selecting only the columns you need (e.g. ID, Title and References). Then do some comparison with an export of the same from your other product.

You could also script this as an API call but of course would require more development work and is ultimately up to you to determine how much time to put into it.


#6

Hi, I am new to TestRail and have some concerns about the “Coverage for Reference” report
So, I have a test suite in which 18 test cases do not have any reference and i have run a couple of test runs for partial test cases
When I created the Coverage for references report, i included all references in that particular test suite.

My query is:
The bar graph generated, shows the no. of test cases having no coverage and some having coverage where as the pie chart shows 100% coverage which is actually not the case.
Refer, attached screenshot

Can someone help me in understanding the graph/report generated?