Subscribe Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security. Subscribe

Covergage for References always == 100%?

I try to make something like test coverage matrix: requirements/tests (references/test-cases)
So i generate report: Coverage for References and see Covered==100%

  1. How Covered can be less than 100% if we can add reference in existed test-case and every reference immediately stay covered after linking?
  2. Is there any ways to add references first, without existed test case for make test cases and link them later?

Hi @ittermit,
not sure what you try to reach and I’m not talking about the metric…

You can have test cases in TestRail without a reference.
You can have Issues in Jira (or whatever Tool you are using) without having a reference on it.

The Jira-plugin doesn’t store the reference itself, it looks in TestRail for it’s issue id and in case shows a reference to TestRail. The info is just stored in TestRail.

I try to make classical test coverage matrix:
requirements (references) / tests (test cases in TestRail)
In report “Coverage for References” - reference coverage now always equals 100%
, because as i understand i can add reference to requirement (jira issue) only in existed test-case and this reference covering immediately.

Final process that i try to realize:

  1. Add requirements (references - new requirements without existed test-cases)
  2. Add test-cases for not covered requirements (references)
  3. Periodically monitoring coverage references/test-cases

Hi @ittermit,

this is a logical problem with the TestRail Plugin in JIRA.
Usually you would need to create a report in Jira, fetching the coverage of your issues with test cases. As mentioned before, this is not possible, because there is to data for references stored in Jira.

If you report in TestRail you ask for cases referring issues. You probably will find cases without a reference. Will say, which cases were created without a concrete requirement. Nice to have, but not your requirement.

I created this metric by exporting a list of jira issues and a list of cases with references and map the lists to get the metric. Could be solved with Excel. Honestly I did so by using the JIRA and TestRail API.

Ok. Final question:
In TestRail reports i see default report template: “Coverage for References”
And in this report i see graphical metric “Reference Coverage” (screenshot in previous message).
Covered == 100%, Not Covered == 0%
What should i do for get “Covered” <100% and “Not Covered” >0% ? :grin:

Do you have cases withou a reference?

Sure, i do have test-cases without references

1197 with references
106 w/o references
Left bubble (graph) not about test-cases, but about references (with coverage by test-cases)

Sorry, by this I have no idea anymore. Assumed both diagrams really base on the same data. :innocent:

Just found this, probably you have to enrich the report…

1 Like

Thanks a lot! :wink: