Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Simple requirements reporting to allow test/defect tracebility


#1

Enhancement request:
Add a very basic requirements management feature to Projects to allow linking of test cases, and thus defects to requirements. The requirements need only be as simple as an description/name, ID field and/or a URL field to link to where the full documentation for the requirement is stored.

I couldn’t find a way to do this through customisations.

This would then need a traceability matrix report(s).

  • A matrix of Requirements to Test cases (columns and rows with a tick where a test helps verify a requirement)
  • A simple Requirements to Defects report (For each requirement, a list of associated defects)

With the matrix I suggest rotating each X coordinate heading 90 degrees to reduce the overall space required to print the report.

With the Defects report I suggest listing the requirements without defects separately, so that the problem areas of the software under test are grouped together for easy assessment of known risks.


#2

Hi Dave!

TestRail doesn’t have a built-in requirement management module, because we found that most teams prefer to manage their requirements/user stories/feature requests within their issue tracker such as JIRA, wiki software or dedicated requirement management tools. That said, you can link test cases to externally stored requirements via TestRail’s References field. You can learn more about the References field here:

http://docs.gurock.com/testrail-integration/references-introduction
http://docs.gurock.com/testrail-integration/references-urls

TestRail also comes with various reports for requirements and other metrics. For example, you can generate traceability and coverage reports for requirements or defects from the Reports tab (such as Cases > Coverage for References or Defects > Summary for References).

I hope this helps!

Cheers,
Tobias


#3

Does this allow one test to meet multiple requirements? Is it many to many?


#4

Hi Dave,

Yes! You can enter multiple references/requirements to a test case (just comma-separate them) and also link references/requirements to multiple cases. The reports were also designed for this. For example, in the screenshot above you can see that requirement RF-2 is linked to two different test cases (C9192 and C9193).

I hope this helps!

Cheers,
Tobias


#5

Thanks Tobias,

What I couldn’t gather from the screenshot is if one test could have multiple requirements.

Cheers


#6

I’d really like to see traceability reported as an X by Y matrix, where coverage is represented by a shaded box. The shaded box and the test name could shaded with the latest result to provide a lot of information at a glance.
Any requirements without coverage should also be highlighted.
Example attached:


#7

Hi Dave!

Have you already taken a look at the Results > Comparison for References report? As you can have multiple runs/plans for the same suite/case repository active at the same time, this report would generate a matrix for the references/requirements/cases and test results. The Latest column would also include the coverage across all test runs and this would represent the most recent test result (per test).

I hope this helps!

Cheers,
Tobias


#8

I have seen that report. It’s close, but still not as quick and simple to assess coverage as the X/Y matrix. The matrix provides a wealth of information in a concise report.

I notice that the References or Test IDs columns are not sorted in your report. Even if they were sorted or sortable it would still be less efficient for assessing coverage than the X/Y matrix.


#9

Hi Dave,

Thanks for your reply! A basic X/Y matrix would only work for the simplest use case with a single test run but I understand that this view would be more compact. The report in the screenshot is actually sorted by reference ID (as would the other reference/requirement-based reports).

Thanks again!

Cheers,
Tobias


#10

Ah yeas, sorted by Ref ID. I was looking at an old screenshot and not my new one when I made that comment and it must be something that was fixed along the way.

As for the results in the X/Y matrix, the latest results per test case would certainly be my preference. That said, if an X/Y matrix didn’t show any result data, the information would provide valuable coverage information in a quick to consume format that I’m sure many would appreciate.


#11

Happy to look into this and I’m sure that would be great to have. Thanks again for the suggestion!

Cheers,
Tobias


#12

Hi, I’m currently looking at TestRail as a possible testing tool to replace our current one. The Test Manager requires a test matrix in the exact style as suggested by davef. Was any progress ever made on Implementing something like this?

Regards,
Dan


#13

Hi, also interested if this was progressed?


#14

Hi all,

Thanks for your feedback! We still have plans to review adding a more simple X/Y report format, however we don’t have any timeframes/ETA to provide for this. In the meantime, we would recommend using the existing coverage/comparison reports as these would already provide the coverage details as mentioned by Tobias previously. Hope this helps!

Regards,
Marco