Join 34,000+ subscribers and receive articles from our
blog about software quality, testing, QA and security.

Clarification regarding Jira Testrail integration


#1

Guys, I really want to make Testrail work. Please help me here:

  • JIRA issue should refer to a test run, and the test run should include whatever test cases it needs. (If I have a feature got modified 40 time, I do not want to see the affected test case with 40 reference link, and have to manually clean them up.) Given the nature of jira story or epic is to require a behaviour change, and test cases ultimately represent that request, there should be a clean & flexible decoupling between the test case & jira issue/epic, which is the test runs.)
  • JIRA issue Test Result should display a test run result, and not all the test run for the test case. (I may have 40 test runs referred the same test case, and I do not want to see 40 entries in the Jira Issue Test Result)

Please tell me I have configure it incorrectly and I am possible to do the above.

cheers


#2

Any support from the folks working for Testrails?


#3

Hi there!

Thanks for your posting. I think there’s a bit of a misunderstanding of how the integration works and how test cases are usually used and we haven’t seen a similar request before.

You would usually link your test cases to user stories in JIRA via the References field (not specific runs). If you have a user story you might have one or multiple test cases linked to them. You would however not usually need to duplicate your test cases in TestRail. So you would always see the same test cases linked to the JIRA issue. It sounds like you might duplicate your test cases regularly? This wouldn’t be the correct way to manage your test cases in TestRail as this wouldn’t be needed. You would just organize your test cases in TestRail by features and start runs for new testing phases/releases/builds and reuse your existing test cases, not duplicate them.

So in JIRA you would see the referenced test cases in the sidebar. And the displayed test results for the issue would include both the results for the linked cases, as well as any results linked to this issue via the Defects field (though this would usually be more relevant for bug report issues, not user stories). As all test results for the cases are relevant (as they are related to this user story, i.e. feature), they would be helpful in understanding the quality of the user story’s implementation.

I hope this helps!


Testrail to Jira integration - referencing a test case to multiple stories and getting the correct results
#4

I understand how current integration work. And I am expressing your current implementation
is not flexible enough and conceptually causing maintenance hell.

Given:

  • Test cases expresses the app’s behavior, it lives through out the life of the app
  • One behavior may be alter by many stories through out time
  • Jira issue represents the work to modify that behavior, it is valid until the end of the effort

if Test cases & jira issues are linked directly:

  • You will have meaningless jira issue key trailing every test cases, visa versa
  • maintenance & confusion is gonna get worse if we start to split & merge jira stories & test case (which is not avoidable)
  • ps, the direct linkage also prevent the flexibility of splitting & merging jira issues

The correct linkage btw jira issue & test cases should be through test runs, given:

  • story’s life ends when it is verifies by a test run
  • test cases are living spec of how the app shall behave (should not be duplicated)
  • development shall be driven from test case (behavioral spec)

I really want to use the app, that is why i am expressing my concern. Nowadays,
test management tools cannot just be a doc repository. It needs to facilitate behavior, share understanding & collective ownership btw product owners, quality engineers & developers.

I hope you guys are open enough to look into it.

cheers


#5

Hi Luk,

Thanks for the additional feedback! You can also link tests/results to issues per test run in addition to/instead of the References field on the test case level. This would require linking issues via the Defects instead. Although this field has a slightly different purpose originally (it’s commonly used to link results to bugs managed in JIRA), you could still use if you prefer to link issues on the run level instead.

Cheers,
Tobias