I am looking to set up an issue/bug tracker for our small test team that integrates into TestRail. I know Redmine is supported so I looked online for a hosted Redmine solution, and ended up horsing around with PlanIO. It’s pretty sweet. But it’s not clear to me that it is ‘pure’ Redmine, so I don’t know if it’s supported with TestRail. Does anyone have any insight into this? If it is not, can someone recommend a hosted Redmine provider?
I am going to partially answer my own question here in case anyone else is looking. I received email response from the Planio team, and they said this:
Since Planio is based on Redmine with some extensions and improvements, all integrations targeting Redmine should work out-of-the-box without change. We always strive to retain this compatibility in order to be part of a healthy eco system. As such, although we have no experience with it, I’m sure that TestRail will work just fine with Planio.
With a little configuration in the Admin…Integration section of TestRail I was able to point TestRail to my instance of Planio using the Redmine setting in the dropdown. And I can successfully ‘push’ a new issue from TEstRail into Planio.
HOWEVER - I’m not convinced that the integration is everything that I want. The main issue seems to be that once an issue is created, any changes to the test case in TestRail do not get propagated to Planio after the initial issue has been entered. Likewise, any changes made in Planio don’t seem to get propagated back to TestRail. Using the Push button again just results in a new issue, which is not what I want. I am going to play with it for a while and see what I can discover, and I will report back here.
Thanks for your posting. Yes, PlanIO works with our Redmine integration and this allows you to push & view Redmine issues directly from TestRail’s user interface. To update issues in Redmine (e.g. closing them or setting them to “fixed”), you would switch to Redmine and just change the status or add additional details and pushing issues is mostly meant for reporting bugs & reports.
Regarding seeing changes in Redmine in TestRail:
The Defects tabs show the current live status for all issues of a run/plan/milestone/case:
You can also hover over a Redmine issue ID everywhere in TestRail and this loads the current status and details in a popup windows:
I hope this helps!
Thanks Tobias. I think what I was really looking for was whenever a bug was set to Fixed status, Test Rail would automatically update all tests that are blocked or failed by that issue into Retest status. This is a big timesaver. It sounds as if this is not possible with Redmine or Planio.
I’m currently playing with Jira integration - is this feature available with Jira?
Changing an issue status in Redmine wouldn’t update the test status in TestRail and this is the case for all integrations. TestRail doesn’t do this automatically and we think a manual approach has many advantages. For example, only because an issue changed in Redmine doesn’t mean that the testers can already retest a test and they may not have access to the latest version. It’s also a good idea to add feedback or a comment or similar to a test when changing the status and this is also only possible with a manual approach.
I hope this helps!
Thanks for the response. I have seen the same response to others asking essentially the same question here. I do disagree however. The manual approach is very cumbersome and error-prone for anything more than a handful of testcases. If there are many testcases in the Fail status due to a specific issue, having to set them all manually back to Retest is tiresome. Perhaps we could do this with a filter?
I understand your view about “because an issue changed in Redmine doesn’t mean that the testers can already retest a test and they may not have access to the latest version.” But my view is that if the bug fix and code delivery workflow is well defined and controlled, this may not not be an issue.
Please consider this feature in an upcoming release. It would be best implemented with a control that allows the feature to be set on or off, so that people who do NOT desire it would not have to deal with it.
Best regards, Jeff
Thanks for your feedback. Yes, we do already have this on our feature request list for teams that prefer a more automated approach. Not all issue trackers would support this though and this requires some kind of notification or plugin architecture to detect status changes. JIRA has support for this with webhooks but most other issue trackers probably don’t. It’s planned to look into this for a future version, happy to add another vote.