Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Initial Impressions


#1

I’ve done some evaluation on TestRail today with a colleague and we’re both very pleased with the product. Here are a couple of our thoughts:

  1. It would be nice if we could customise the reports further (e.g. cover page, headers, etc.) - I think this may be on your roadmap already.

  2. The markup for numbered lists isn’t obvious (I’m used to using ‘#’ in JIRA, now I have to use ‘1.’)

  3. The “link” in a Test Case (reference [C2]) with a precondition of [C1] points to the original Test Case [C1]. But when the Test Case [C2] appears in a test run [T45], the precondition link should point to the instance of the test case within the run (i.e. [T44]).

  4. It would be nice if test cases with preconditions couldn’t be complete until the precondition test was passed.

  5. How much scalability testing has been done on systems with, say, 300 users and 100,000 test cases?

  6. Any plans to provide integration with Atlassian’s Crowd (Single Sign-On)? Management of users and default groups would be nice (as well as Admins).

  7. I’ve read some topics on this forum regarding being able to hide/minimize projects, etc. This gets my vote.

  8. On the whole, TestRail looks like a great product. There’s a good chance that we’ll use it if we can get a few of these minor things looked at. :slight_smile:


#2

Hello,

Thanks for your message. Great to hear that you are pleased with TestRail. Let me answer your requests one by one:

  1. Customize reports: as you mentioned, providing better reporting capabilities is planned and will likely be one of the first things we will work on early next year.

  2. Formatting: The Markdown formatting could certainly be improved (we also would like to support some additional formatting options in the future). We are also considering migrating the text fields to a rich-text editor at some point (HTML-based).

  3. Case/test links: This was the idea when we first added support for links, but we didn’t implement this back then for a technical reason. I will make sure that we will look into it again and have added it to our feature request list.

  4. Links: We might add this, but it’s more likely that we would add a Dependencies field instead of using the Preconditions links for this (as the field can be removed/customized). We have this on our feature request list already and we had requests for this before.

  5. Scalability: We have customers who use TestRail with large teams and with 10.000s of test cases. Using TestRail with a large number of test cases is supported, but you will need to separate the test cases into multiple suites (this is a very good idea in general anyway for various reasons). I would be happy to discuss your scalability concerns via email if you like as I would need some additional details in order to explain how TestRail should be setup for such an environment (dg at gurock dot com).

  6. SSO: TestRail can be integrated with almost any single sign-on system by creating a small PHP script and I expect this wouldn’t be difficult to do for Atlassian Crowd. We would be happy to help with this if needed.

  7. Closing/minimizing projects: agreed, I’m sure we will have this at some point.

Feel free to email us/post a message if there’s anything else, we are always happy to help. :slight_smile:

Thanks,
Dennis