Join 34,000+ subscribers and receive articles from our
blog about software quality, testing, QA and security.

What's on your roadmap?


#1

#updated 24.07.2017#

Hi,

Apologies if my first post is a ‘moan’. The product is great, but has it’s limitations in enterprise settings. I’d like to know more about the future.

I have several clients that, based on evaluating a range of test case management products, selected Test Rail. Tools you were up against included Zephyr Add-On, Zephyr Enterprise, HP ALM, IBM CLM, QASyphony.

Test Rail’s comparative price, usability, API and not being overly focused on one context, such as web application testing, made it a clear choice, at the time.

However, as time has moved on, the combination of lack of features and the increasing availability of alternative test management tools, is creating an impetus to look elsewhere. Alternatives are also proving attractive due to the price breaks and licence flexibility beyond 250 users.

It would therefore be great to see a Roadmap showing some clear intent over the next 12 months of what the Test Rail team are prioritizing.

Perhaps it could be possible to set-up a public instance of JIRA so that licence holders can interact with you to vote on issues

Key features everyone seems to be asking for (ordered by urgency):

Enterprise features

  1. Fix Security problems
  2. Automating Site Administration tasks
  3. Flexible licensing options & price breaks beyond 250 e.g.:
    – ‘Free’ read-only access to Test Rail data
    – ‘Robot’ use of the API / multi-use licences
  4. Verbose system logging for analytics
  5. Delegating of Project User/Permission Administration without needing Site Administration

Specific features/bugs

  1. Logging of ‘deleted’ test cases e.g.: showing up in project activity feed
  2. Custom Fields to be automatically included in Reports
  3. Segregate reporting/dashboards across different test workflows e.g.:
    – Classic (Untested/Pass/Fail)
    – Exploratory (Not Started//Concluded)
  4. Pausing of tests without a need for a separate Test Run e.g.: In Progress state
  5. Confluence macros for creating test summaries (or perhaps examples or an SDK)
  6. Default creation of JIRA issues to specific project

My problem is that without visibility or news of when, even if within the next 12 months, the people I am working with will move to other tools.

Hope you can help

*A quick update. An associate of mine is reporting that his client has decided to freeze expansion and is deploying alternative strategy.


#2

We are in the same situation.
Tesrail is part of the list of possible choices. On the other hand without a clear roadmap we will not be able to commit.


#3

I have the same problem and have spoken to the some of their customer support team and gotten the run around. Their lack of interest in supporting the Enterprise’s customer needs has prompted me to look else were. The have a great tool but support is horrible. I plan to migrate to a new platform by the time my license expires .


#4

More than new features, I’m getting tired of waiting for some of the bugs and basic functionality that have been pending forever. Looks like they are either understaffed or simply don’t care. What are some of the other alternatives to Testrail you folks are looking at? I have a fast growing organization and need to make a determination soon if Testrail is going to be in our long term plans.


#5

What are some of the other alternatives to Testrail

Hi Sriram,

Thanks for adding weight to the discussion. I don’t want to use the article to promote the alternatives just yet but the best source, for me, is the Atlassian Market place. As by default most look for JIRA integration

https://marketplace.atlassian.com/search?query=test


#6

Hi all,

Thanks for your feedback! We still have plans to look into many of the features that have been listed here and throughout the rest of the forums as well as many other features that have been requested via email. Like many software companies (e.g. Atlassian/JIRA and others), we don’t publish our roadmap publicly as we prefer not to overpromise and underdeliver. We have an internal issue tracker that we use to track these feature requests and customer feedback, and in our experience having this information public just adds to the frustration when a feature isn’t implemented on the next version. The majority of new features/changes to TestRail have been based on customer feedback, and there are many factors (with the biggest factor being customer feedback) that goes into prioritizing various feature requests. In general, we work on many features in parallel, and when a feature is ready we implement it on the next version. Throughout this process priorities can change, and so we prefer not to specify any timeframes for this reason.

These forums provide a very small glimpse of the amount of customer feedback we receive for various features, and we always take note to add any new feedback/votes to a feature request. In some cases, feature requests are too specific for a specific team and it wouldn’t be a good fit for TestRail as it isn’t generic enough to be helpful for the majority of TestRail users/customers. That doesn’t mean we don’t plan to review a feature for a future version of TestRail, but that does help to give you an idea on how we prioritize these. We are also looking into new ways to retrieve various feedback (e.g. via anonymous surveys, etc) from more TestRail customers so we can have a better idea of how teams use TestRail and what could help improve productivity/etc.

Again, we thank you for your feedback and as I mentioned previously, the majority of new features/changes to TestRail have been based on customer feedback such as this. I hope this helps to clarify our position on providing timeframes/roadmaps.

Regards,
Marco


#7

Hi all,

Just to confirm Marco’s point, the reason we and most other software vendors (many of which have been listed here in the thread, including Atlassian) don’t have public roadmaps is that most vendors have changed to a pretty agile development methodology so most vendors wouldn’t want to commit to specific feature timelines a year or longer in advance. I believe this is also the honest way to work with customers, as we wouldn’t want to promise any specific feature timeline in advance that we might not be able to keep as a different feature or release theme got a higher priority. I believe this is a big reason most vendors have the same policy we have. If you compare TestRail’s release history with other test management tools, TestRail has been one of the most actively developed products in the market with new free productivity upgrades regularly.

One additional thing I just wanted to address here, as I think it’s an important one, was the question about “Fix Security problems”. Please note that there aren’t any known (to us) security issues with TestRail’s latest version. I assume this request was more about requested security features such as SAML integration (which we plan to review in a future update), but we don’t have any open/active security issues and we have standard ways to report security issues (found here), and TestRail has a great security track record.

Please also note that TestRail is used by many of the largest QA/testing teams without issues for years, e.g. as seen here, and we understand that additional features for larger teams are always useful, and they are on our feature request list.

I hope this clarifies why we and many other software vendors use this release and feature approach!