Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

TestRail 5.7 Released

#1

We’re pleased to announce that TestRail 5.7 is now available and has been released to all users of our cloud infrastructure. With this release, we’re also proud to present our new TestRail Enterprise product tier, with premium support, a faster API, and features geared towards enterprise teams: SAML 2.0 Single Sign-On support, TestRail Auditing capabilities, Configurable Backups and On-Demand Restorations (for cloud users).

Long time users of TestRail will be especially pleased to learn that this release contains many long requested and awaited for features including Reporting and Attachment API endpoints, and some tweaks to how editing test results works along with an additional Modify permission in the user roles area.

Below is a summary of the changes in 5.7. For more details, refer to our 5.7 release blog post.

SAML 2.0 Single Sign-On Support

TestRail’s SSO feature allows administrators to integrate TestRail with their preferred SSO identity provider (IDP) using the SAML 2.0 protocol. In practice, this means that the management of users can be streamlined, by creating them once in the IDP and then providing them with access to whichever applications are required by the user – TestRail, in this case. Once configured, TestRail SSO will automatically authenticate new users that have been authorized to use TestRail in the IDP. This means that testers can log in once, and get on with their work. And administrators no longer need to concern themselves with the day to day management of users in TestRail. Instructions for configuring and using SSO can be found in the TestRail documentation.

Single Sign-On is a TestRail Enterprise feature. Standard TestRail customers will not be able to use SSO functionality unless they have upgraded their server instance with an Enterprise license, or their cloud instance with an Enterprise subscription. We’ll announce the availability of TestRail Enterprise edition for purchase very soon.

TestRail Audit Log

TestRail 5.7’s new audit logging system helps administrators track changes across the various entities within their TestRail instance. With audit logging enabled and the audit level set to high, administrators will be able to track every entity that’s created, updated or deleted in their installation. Instructions for configuring and using auditing can be found in the TestRail documentation.

Auditing is a TestRail Enterprise feature. Standard TestRail customers will not be able to use auditing functionality unless they have upgraded their server instance with an Enterprise license, or their cloud instance with an Enterprise subscription. We’ll announce the availability of TestRail Enterprise edition for purchase very soon.

Configurable Backup Time and On-Demand restoration

If you’re using a cloud instance, your TestRail 5.7 administration console now enables you to specify at which time (approximately) your daily backup should be taken, so as to minimize the potential for disruption to your team while the backup process is taking place. Administrators can specify the Preferred Backup Time window, see when the last backup was completed, and in an emergency – restore the last backup taken, overwriting any subsequent changes. Instructions for configuring backup times and initiating a restore can be found in the TestRail documentation.

Configurable Backup and Restoration is a TestRail Enterprise feature. Standard TestRail customers will not be able to use configurable backup and restore functionality unless they have upgraded their server instance with an Enterprise license, or their cloud instance with an Enterprise subscription. We’ll announce the availability of TestRail Enterprise edition for purchase very soon.

API Attachments
In TestRail 5.7 we’ve provided some new API endpoints that enable API users to identify and export existing attachments, upload new ones and delete attachments where needed. Attachments can be identified via existing get_result(s) endpoints. Two new endpoints have been provided to upload, export and delete attachments. Full instructions for using the new endpoints can be found in the Attachments API documentation.

API Reporting
We’ve also provided API endpoints that enable API users to identify and run reports, so long as those reports are identified as API template reports. Before you can start running reports via the API though, you need to make sure your reports have been marked as accessible via the API in the report settings. Once created, reports accessible to the API are visible in the new API Templates section.

Two new endpoints have been created to handle API accessible reports. Read full instructions for use in the Reports API documentation.

API Filtering
We’ve extended the get_cases API endpoint so it supports filtering, to ensure that teams can get back only the cases they want. Three filters can be applied, individually, or combined. Read the Cases API documentation for more details.

Modify Test Results Enhancements
By popular request, we’ve provided testers with the capability to edit test result statuses should they need to. We’ve also provided an additional Modify permission that can be enabled by administrators, providing users with the permission with the capability to modify test results for other users (while still honoring the existing Editing Test Results UI setting), should they need to. Read the How-To guide for more information.

New Jira Cloud Plugin
We’ve provided a new Jira Cloud plugin which enables TestRail users integrating with Jira Cloud to use their email addresses and API tokens to authenticate the integration, instead of the existing username and password method. This does require some actions after upgrading, so please read the guide here for more information.

Upgrading to TestRail 5.7
Upgrading to TestRail 5.7 is easy. We’ve included all the required details below to get TestRail 5.7 up and running, depending on the edition you use:

  • TestRail Enterprise : TestRail Enterprise licenses will be available soon. Please contact us to learn more.
  • TestRail Cloud : Most accounts of TestRail Cloud already use the new version. Any accounts which don’t already have it can expect the upgrade over the next few days or so.
  • TestRail Server (licensed) : You can download the latest version or renew your support plan from your customer portal account. Please remember to backup your database before updating!
  • TestRail Server (trial) : Please contact us to upgrade your download trial.
  • New User : Want to try TestRail? Get a free trial.
Feature Request: SAML SSO via OKTA
#2

So good being able to change the Test Results status now. Thanks for listening and implementing this feature - makes life a lot easier for us.
:love_you_gesture::love_you_gesture:

#3

The new enterprise tier feels like a bit of a slap in the face for those who have been waiting patiently for SSO. There was no mention (that I am aware of) of a new licensing model in any of the discussions around this functionality.

We - like most companies - have a FY budget; things like licenses need to be planned well in advance. It’s great that you’ve finally delivered the functionality in this release, but it’s unlikely we are going to be able to use it until our current license expires, assuming we can afford the price difference.

Can you please explain why the enterprise edition would have a faster API? What does that mean exactly?

1 Like
#4

I also agree with you. We also did need to have the budget plan in advanced. And you also will be “surprised” when you renew this year’s license. The server license renew fee will has gone up by 40% increase.:frowning: And their explanation is that they need to hire more employee, need more resource to handle user requests etc.

Actually, I can accept the price increase like x% every year to get better service from them. And the price is increasing stably, not “surprised”.

For the new TestRail enterprise plan, it also can be accepted if they did provide more really enterprise features and premium support like 7x24 hours urgent call/web-ex sharing/troubleshooting, fast user request response and so on.
But currently, we don’t feel the better service, feature request response. It is more like spinning off some features and selling them separately.

1 Like
#5

Hello, Glenn. We understand that you have questions about the new TestRail Enterprise product tier. We anticipate that additional information about the Enterprise edition, including features and pricing, will be announced very soon.

#6

Hello, Jeffrey. Additional information about the Enterprise edition will be announced very soon, including features, pricing, and details about priority support.

#7

Awesome to see new features roll out! I’m personally hesitant about the new Enterprise tier but will reserve judgment until more details come out.

Wanted to comment regarding the new ability to edit a test result:
This is a very welcome functionality change that we have been wishing for. However the options given in the Administration > Site Settings > User Interface are too limited, especially 3 days being the maximum set time before “unlimited”. For example the use case you give in your documentation (“Tests can be effectively paused to be picked up at a later time. For example, a test can be given a status of “In Progress” and then passed/failed later on.”) is very unlikely to be doable unless the setting is set to Unlimited. Ideally an admin should be able to set the time a test result can be edited for but even more options (i.e. 7/14/30 days) would solve this issue.
I’m sure I saw a feature request regarding this issue but can’t find it now.

Thanks
Ory

#8

OKAY. BTW, Configurable Backup Time & On-Demand Restoration is also available in Server edition, OR it is only for Cloud edition?

#9

Hello Jeffrey. Server admins already have the ability to manage the backup time and perform on-demand restoration on their own server; so this feature isn’t applicable for Server edition.

#10

I’ll answer my own question here, based on this link: https://blog.gurock.com/enterprise-edition/

The API is not faster, it is less rate-limited. It does not apply to TestRail Server customers.

The only thing TestRail Server customers get with their Enterprise license is SSO and auditing.

1 Like
#11

Is there any plan in the future to support OpenID Connect or OAuth?

#12

Do you think those worth the double increase in price? I surely don’t think so.

Feels like TestRail as a company just use the old customers to support developments of new features and dump them with the new Enterprise tier.

Really scummy business practice. I honestly regret the day I recommend TestRail to the managements.

1 Like
#13

I think the following features are also desired:

  1. Configuration management so data can be versioned and reverted if needed
  2. Multitenancy so each project can manage their own setup and access
  3. A customer facing support ticket tracking system to handle technical support and feature requests from TestRail customers