Join 34,000+ subscribers and receive articles from our
blog about software quality, testing, QA and security.

[UI Script][Plugin][PoC] TestRail plugins — working demo


#1

Overview

The article describes how 3rd party web-apps can be safely and seamlessly integrated into TestRail.

Preface

As you might know, I’ve been making some hopefully useful UI scripts for TestRail here lately. UI scripts are a very simple but powerful extension point because they are executed right in the browser, providing access to current context, which means they allow to shape and bend TestRail interface to enhance usability and make QA engineers’ lives easier.

However, so far it has been discouraged to write complex UI scripts that depend on TestRail’s markup or front-end code too much because those aren’t guaranteed to be stable. The advocated purpose of UI scripts was boiled down to performing some REST API calls (since those are stable) and small UI changes like adding/removing a button (which are easy to adjust as TestRail UI gets refactored). For numerous times I heard from the Gurocks that my next big UI script idea should wait until they come up with better extension APIs and provide a plugin framework, while I was pretty sure the existing functionality could accommodate larger plugins already.

Finally, after facing some disbelief, I’m pitching my concept to you :yum:

Details

As described on the picture, the plugin is a web application that may be hosted anywhere, be it a cloud managed by 3rd party, or your in-house instance. The only important bit is that if your TestRail instance uses SSL (e.g. hosted on TestRail cloud), the plugin must use SSL too. Add a back-end, and the plugin may have its own server-side logic and a database — yay!

The plugin is embedded into TestRail page using an <iframe>. This may sound like an old-school approach to WWW, but it’s not — for instance, Facebook and Disqus still use iframes for their plugins. Unlike UI scripts, iframes load their content in a “sandbox”, therefore not interfering with host document’s markup, styles, or scripts. This allows to off-load all the UI heavy lifting to the iframe while the UI script is doing exactly what’s advocated — figuratively speaking, just adding a button! Perfect!

To establish two-way communication between TestRail and the plugin, Cross-document messaging (XDM) is used. This technology is not new, and is suprisingly well supported. With proper XDM implementation, the UI script loaded in TestRail has full control over what data to expose to a requesting side and what data to accept back.

For UI scripts this means a few new responsibilities:

  • managing plugin lifecycle: ensuring proper handshake and termination;
  • handling XDM events and exchanging data with the plugin;
  • ensuring security by limiting the data sent to requesting sides and ignoring potentially malicious XDM events;
  • for plugins that require to identify TestRail installation and/or user (e.g. for licensed plugins) — also storing and providing unique IDs per customer.

Security considerations

The prospect of having 3rd party web documents loaded in your browser window may feel concerning. However, as said before, in a proper XDM implementation the UI script effectively limits what an iframe can access. Both the UI script and the plugin code are client-side JavaScript, meaning that you or any of your front-end devs can review what data gets exposed and make sure there’s no malicious code.

If a plugin needs to process your data on its back-end, their operators are expected to provide you a User Agreement and Privacy Policy, which you sign if agreed, just like with any SaaS. Or they may offer on-premises licenses as well.

Demo

To support my claims I have prepared a small proof-of-concept. In this demo, the plugin creates an “Add Test Case” form based on your setup (custom fields, priorities, milestones etc. fetched via REST API), which allows you to create a new case.

The plugin is hosted on GitHub Pages: https://actinarium.github.io/testrail-plugin-demo/.
GitHub repository is here.

To try the demo yourself use this UI script.

:warning: Because of this issue, if you have Date fields and your TestRail’s locale is different from English (US), adjust dateFormat variable on line 10.

If you are lucky enough, you should successfully run the plugin and create a test case.
If not, here’s how it should look.

As this proof-of-concept was put together in a few days, it contains a lot of spaghetti code, especially the XDM part. The next step would be to completely refactor it and provide a cleaner API. However, you can still look at the XDM logic around lines 83–150.

On a side note, the UI script doesn’t use TestRail dialogs — it just mimics their look and feel with custom markup and styles.

Possibilities and limitations

At this point you might say, ‘OK, cool, but I could create test cases before. What else can plugins do?’

In the demo you have already seen a custom form constructed based solely upon the REST API data. This is just a start. Imagine alternative feature-packed test plan editors, maybe even visual ones like this, on-the-fly diagramming and mock-up sketching tools, custom visualizations, and more! Think of your favorite Confluence and JIRA plugins — maybe there is a place for them in TestRail.

There are some upsetting limitations though, mostly coming from limited REST API functionality. From the infamous lack of Attachments API to the issues with existing endpoints, and essentially having no REST coverage for Todos, Reports, Users, Groups, Roles, and Permissions at all.

On the bright side, there’s already plenty to experiment with.

That was quite a read. But you aren’t on Gurock team! Why spending time on this?

First of all, it’s fun. It’s also a good practice for a wannabe front-end developer. Plus, some karma points for sharing my findings with others.

Secondly, even though it’s not my topmost priority at the moment, I have a few plugin ideas myself. Nowadays that’s pretty hard to discover a new niche, especially in enterprise sector. Being the author of the first TestRail plugin (technically the demo counts as a plugin, right?) and opportunity discoverer is kinda a big deal :slight_smile:

What’s next?

First of all, I need to accomplish my primary task (make and publish an Android app) really badly.
After that, I guess, I’ll dive into shaping this PoC into a more solid plugin framework foundation.

Of course, the suggested concept will only work well with the plugins that are self-contained web-apps, which are displayed full-screen, or as a dialog, or as a block on a page. So in addition to aiming for Web UI :left_right_arrow: REST feature parity, obviously there’s still a need for a “native” plugin framework that will allow integrating into TestRail pages without worrying about compatibility.


Thanks for reading, and waiting for your feedback in the comments! :slight_smile:

With best regards,
Paul “Actine” Danyliuk