Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Submit remote test automation without specifying tests

I was giving consideration to how our users could execute a remote suite of tests based on test configuration parameters such as a templateId. Here’s what I was thinking:

User creates a plan and selects a RemoteTestTemplateId configuration. The user does not filter for any tests because they want to rely on the remote system to run and submit results back to TestRail. I know I have to eventually link up with an existing case in later steps via add_result_for_case.

User opens the test run and clicks the custom “Start Run” button which sends the TestRunId and RemoteTestTemplateId to the remote system.

The remote system uses the RemoteTestTemplateId to execute a series of tests. As each test completes, we make a call to TestRail to find the matching Case and use that to log a Result to the TestRunId via add_result_for_case.

While attempting this workflow, I was somewhat blocked by the fact that your TestPlan wants the TestSuite to have at least one test. Adding an arbitrary test would be an ugly workaround that I wouldn’t ask of our users.

Is there another way to accomplish a workflow like this without having to specify at least one test?

Hi Alex,

Thanks for the post! Currently, there is not a workaround to being able to create an empty test run. We are happy to consider this ability for a future release of TestRail and I will add your vote to this feature request.

In the meantime, you may be able to trigger your automation run outside of TestRail and then create the test plan or run once you have executed tests. This would allow you to create your runs as needed along with the test cases being tested.

Alternatively, within TestRail’s UI, the only workaround would be selecting at least 1 arbitrary test case, or using the ‘include all’ option. Once the run is created, you can later update it to include only the test cases needed, which will then remove the extra tests. Since your automation appears to already update the run based on the tests being used, using ‘include_all’ may be the easiest option since the updating work is already done for you.

Hope this helps!
Jon

Thanks for your response. I’ll look into alternative methods.