Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.

[Automation] Jenkins CI automation integration


Hi there,

The team at SimplyMeasured has developed and published a Jenkins plugin to automatically submit JUnit-style automated test results to TestRail. You can find the source code for the plugin here:

Thanks for publishing this code, I’m sure it’s useful for other customers as well.


I’m one of the developers on the team that created the plugin. We welcome pull requests and issues on the GitHub repository.

Currently, it creates a new Test Run within the specified project for every build. We have a suite of integration tests in Ruby/RSpec for each project, and we use the ci_reporter gem to convert the RSpec results to JUnit-style results, and then upload them into our TestRail instance using this plugin.


Rob Roland
Lead Software Engineer
Simply Measured, Inc.


Thanks Rob, it’s appreciated!


It looks like the new version of Jenkins (1.581) has broken the testrail plugin. Are there any plans to update the plugin to work with the new version of jenkins?

Here’s the error that I get:

ERROR: Publisher testrail.testrail.TestRailNotifier aborted due to exception
com.fasterxml.jackson.databind.JsonMappingException: (was java.lang.NullPointerException) (through reference chain: testrail.testrail.TestRailObjects.Results[“results”])
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(
at com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(
at com.fasterxml.jackson.databind.ObjectMapper._configAndWriteValue(
at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsString(
at testrail.testrail.TestRailClient.addResultsForCases(
at testrail.testrail.TestRailNotifier.perform(
at hudson.tasks.BuildStepMonitor$1.perform(
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(
at hudson.model.Build$BuildExecution.post2(
at hudson.model.AbstractBuild$
at hudson.model.Run.execute(
at hudson.model.ResourceController.execute(
Caused by: java.lang.NullPointerException
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(
… 15 more
Finished: FAILURE



Tobias from TestRail here. The Jenkins integration was created/is maintained by a customer (please see above) and we unfortunately don’t have control over/experience with this particular plugin. You can try to contact the author via GitHub.



There is an issue with the Testrail plugin after every Jenkins upgrade/downgrade.

Here are the steps to fix it:
From JENKINS_HOME directory, delete testrail.testrail.TestRailNotifier.xml file.
Restart Jenkins.
Go to Jenkins Management page --> Configure System. Add Jenkins server configuration credentials as it will be wiped out.
You should be good now.

Hope it helps.


Thanks for your help with this, venkat, that’s appreciated!



[quote=dgurock]Hi there,

The team at SimplyMeasured has developed and published a Jenkins plugin to automatically submit JUnit-style automated test results to TestRail. You can find the source code for the plugin here:

Thanks for publishing this code, I’m sure it’s useful for other customers as well.[/quote]

Hi there,

The URL to Simply Measured github plugin returns a 404 error. Could someone please post the updated URL?



Hello Phil,

I believe the Simply Measured plugin/project is now longer publicly available I’m afraid. You could try to contact the author of the plugin.



Hello Tobias,

Thanks for the reply. One of my collegues actually managed to find a fork of the original Simply Measures plugin project.

You can find it here;

We’ve installed it and it works! V Happy:D



That’s great, Phil, and thanks for sharing this!




We built and deployed the plugin in a sandbox machine and when trying to push results I’m getting the following -

Uploading results to TestRail.
Failed to add results to TestRail.
status: 400
body :
{“error”:“Field :results cannot be empty (one result is required)”}
Build step ‘Publish test results to TestRail’ marked build as failure
Finished: FAILURE

Any idea what we might miss here? Obviously there’s something wrong with missing results but the input is a JUnit xml format file produced by SoapUI.



Hi acelow

The plugin expects to find a file called “results.xml” with your junit test results in them. You should point the Jenkins plugin to this file using the “Test Report XMLs” plugin variable. This location is relative to where the test is executed. For example if your tests execute in /mnt/data2/jenkins/workspace/fancytest then your XML report variable should be just “results.xml”.

If you have created the result.xml file deeper down then the relative address will be required. For example for a path underneath where your tests are executed, let’s say in testrestults, it should be like this;


Hope this helps.



A nice option for this would be the ability to create a test run and then kick off a Jenkins job from there (clicking a button), with results coming back to the source test run. I had a Rube Goldberg way of doing this directly from my own codebase, but it was very ugly and maintenance intensive and I have since taken another path.


Thanks sysmech

I’m currently POCing the plugin so my results.xml is “floating” in some arbitrary location and it’s not related to a specific job.
Does it have to be in a Jenkins workspace? Cant I provide an absolute path the the “Test Report XMLs” plugin variable?


Hi acelow

As far as we have tested, the results.xml file has to be in the job workspace area. We are using the plugin “as is” at the moment, however in pulling it apart find out what we had to do with the results we noticed that there is already code in there to handle multiple unnamed result files: the code of which appears not to be implemented.

We’ll be changing the plugin later in the year to make it more flexible. Currently it creates a test run each time it is executed and we wan’t to be able to update an existing run if one already exists.


How are you managing run of automated tests and results through api

Hi Sysmech,

updating an existing test run with each subsequent run will lose a case history. Don’t you want to keep track of it?
What we’re not able to confirm now (because of the restriction above) is will the plugin create new cases if the ones that ran are not already in testRail or do we need to manually create them in TR before running the tests?
Basically the use case is to gather our Unit tests and other automations results also in TR but we don’t want to write them down explicitly in TR but to have them updated in TR according to the returned XML.

Need a sample xml or template to push the testresults from jenkins to testrails

Hi Sysmech/Acelow,

I am also getting the same error, as mentioned by Acelow, even when i place the results.xml file insde the jenkins workspace folder.
However, when i place the xml file inside the jobs folder ( instead of the workspace folder) , i get the following error:
Failed to add results to TestRail.
status: 400
body :
{“error”:“No test results or comments so far.”}
Build step ‘Publish test results to TestRail’ marked build as failure
Finished: FAILURE

I am not sure if i am missing anything in the Junit style xml report format since i am creating the xml myself.

If any of you can help with the specific format that the TestRail interprets ?
and also, for eg, Jenkins is Building in workspace “D:\testrail-jenkins-plugin-master\work\workspace\TestRail”. So should i place the xml inside the TestRail folder ?



Hi. Where can I find this variable?


Hello sysmech,

I grabbed the plugin from the link you provided and I downloaded the zip file and changed the extension from zip to hpi and then uploaded the plugin via UI.

I see the following :
javax.servlet.ServletException: java.lang.NullPointerException
at hudson.PluginManager.doUploadPlugin(
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.kohsuke.stapler.Function$InstanceFunction.invoke(
at org.kohsuke.stapler.Function.bindAndInvoke(
at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(
at org.kohsuke.stapler.MetaClass$1.doDispatch(
at org.kohsuke.stapler.NameBasedDispatcher.dispatch(
at org.kohsuke.stapler.Stapler.tryInvoke(
at org.kohsuke.stapler.Stapler.invoke(
at org.kohsuke.stapler.MetaClass$3.doDispatch(
at org.kohsuke.stapler.NameBasedDispatcher.dispatch(
at org.kohsuke.stapler.Stapler.tryInvoke(

Please help!!