Join 34,000+ subscribers and receive articles from our
blog about software quality, testing, QA and security.

Upload automated_test_results.xml to testrail


#1

Hi fellow testers,

I have recently joined the forum and I would like to ask about uploading automation results to Testrail.

  • I execute some automation tests using phpunit and selenium.
  • The results are output to a results.xml file
  • I would like to upload this results file to testrail
  • Where do I start to read up about this?
  • Has anyone done this already and if so, could they give me some tips?

Thanks
Conor


#2

Hey Conor,

Thank you for joining the forum and thank you for posting your question. Test results may only be added to your tests by the following two methods:

  • Manually
  • Via API

Uploading results for your tests via XML is not supported at this time. If you would like to look over our API documentation, you can find that here:

http://docs.gurock.com/testrail-api2/start

I would recommend looking into this as you already mentioned you are using PHPUnit and Selenium. You could leverage the TestRail API to have Selenium automatically send results directly to TestRail.

If you have additional questions, please let me know.


#3

Hi Marty,

I have extracted the information I need to upload to Testrail from the XML file and placed it in a *.json file in the form:
{
“results”: [
{
‘case_id’: 1,
‘status_id’: 1,
‘comment’: “This test 06/03/2017”
}
]
}

I try to upload this file via the php code:
$uploadResults = ‘jsonFileToUploadResults.json’;
$result = $client->send_post(
‘add_result_for_case/25’,
$uploadResults
);

I will be uploading more than 1 result. Is the any examples of a bulk upload of result via PHP?
When I execute the above code, I get:
Fatal error: Uncaught exception ‘TestRailAPIException’ with message ‘TestRail API returned HTTP 400 (“Field :case_id is a required field.”)’ in C:\xampp\htdocs\conorTest\testrail-api-master\php\testrail.php:151
Stack trace:
#0 C:*\testrail-api-master\php\testrail.php(87): TestRailAPIClient->send_request(‘POST’, 'add_result_for…’, ‘jsonFileToUploa…’)
#1 C:*\testrailPostAutomatedResults.php(88): TestRailAPIClient->send_post(‘add_result_for_…’, ‘jsonFileToUploa…’)
#2 {main}
thrown in C:*\testrail-api-master\php\testrail.php on line 151


#4

Hi Conor,

Thanks for your reply! It looks as though you’ve only included the run ID in your API call and not the case ID, so you would also need to add your case ID to the call “add_result_for_case/25/:case_id” as mentioned in our documentation:

http://docs.gurock.com/testrail-api2/reference-results#add_result_for_case

You can also find more information on submitting results in bulk using the add_results_for_cases method on our website, including some request examples:

http://docs.gurock.com/testrail-api2/reference-results#add_results_for_cases

Hope this helps!

Regards,
Marco


#5

Hi Marco,

I am developing code using php. The steps I follow are:

  • I successfully add the Automation test run to Testrail:
    $result = $client->send_post(‘add_run/1’,
    array(
    “name”=> “Test run generated by Automation script at $currentTimestamp”, // Testrun name to add to milestone below
    "include_all"=> false, // It will add all testcase by default, so keep this at false
    "case_ids"=> [1, 212], // Testcases to add to Testrun. ‘Test Cases’ tab > hover over testcase ans see no. in status bar at bottom
    "milestone_id"=>3 // Get this No.: Milestones tab > hover over the milestone you want. See milestones/view/3
    )
    );

  • I execute some automation scripts and gather the results in a json file of the format:
    {
    “results”: [

              {
                      "case_id": "1",
                      "status_id": "1",
                      "comment": "This test was executed on 06/03/2017"
              },
              {
                      "case_id": "212",
                      "status_id": "1",
                      "comment": "This test was executed on 06/03/2017"
              }
      ]
    

}

  • I am trying to use the following function to place the results on testrail for testrun 26:
    $result = $client->send_post(
    ‘add_results_for_cases/26’,
    array(
    $jsonFileContent
    )
    );
  • $jsonFileContent = the json above
  • My problem is, I get the error:
    Fatal error: Uncaught exception ‘TestRailAPIException’ with message ‘TestRail API returned HTTP 400 (“Field :results is a required field.”)’ in C:\xampp\htdocs\conorTest\testrail-api-master\php\testrail.php:151
    Stack trace:
    #0 C:\xampp\htdocs\conorTest\testrail-api-master\php\testrail.php(87): TestRailAPIClient->_send_request(‘POST’, ‘add_results_for…’, Array)
    #1 C:\xampp\htdocs\conorTest\testrailPostAutomatedResults.php(67): TestRailAPIClient->send_post(‘add_results_for…’, Array)
    #2 {main}
    thrown in C:\xampp\htdocs\conorTest\testrail-api-master\php\testrail.php on line 151

Can you offer any help?
Thanks
Conor


#6

Hi Conor,

Thanks for your reply! In general, we wouldn’t be able to troubleshoot specific custom code for your automation script. Looking at the json content you provided, this would be a valid request if you submit this request directly via the API, so there’s likely some other issue with the script code itself that could be causing this (as the error mentions that it’s not finding the “results”: from your json file). Hope this helps!

Regards,
Marco


#7

Hi,

Can you please provide a simple example for using the add_results_for_cases request in the php language. There is no help on the web very basic help on your website. I need some direction if I am to keep my subscription with testrail. What I am looking for is an example: eg.
$result = $client->send_post(
‘add_results_for_cases/26’,
array(
{
{
“case_id”: “1”,
“status_id”: “1”,
“comment”: “This test was executed on 2017-03-14 10:28:44”
},
{
“case_id”: “212”,
“status_id”: “1”,
“comment”: “This test was executed on 2017-03-14 10:28:44”
}
)
);


#8

Hi Conor,

Thank you for the follow up post. We do have example requests on our site along with our API documentation. I would recommend looking at these. If you have someone there that is familiar with API programming and can help you, we would recommend reaching out to that person.

http://docs.gurock.com/testrail-api2/start

Below is an example request for the add_result method. This along with our documentation should give you ample information to complete this request.

{
“status_id”: 5,
“comment”: “This test failed”,
“elapsed”: “15s”,
“defects”: “TR-7”,
“version”: “1.0 RC1 build 3724”,

“custom_step_results”: [
{
“content”: “Step 1”,
“expected”: “Expected Result 1”,
“actual”: “Actual Result 1”,
“status_id”: 1
},
{
“content”: “Step 2”,
“expected”: “Expected Result 2”,
“actual”: “Actual Result 2”,
“status_id”: 2
},

]

}


#9

Hi Marty,

I have been reading all the documentation and I still cannot get it add_results() working for php. I cannot find a working code example anywhere on the web, so I am back looking for direction from you. I am trying to submit the results of automated testing suite to testrail using the add_results() function.

I want to submit these results to a testrun I have created in testrail. I am trying:
$result = $client->send_post(
“add_results/26”,
array(
’{
“results”: [

                {
                        "test_id": 1,
                        "status_id": 1,
                        "comment": "This test was executed on [2017 - 03 - 15 10: 17: 44]"
                }, 
                {
                        "test_id": 212,
                        "status_id": 1,
                        "comment": "This test was executed on  [2017 - 03 - 15 10: 17: 44]"
                }
        			]
			}'
		)
);

But it returns the error:
Fatal error: Uncaught exception ‘TestRailAPIException’ with message ‘TestRail API returned HTTP 400 (“Field :results is a required field.”)’ in C:\xampp\htdocs\conorTest\testrail-api-master\php\testrail.php:151
Stack trace:
#0 C:\xampp\htdocs\conorTest\testrail-api-master\php\testrail.php(87): TestRailAPIClient->_send_request(‘POST’, ‘add_results/26’, Array)
#1 C:\xampp\htdocs\conorTest\testrailPostAutomatedResults.php(60): TestRailAPIClient->send_post(‘add_results/26’, Array)
#2 {main}
thrown in C:\xampp\htdocs\conorTest\testrail-api-master\php\testrail.php on line 151

Can you help?


#10

Hi Conor,

Looking at your request, it does not look correct as per the example that I posted. Is it possible that you have someone in your group or another group within your organization that is familiar with API usage that can help you with this? I could additionally suggest trying this as a simple CURL request to send to the API to see if that works as well.