Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Sample Test Plan and Test Run automation code


#1

I got basic script code running which can be used to create a new test plan instance with a defined set of test runs. I Developed this code on a Mac using cURL as the interface technology. The code below runs on my machine against a cloud instance of TestRail when I insert my actual username, password and URL. This code creates a new test plan instance in project 1 and 4 new test runs based on defined configurations. I tried to comment the variables you might want to change but this is a quick start for anyone trying to automate this. Feel free to use any or all of this code to start your project.

#!/bin/bash

# This sample code will create a test plan with 4 defined test runs in a cloud hosted TestRail install

# General comments for all commands below
# Replace USERNAME and PASSWORD with your appropriate username and password values. These values should not
# be in quotes of any kind
# Replace EXAMPLE with the name of your cloud repository in TestRail

# Set this variable to the expected ID of the test plan about to be created. Y 
TestPlan=144

# Create the test plan
# Replace 'Hard Coded Name' with the desired test plan name. 
curl -s -u USERNAME:PASSWORD -X POST \
-d '{"name":"Hard Coded Name","milestone_id":3}' \
-H "Content-Type: application/json" \
'https://EXAMPLE.testrail.com/index.php?/api/v2/add_plan/1'


# Create the first run for the new test plan
# Replace 'Run 1' with the target name for this test run
# Update the 'Assignedto_id value to the appropriate resource id in your respository
# Update the 'config_ids list to correspond to the applicable configurations in your project
# This code will create a test run containing the 5 test cases listed under 'case_ids' for the specified config_id
curl -D- -u USERNAME:PASSWORD -X POST \
-d '{"suite_id":1,"name":"Run 1","assignedto_id":1,"include_all":true,"config_ids":[1,2,3,4],"runs":[{"case_ids":[6,7,8,9,10],"config_ids":[1]}]}' \
-H "Content-Type: application/json" \
https://EXAMPLE.testrail.com/index.php?/api/v2/add_plan_entry/$TestPlan

# Create the second test run
curl -D- -u USERNAME:PASSWORD -X POST \
-d '{"suite_id":1,"name":"Run 2","assignedto_id":1,"include_all":true,"config_ids":[1,2,3,4],"runs":[{"case_ids":[6,7,8,9,10],"config_ids":[2]}]}' \
-H "Content-Type: application/json" \
https://EXAMPLE.testrail.com/index.php?/api/v2/add_plan_entry/$TestPlan

# Create the third test run
curl -D- -u USERNAME:PASSWORD -X POST \
-d '{"suite_id":1,"name":"Run 3","assignedto_id":1,"include_all":true,"config_ids":[1,2,3,4],"runs":[{"case_ids":[6,7,8,9,10],"config_ids":[3]}]}' \
-H "Content-Type: application/json" \
https://EXAMPLE.testrail.com/index.php?/api/v2/add_plan_entry/$TestPlan

# Create the fourth test run
curl -D- -u USERNAME:PASSWORD -X POST \
-d '{"suite_id":1,"name":"Run 4","assignedto_id":1,"include_all":true,"config_ids":[1,2,3,4],"runs":[{"case_ids":[6,7,8,9,10],"config_ids":[4]}]}' \
-H "Content-Type: application/json" \
https://EXAMPLE.testrail.com/index.php?/api/v2/add_plan_entry/$TestPlan

end of file

You can also auto detect the new test plan ID using the jq library like the code below. This example is written to take the test plan name and projectID as command line parameters to the script

#!/bin/bash

TestPlan=144

# Create the test plan
#Construct the curl command
command=""
command='curl -s -u bruce.k.benton.-nd@disney.com:Disney11 -X POST'
command+=" -d '{\"name\":\""
command+=$name
command+="\",\"milestone_id\":3}'"
command+=" -H 'Content-Type: application/json'"
command+=" 'https://bkbdisney.testrail.com/index.php?/api/v2/add_plan/"
command+=$projectID
command+="'"

TestPlan=$(eval "$command" | jq -r .id)

#2

Hello Bruce,

Thanks for sharing this! We also have official bindings for the API for various programming languages (Java, Python, PHP, etc.):

http://docs.gurock.com/testrail-api2/start

Cheers,
Tobias


#3

I’m working through the Java interface now but the examples are pretty limited. I figured I could provide additional sample code for real world scenarios as I get them implemented. Make it a little simpler for the next person:-)

Which HTTP library do you suggest for the Java implementation. Right now I am working though that issue and getting basic authenticated connections working. I can connect to a web service I wrote that has no authentication but I cannot connect to the authenticated TestRail web Service


#4

why du you need a HTTP library at all ? When you use the provided Java binding, the whole http setup/authentication etc is not relevant for you.


#5

I went through the documentation once and it seemed to be a cyclic reference. I could not find the actual binding so I was implementing HttpUrlConnection code to interface to the service.

When I go to the linked Git repository I see the client example under the Java node but I do not see the package anywhere. When I look at the readme it links me back to the API docs that pointed me to Git in the first place.

Are there more complete installation instructions anywhere? These seem incomplete or out of date.


#6

for java bindings all described is here http://docs.gurock.com/testrail-api2/bindings-java

these are complete and working


#7

With all due respect, all is not explained there. As I said above the Java folder contains a sample client file and a sample exception file under java\com\gurock\testrail. The instructions refer to a “binding” and a “package” as residing in the directory. However the help content does not explain what these are (jar files?) and I can’t find anything in this directory other than the previously mentioned client and exeception files and a readme that just points back at the API online docs. specific information about the exact filename of the “binding” and “package” along with a specific directory where it resides would be most useful.


#8

Hi Bruce!

Thanks for your feedback. The entire API binding for Java is just two files in source code format (the client and the exception) and all the basics are covered by the binding. This includes the full HTTP request/response handling, authentication as well as JSON formatting. You can simply add the com.gurock.testrail package/source to a standard Java project and can then use the binding as follows:

APIClient client = new APIClient("http://<server>/testrail/");
client.setUser("..");
client.setPassword("..");
JSONObject c = (JSONObject) client.sendGet("get_case/1");
System.out.println(c.get("title"));

So, it’s just setting the username and password and you can then call API methods via sendGet (for read) or sendPost (for write methods). TestRail’s API uses the JSON format internally and TestRail implements this via JSONObject and JSONArray objects from org.json.simple.

I hope this helps!

Cheers,
Tobias