Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Select specific test cases and run those *only when you hit "Run Automated Tests button" on Testrail Web UI (button created using UI script)


#1

I am trying to use Testrail as a test case management system and so, integrating TestRail with the Jenkins would be useful.

This is what I want to achieve:

Lets say I manually create three test cases in TestRail with case ID’s C1, C2 and C3 and these test cases will have some unique automated test (testNG) names such as A1, A2, and A3 (in more info, there will be a field in testrail with such a unique information). I will annotate my testNG tests with the case ID’s from TestRail.

For instance,

{case id = C1}
@Test…{
Assert…
}

When I select only 2 test cases with ID’s C1 and C2 and hit “Start Automated Tests” button on TestRail UI, it will run a Jenkins job from TestRail (considering I have already implemented UI script for TestRail that has this button).

However, how do I pull those 2 specific test cases with their ID’s so that I can match them
with my testNG tests and run those tests only?

I am able to get the specific test case information locally on my machine via TestRail Python API. But the question is how do that selection on UI will interact and pass data to the operation of Automated Tests Button trigger?

Advise and suggestions.


#2

Hello Pratik,

Thanks for your posting! The idea would be to use the current selection (if any) to submit the test IDs to your trigger script. The trigger script can then limit the test execution to just the selected tests (you can find out the mapping between tests and cases with the get_tests API call for a run). To get the selection, you can use TestRail’s App.Tables.getSelected JavaScript method, or better some custom JS code that checks the ‘checked’ status of the checkboxes in the tables.

Cheers,
Tobias


#3

Hi Tobias,

Thanks for your response and I did not quite really get how to use App.Tables.getSelected method. Can you write me an example of how to do it? I am a newbie to TestRail and an example could be helpful.

This is what I want to achieve:

  1. Select test cases C1 and C2 on Testrail.com (there are only 3 test cases currently in my suite called “Autotests Suite”).
  2. Now, when I hit “Start Automated Tests” button on the following UI script:

name: Trigger Tests
author: Pratik Jaiswal
version: 1.0
includes: ^runs/view
excludes:

js:
(document).ready( function() { /* Create the button */ var button = (‘

Start Automated Tests
’);
  /* Add it to the toolbar */
  $("#content-header .content-header-inner").prepend(button);		
  /* Bind the click event to trigger the automated tests */
  $("a", button).click(
  	function()
  	{
  		$.ajax(
  		{
  			url: "trigger.php",
  			success: function()
  			{
  				location.reload();
  			},
  			error: function()
  			{
  				App.Dialogs.error(
  					'An error occurred while trying to trigger the automated tests.'
  				);
  			}
  		});
  		App.Dialogs.message(
  			'The tests are being processed in the background and the results are automatically posted back to TestRail. This page refreshes once the tests are finished.',
  			'Confirmation'
  		);
  		return false;
  	}
  );

}
);

above script should check this selection of C1 and C2 and throw the data for those test cases into the file or something. The trigger.php which is shown in the above code only has a command to run a Jenkins job.

After I get these test cases information into the file, I want to provide that as an input to my job (I can take care of that by doing some parsing). However, getting the real time test cases information from selection and throwing into the file would be of more priority.

A detailed example will be useful (once again I am a newbie)


#4

Sure, I’m happy to help! You can get the list of selecteds test as follows:

var test_ids = App.Tables.getSelected($('#groups'))

This returns a JavaScript array with the IDs. To pass this to your trigger script, you can format this as a comma-separated string as follows:

.. = test_ids.join(',');

It’s not directly possible to get the case IDs on the run page and you would need to pass the test IDs to your trigger script instead. In your trigger script, you can then call the get_tests API methods for the current run to map the test to case IDs (the case IDs are returned as part of the response).

I hope this helps!

Cheers,
Tobias


#5

I am not sure if I get you exactly. Please find the questions in the code below.

UI SCRIPT

name: Trigger Tests
author: Pratik Jaiswal
version: 1.0
includes: ^runs/view
excludes:

js:
(document).ready( function() { /* Create the button */ var button = (’

/* Create the button */
var button = $(‘

Start Automated Tests
’);

/* Add it to the toolbar */
$("#content-header .content-header-inner").prepend(button);

/get the test ids of test cases selected on TestRail/
var test_ids = App.Tables.getSelected($(’#groups’))

/* Question: Where to format this array into comma separated string?
/* Question: How to pass this to the trigger script shown right after this script? */

/* Bind the click event to trigger the automated tests */
("a", button).click( function() { .ajax(
{
url: “trigger.php”,
success: function()
{
location.reload();
},
error: function()
{
App.Dialogs.error(
‘An error occurred while trying to trigger the automated tests.’
);
}
});

  	App.Dialogs.message(
  		'The tests are being processed in the background and the results are automatically posted back to TestRail. This page refreshes once the tests are finished.',
  		'Confirmation'
  	);
  	return false;
  }

);
}
);

TRIGGER.PHP - TRIGGER SCRIPT**

<?php

function httpPost($url,$params)
{
$postData = ‘’;
//create name value pairs seperated by &
foreach($params as $k => $v)
{
$postData .= $k . ‘=’.$v.’&’;
}
$postData = rtrim($postData, ‘&’);

$ch = curl_init();
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_HEADER, false);
curl_setopt($ch,CURLOPT_RETURNTRANSFER,true);
curl_setopt($ch, CURLOPT_POST, count($postData));
curl_setopt($ch, CURLOPT_POSTFIELDS, $postData);
$output=curl_exec($ch);
curl_close($ch);
return $output;

}

$params = array(
“APP_BUILD_NUMBER” => “lastSuccessfulBuild”
);

echo httpPost(“http://user:34096b54@mobile-jenkins.pratikjaiswal.com:8080/job/checkrail-xxx-git/build”,$params);

?>

So by echoing httpPost, I am only triggering a Jenkins job when I click on Start Automated Tests button. However, I want to give this job a input as a file which will hold the content of test case ID’s obtained from selection such as package name, class name, method name, etc.****_ (re: JavaScript array)._ Can you help me in getting this example to work / maybe write me one? .

Thanks for your previous response.


[Automation]: Select specific test cases from test run and start tests
#6

Hi Pratik,

Sure, I’m happy to help. The idea is to pass the selection to the trigger.php script and this could look as follows, for example:

url: "trigger.php?ids=" + test_ids.join(',')

On the trigger script side, you can then extract the IDs from the $_GET variable and take it from there.

http://php.net/manual/en/reserved.variables.get.php

I hope this helps!

Cheers,
Tobias


#7

Hi Tobias,

Thanks for responding! Here is what I have tried:

Say I have 3 tests T1, T2 and T3 and I have selected T1 and T2 during the test run. I hit Start Automated Tests button (created via ui script) to call trigger.php residing on the local Windows server.

In the UI script, I have declared a var called test_ids as below:

var test_ids = App.Tables.getSelected($(’#groups’))

and my ajax call has a url field as:

url: “trigger.php?ids=” + test_ids.join(’,’)

…as per your suggestion and I believe this will hold the tests T1 and T2 (a/c to my understanding).

Here is the complete script:

name: Automated Tests Trigger
description: This script triggers automated tests for the test cases selected during the test run here on TestRail.
author: Pratik Jaiswal
includes: ^runs/view
excludes: 

js:
$(document).ready(
	function() {

		/* Create the button called Start Automated Tests*/
		var image = uiscripts.env.resource_base + "images/icons/running.png";
		var button = $("<span class=\"actions\"><a href=\"javascript:void(0)\"><img width=\"8\" height=\"16\" src=\"" + image + "\"/> Start Automated Tests</a></span>");

		/* Add the button to the toolbar */
		$("div#contentHeader span.actions:last").before(button);
		
		/*Get the test id's from the selection made during the current test run*/
		var test_ids = App.Tables.getSelected($('#groups'))

		/* Add click event to the button that we have created above. We are making an AJAX call to run trigger.php which is 
		located on our local Windows server. At the same time, we are passing the test ids for the current selection and you can
		see the selection formatted as a comma separated string.*/
		$("a", button).click(
			function()
			{
				$.ajax(
				{
					url: "trigger.php?ids=" + test_ids.join(','),
					dataType: "json",
					type: "POST",
					success: function()
					{
						location.reload();
					},
					error: function()
					{
						App.Dialogs.error(
							'An error occurred while trying to trigger the automated tests.'
						);
					}
				});

				App.Dialogs.message(
					'The tests are being processed in the background and the results are automatically posted back to TestRail. This page refreshes once the tests are finished.',
					'Confirmation'
				);

				return false;
			}
		);
	}
);

And my trigger.php on the local server has the following content:

<?php

echo 'Hello ' . htmlspecialchars($_GET["test_ids"]) . '!';
file_put_contents("myget.txt", $_GET["test_ids"]);

?>

However, nothing is getting stored into the file “myget.txt” but test_ids array should hold the content of tests T1 and T2?. Correct me if I am wrong.

Thanks again, Pratik!


#8

Hello Pratik,

Thanks for your reply! I would recommend reviewing this by stepping through your JavaScript code using the browser debugging tools and checking the arguments of the JS request. One thing I noticed that the data is passed via the ids GET parameter but you are checking for test_ids instead in your code.

I hope this helps!

Cheers,
Tobias


#9

Hello Tobias,

I have corrected the var from “test_ids” to “ids” and still I do not get any result into the array that I am trying to print into the file.

Here is how I am getting the test ids for the current selection:

var test_ids = App.Tables.getSelected($('#groups'))

And my AJAX requests to the following URL:

url: "trigger.php?ids=" + test_ids.join(',')

My server side script tries to print the content of above "ids" array by:

file_put_contents("myget.txt", $_GET["ids"]);

Stepping through debugging tools did not help either. Can you help me in making this example to work?

Thanks!
Pratik


#10

Hello Pratik,

Thanks for your reply! Have you checked that the test_ids variable as well as the URL actually contain the IDs? I would recommend stepping through your code with a JS debugger (inside your browser) and this would help understand where the issue is with the script:

I hope this helps!

Cheers,
Tobias


#11

Hi Tobias,

Thanks for the reply. I have debugged by Chrome JS console and looks like it says test_ids is undefined and I am getting Array[0] in that test_ids var.

Here is my declaration of var test_ids:

var test_ids = App.Tables.getSelected($('#groups'))

What does the #groups represents/holds? I want to fetch test_ids and their related details.

Thanks!
Pratik


#12

Thanks, Pratik! #groups is the area with the tables and getSelected returns all selected tests/rows in this area. I’ve just tested this with a small example and I could verify that the feature works as expected. The following UI script prints the current selection every second to the JS console:

name: Log current test selection
description: Logs the current test selection to the JS console
author: Gurock Software
version: 1.0
includes: ^runs/view
excludes: 

js:
$(document).ready(
    function() {
        setInterval(
            function()
            {
                console.log(App.Tables.getSelected($('#groups')));
            },
            1000
        );
    }
);

You would need to open the JS console in the browser debugging tools and you should see the current selection every second:

I hope this helps!

Cheers,
Tobias


#13

Hi Tobias,

That was super helpful, now I know where I was going wrong in declaring the JS var, probably the reason of my less familiarity with AJAX. Now how do I fetch the associated fields with the tests? For ex. dump all the titles of the selected tests as we are doing for the Test IDs above. Similarly, I have my own automated_tests_name field for every test and I want to dump that /show on console that too.

Thanks for your help.

Pratik


#14

Hello Pratik,

Great to hear that! You would use the get_tests API method to get the list of cases and can then use your test IDs to extract your subset of tests you want to test. get_tests also includes the case ID and case title as well as custom fields.

http://docs.gurock.com/testrail-api2/reference-tests

As your trigger script runs server-side, it wouldn’t be possible to send messages to the JS Console but you can use alternative logging features (such as syslog or error_log):

http://php.net/manual/en/function.syslog.php
http://php.net/manual/en/function.error-log.php

I hope this helps!

Cheers,
Tobias


#15

Thanks, I will try this out and follow up.


#16

Sounds good and just let me know in case any questions come up!

Cheers,
Tobias