Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Integrating TestRail with TFS - custom or official solution?


#1

We are looking into integrating TestRail and TSF. The link (view/add) between the two works fine.
Now the big question is: does anybody have any experience with using a custom plugin to integrate TFS so pushing bugs becomes an option?
I have searched the forum and have not found much. Is an official plugin still in the making or even planned?

Thank you for your feedback.


#2

I just finished creating my own TFS plugin for looking up existing defects and pushing a new one to TFS. I can help you.


#3

Cheers! That would be great. So far I have only worked with TestRail/Jira, so I am not entirely sure how much functionality to expect.

Thus, beginning with some general questions:
The elements I would like to push are title, iteration, activity, severity, area, steps to reproduce, and acceptance criteria. Is that feasible?

Will account and password have to be synchronised between TFS and TestRail?

Does the plugin have to “speak” in the language and format used in the TFS REST?

I am not great at coding, so I might have to involve a colleague who can help me.


#4

My IT department created a TestRail service account that has read/write access to the TFS project being pushed to. This was the only way to do it because of the way our network authentication is setup. We couldn’t use individual user account/password. When you push a defect from TestRail the service account name is the one used as the “Entered By”.

The only elements I push are the Title and Steps to Reproduce. All the other ones are entered into TFS with the default values. I’m still trying to find a way to push other fields. The problem with pushing a field that is a selection from a list in TFS is that you need to know the ID of the selected item. That means you have to some how retrieve the list of IDs from TFS and present them in the TestRail dialog. I haven’t found a way to do this yet.

The following code is what I am using. You will have to use your server names and path.

<?php if (!defined('ROOTPATH')) exit('No direct script access allowed'); ?> <?php class TFS_defect_plugin extends Defect_plugin { private $_api; private $_server; private $_collection; private $_project; private $_user; private $_password; private static $_meta_defects = array( 'author' => 'your company name', 'version' => '1.0', 'description' => 'TFS workitem plugin for TestRail', 'can_push' => true, 'can_lookup' => true, 'default_config' => '; Please configure your TFS connection below. ; Production Server = ; Development Server = ; Collection = [connection] server= collection= user=%tfs_user% password=%tfs_password% '); private static $_meta_references = array( 'author' => 'your company name', 'version' => '1.0', 'description' => 'TFS reference plugin for TestRail', 'can_push' => false, // Lookup only 'can_lookup' => true, 'default_config' => '; Please configure your TFS connection below. ; Production Server = ; Development Server = ; Collection = [connection] server= collection= user=%tfs_user% password=%tfs_password% '); public function get_meta() { if ($this->get_type() == GI_INTEGRATION_TYPE_REFERENCES) { return self::$_meta_references; } else { return self::$_meta_defects; } } // ********************************************************* // * CONFIGURATION // ********************************************************* public function validate_config($config) { $ini = ini::parse($config); if (!isset($ini['connection'])) { throw new ValidationException('Missing [connection] group'); } $keys = array('server', 'collection'); // Check required values for existance foreach ($keys as $key) { if (!isset($ini['connection'][$key]) || !$ini['connection'][$key]) { throw new ValidationException( "Missing configuration for key '$key'" ); } } $server = $ini['connection']['server']; $collection = $ini['connection']['collection']; // Check whether the server is a valid url (syntax only) if (!check::url($server)) { throw new ValidationException('Server is not a valid url'); } if (isset($ini['push.fields'])) { // Rules to verify custom fields foreach ($ini['push.fields'] as $field => $option) { if ($option != 'on') { continue; } $this->_validate_field($ini, $field); } } } public function configure($config) { $ini = ini::parse($config); $this->_server = str::slash($ini['connection']['server']); $this->_collection = $ini['connection']['collection']; $this->_user = $ini['connection']['user']; $this->_password = $ini['connection']['password']; } // ********************************************************* // * PUSH // ********************************************************* public function prepare_push($context) { // Return a form with the following fields/properties return array( 'fields' => array( 'title' => array( 'type' => 'string', 'label' => 'Title', 'required' => true, 'size' => 'full' ), 'sqa' => array( 'type' => 'string', 'label' => 'SQA', 'required' => true, 'remember' => true, 'size' => 'compact' ), 'description' => array( 'type' => 'text', 'label' => 'Steps to Reproduce', 'required' => true, 'rows' => 20 ) ) ); } public function prepare_field($context, $input, $field) { $data = array(); // Take into account the preferences of the user, but only // for the initial form rendering (not for dynamic loads). if ($context['event'] == 'prepare') { $prefs = arr::get($context, 'preferences'); } else { $prefs = null; } switch ($field) { case 'title': $data['default'] = $this->_get_title_default($context); break; case 'sqa': $data['default'] = $context['user']->name; break; } return $data; } public function validate_push($context, $input) { } public function push($context, $input) { $api = $this->_get_api($context); return $api->add_defect($input); } // ********************************************************* // * LOOKUP // ********************************************************* public function lookup($workitem_id) { $api = $this->_get_api(); $workitem = $api->get_workitem($workitem_id); if ($workitem == Null) { throw new ValidationException('workitem = null'); } //d($workitem); // Build the status_id based on the state property if ($workitem->fields->{'System.State'} == 'Closed') { $status_id = GI_DEFECTS_STATUS_CLOSED; } elseif ($workitem->fields->{'System.State'} == 'Resolved') { $status_id = GI_DEFECTS_STATUS_RESOLVED; } else { $status_id = GI_DEFECTS_STATUS_OPEN; } if (isset($workitem->fields->{'Microsoft.VSTS.Common.StackRank'})) { $StackRank = $workitem->fields->{'Microsoft.VSTS.Common.StackRank'}; } else { $StackRank = ''; } if (isset($workitem->fields->{'Microsoft.VSTS.TCM.ReproSteps'})) { $ReproSteps = $workitem->fields->{'Microsoft.VSTS.TCM.ReproSteps'}; } else { $ReproSteps = ''; } // If the workitem is a bug these fields will be displayed in the popup window if ($workitem->fields->{'System.WorkItemType'} == 'Bug') { return array( // * required properties 'id' => $workitem->id, 'title' => $workitem->fields->{'System.Title'}, 'status_id' => $status_id, 'status' => $workitem->fields->{'System.State'}, // * optional properties //'url' => , 'description' => $ReproSteps, 'attributes' => array( 'Type' => $workitem->fields->{'System.WorkItemType'}, 'Status' => $workitem->fields->{'System.State'}, 'Created By' => $workitem->fields->{'System.CreatedBy'}, 'SQA' => $workitem->fields->{'Schilling.VSTS.Bug.SQA'}, 'Stack Rank' => $StackRank, 'Severity' => $workitem->fields->{'Microsoft.VSTS.Common.Severity'} //'Iteration' => $workitem->fields->{'System.IterationPath'} ) ); } else // the workitem is a Product Backlog Item, display these fields in the popup window { return array( // * required properties 'id' => $workitem->id, 'title' => $workitem->fields->{'System.Title'}, 'status_id' => $status_id, 'status' => $workitem->fields->{'System.State'}, // * optional properties //'url' => , 'description' => $workitem->fields->{'Microsoft.VSTS.Common.DescriptionHtml'}, 'attributes' => array( 'Type' => $workitem->fields->{'System.WorkItemType'}, 'Status' => $workitem->fields->{'System.State'}, 'Created By' => $workitem->fields->{'System.CreatedBy'} ) ); } } // ********************************************************* // * Get title for the workitem // ********************************************************* private function _get_title_default($context) { $test = current($context['tests']); $title = 'Failed test: ' . $test->case->title; if ($context['test_count'] > 1) { $title .= ' (+others)'; } return $title; } // ********************************************************* // * API / CONNECTION // ********************************************************* private function _get_api() { if ($this->_api) { return $this->_api; } $this->_api = new TFS_api($this->_server, $this->_collection); return $this->_api; } } // ******************************************************* // * API connection to TFS // * If you need data from TFS, this is where you put it // ******************************************************* class TFS_api { private $_path; private $_curl; private $_createmeta = array(); // *************************************************** // * Constructor // *************************************************** public function __construct($server, $collection) { $this->_path = $server.$collection; } // *************************************************** // * Get work item by ID // *************************************************** public function get_workitem($id) { // Initialize the REST request and fetch the JSON result. $restUrl = $this->_path.'_apis/wit/workitems/'.$id; $ch = $this->getRestClient($restUrl); $data = curl_exec($ch); return $workitem = json_decode($data); } // *************************************************** // * Add a new defect // *************************************************** public function add_defect($options) { $data = $options; if (isset($options['title'])) { $data['title'] = $options['title']; } if (isset($options['description'])) { $data['description'] = nl2br(html::link_urls(h($options['description']))); } $restUrl = $this->_path.'xxxxxxx/_apis/wit/workitems/$bug'; //The xxxxxxxx is your TFS project $response = json_decode($this->postRestClient($restUrl, $data)); $id = $response->id; if (!isset($id)) { $this->_throw_error('No workitem ID received'); } return (string) $id; } // *************************************************** // * Get an initialized REST request using integrated security. // * // * @param mixed $url REST URL // * @return resource // *************************************************** private function getRestClient($restUrl) { //throw new ValidationException($restUrl.' in getRestClient'); $headers = array('Accept: application/json;api-version=1.0'); $curl = curl_init(); curl_setopt($curl, CURLOPT_URL, $restUrl); curl_setopt($curl, CURLOPT_HTTPHEADER, $headers); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); curl_setopt($curl, CURLOPT_HTTPAUTH, CURLAUTH_NTLM); curl_setopt($curl, CURLOPT_UNRESTRICTED_AUTH, true); curl_setopt($curl, CURLOPT_USERPWD, ":"); return $curl; } private function postRestClient($restUrl, $input) { //throw new ValidationException($input['severity']); // Define the new work item properties as a REST PATCH JSON document. $data = '[ { "op": "add", "path": "/fields/System.Title", "value": "'.$input['title'].'" }, { "op": "add", "path": "/fields/Microsoft.VSTS.TCM.ReproSteps", "value": "'.$input['description'].'" } ]'; // Define PATCH content type and JSON result type. $headers = array( 'Content-Type: application/json-patch+json', 'Accept: application/json;api-version=1.0' ); $curl = curl_init(); curl_setopt($curl, CURLOPT_URL, $restUrl); curl_setopt($curl, CURLOPT_HTTPHEADER, $headers); // Use HTTP PATCH verb curl_setopt($curl, CURLOPT_CUSTOMREQUEST, 'PATCH'); curl_setopt($curl, CURLOPT_POSTFIELDS, $data); // Return a string result to the return variable curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); // Use Windows integrated security curl_setopt($curl, CURLOPT_HTTPAUTH, CURLAUTH_NTLM); curl_setopt($curl, CURLOPT_UNRESTRICTED_AUTH, true); curl_setopt($curl, CURLOPT_USERPWD, ":"); // Get the response and close the resource. $response = curl_exec($curl); curl_close($curl); return $response; } }

#5

Thank you so much! I will see how this lines up with what I have and work from there. I will keep you updated on how it goes.


#6

No problem. I’m also working on adding more fields to push.


#7

Here is an update on my progress:
I have tried the plugin with my server address. The push form opens, but I get a “Trying to get property of non-object” error when pushing. Also the defect lookup returns a “Plugin “TFS” returned an error: workitem = null”.

So I am halfway there.

I am quite certain I have the server and collection wrong:

[connection]
server=http://servername:port/tfs/
collection=DefaultCollection/Evaluation/_workitems
user=%tfs_user%
password=%tfs_password%


#8

Update:
Trying to get property of non-object

happens in this line under add_defect:
$id = $response->id;


#9

In the Administration>Integration section, what do you have as the URL in your “Defect View Url” ? Try just using “DefaultCollection” as your collection.


#10

I got a step further yesterday. Almost there.
I am not sure that the user/password works right. When I hard code my user/password into the [quote]
curl_setopt(curl, CURLOPT_USERPWD, ":"); [/quote] I can push and lookup workitems. Now I need to find a way to do it with variables. `_user . “:” . $_password` does not seem to work.


#11

Did you create the user variables, tfs_user and tfs_password? If you did then you have to go to your account settings and you should see them under the Settings tab. Enter your user name and password there.


#12

Thank you. Yes, I did:

And I filled them accordingly. Somehow that still does not seem to work. When I enter my login credentials in the plugin, it works.

In the procedure I have private $_user, user=%tfs_user% and $this->_user = $ini['connection']['user'];. That ought to set my TFS User as _user. Still, if I use _user in curl_setopt($curl, CURLOPT_USERPWD, ":"); I get an <unknown variable _user>.


#13

I think I figured it out. The TFS_api class doesn’t know about the variables, user and password. You have to pass them into the class. Add the variables to the new TFS_api call. Add them to the constructor and then it should work.

private function _get_api()
{
if ($this->_api)
{
return $this->_api;
}

	$this->_api = new TFS_api($this->_server, $this->_collection, $this->_user, $this->_password);
	
	return $this->_api;
}

class TFS_api
{
private _path; private _curl;
private _user; private _password;

public function __construct($server, $collection, $user, $password)
{
$this->_path = $server.$collection;
$this->_user = $user;
$this->_password = $password;
}


#14

It works! Thank you so very much: I added your correction, but I also needed to insert it in the get_workitem funktion:

public function get_workitem($id)
{
// Initialize the REST request and fetch the JSON result.
$restUrl = $this->_path.’_apis/wit/workitems/’.id; _user = this->_user; _password = $this->_password;

$ch = $this->getRestClient($restUrl, $_user, $_password);

$data = curl_exec($ch);

return $workitem = json_decode($data);

and add_defect function:

public function add_defect($options)
{
$data = options; _user = this->_user; _password = $this->_password;

if (isset($options['title']))
{
	$data['title'] = $options['title'];
}

if (isset($options['description']))
{
	$data['description'] = nl2br(html::link_urls(h($options['description'])));
}


$restUrl = $this->_path.'Evaluation/_apis/wit/workitems/$Bug'; //The xxxxxxxx is your TFS project
		
$response = json_decode($this->postRestClient($restUrl, $data, $_user, $_password));

#15

Great news. If there is any else I can help on, just post it here. I check the forums regularly.

I did figure out how to push the severity to TFS. I created a dropdown field and populated the dropdown with this.

private function _get_severity()
{
return array(
‘1 - Critical’ => ‘1 - Critical’,
‘2 - High’ => ‘2 - High’,
‘3 - Medium’ => ‘3 - Medium’,
‘4 - Low’ => ‘4 - Low’
);
}

The trick is the text before the “=>” is what gets pushed to TFS. So that has to match what is already in TFS.

So far it looks like only you and me are using TestRail with TFS.


#16

Great. I will have to look into that. That is something I suppose we will need to do as well. I am also going to try to prefil the “steps to reproduce” with the content of the test result comment field. I know that is doable with JIRA, so I assume it might work with TFS, too.


#17

Has this solution worked well for either of you now that it’s been over a year?


#18

I also am interested in this option. The talk is a bit to techie for me. I am trying to figure out how to get Testrail and TFS to link and will be trying to understand the solution here.


#19

We use TFS but we don’t do full integration or use the API. We integrate so References and Defect fields can be linked to work items in TFS and that is it. We are a really small group so for now it is doable.