Join 34,000+ subscribers and receive articles from our
blog about software quality, testing, QA and security.

TestRail, Cucumber and BDD


#1

Hi,
We are currently looking at purchasing TestRail but currently we cannot figure out how to integrate our cucumber tests into the application. I can see that it would be possible to push test results via the api but we would also like to automatically create test runs and also define test suites based upon our feature and spec files.

Is this something that can be achieved? Would it envolve dealing with the DB directly?


#2

Hello Liam,

Thanks for your posting. We do have an extended API that allows you to query and create test cases via HTTP requests. This way you could create test cases and “sync” your test cases via the API in addition to submitting test results. The extended API is currently available on request and the plan is to publish the latest version on our website soon. I will make sure to email you the extended API in a moment. It’s also possible to directly access TestRail’s database for custom reports and similar, but it really depends on the exact needs you have.

Thanks,
Dennis


#3

Ok thanks, I will test it out and post back what I find.


#4

Hi Liam,

did you have made any progress on that topic? We also start to figure out how to structure test suites to align them with feature file directory structure and map gherkin features to testrail test cases, scenarios to testrail test steps etc.

Could imagine a git hook to create/update testrails specs after pushing feature files and a cucumber formatter that outputs either testrail compatible json results or maybe directly push the results to the API.

Feedback really appreciated,
inka


#5

Hello,

I was wondering if any progress has been made with API and Cucumber integration.

Thanks


#6

Hello Maksim,

Thanks for your posting. TestRail comes with a very feature-rich and generic API which can be used to integrate most automation tools:

http://docs.gurock.com/testrail-api2/start

While we don’t have any Cucumber plugins at the moment, the API is very straightforward and uses simple HTTP requests to add test results etc. We also have ready to use bindings for various programming languages (.NET, Java, Python, etc.).

I hope this helps and we are happy to help in case you have any further questions.

Regards,
Tobias


#7

I am also wondering if we have a plugins to highlight the Gherkin steps in a test case (for custom steps field), that would be helpful.


#8

Hello Ryan,

Thanks for your posting. There’s currently no official plugin from us for this but I believe this should be possible to implement with a UI script (JavaScript + CSS customizations):

http://docs.gurock.com/testrail-custom/uiscripts-introduction

Regards,
Tobias


#9

Looks like this solution works good!! Thanks Tobias!


#10

That’s great to hear, Ryan, and you are welcome!

Regards,
Tobias


#11

I also integrated Cucumber / BDD into TestRail, but miss the color highlighting. (I didn’t find the time to write an UI-Script for this).

Has someone accomplished this yet and would share it?

EDIT: Just wrote my own UI-Script now :smile:

Nothing special, just highlights the beginning step defention keywords and bolds them.

name: Gherkin / Cucumber step highlighting
description: Highlights the Gherkin / Cucumber step definitions “Given”,“When”,“Then”,“And” and "But"
author: Daniel G.
version: 1.0
includes: ^cases || ^tests/view
excludes:

js:
var searchArray = [“Given”,“When”,“Then”,“And”,“But”];
var searchArrayLength = searchArray.length;

$(document).ready(function () {

for (var i = 0; i < searchArrayLength; i++) {
$(".markdown:contains(’"+searchArray[i]+"’)").each(function () {

var htmlString = $(this).html();
var newHtmlString = “”;

if(searchArray[i] == “Given”) {
newHtmlString = htmlString.replace(searchArray[i], “”+searchArray[i]+"");
} else if(searchArray[i] == “When”) {
newHtmlString = htmlString.replace(searchArray[i], “”+searchArray[i]+"");
} else if(searchArray[i] == “Then”) {
newHtmlString = htmlString.replace(searchArray[i], “”+searchArray[i]+"");
} else {
newHtmlString = htmlString.replace(searchArray[i], “”+searchArray[i]+"");
};

  $(this).html(newHtmlString);
});

}
});

css:

.orange {
color:orange;
font-weight: bold;
}

.green {
color:green;
font-weight: bold;
}

.blue {
color:#6A37FF;
font-weight: bold;
}

.grey{
color:grey;
font-weight: bold;
}


#12

Thanks for sharing this, Daniel, that looks really great!

Regards,
Tobias


#13

Hello there!

I am new to testrail and it’s integration with cucumber features. I want to integrate our all BDD scenarios to testrail and update their results after each execution going further. Any suggestions on how do we do that?
Note: I am working with JAVA.

Thanks,
Dhanesh


#14

Hi Dhanesh,

Thanks for your posting! You can use the Java binding to use TestRail’s API from Java or cucumber:

http://docs.gurock.com/testrail-api2/bindings-java

The documentation includes several examples on how to use the API from Java and we are happy to help in case you have any additional questions.

I hope this helps!

Cheers,
Tobias


#15

Hi Daniel,
is it possible to share the steps that you did in order to integrate Cucumber BDD into testrails?
regards


#16

You can look here what I did:


#17

Hi,

Has anyone got a shared git repo with BBD/Cucumber (Ruby) integration for Testrail?
I understand how the API calls work, but I’m not sure how to integrate them into my setup.

Thanks


#18

@Daniel : Hi Daniel this is Rahul , i would like to know , post processing code when integrated with Cucumber ? how dynamically post method is taking the test case ID from the current scenario name and updating the results in Test Rail ?
My framework is like below :

  1. Jenkins triggers the cucumber tests
  2. After scenario with tag name is present to close the browser before it starts executing the another scenario
  3. Final step , Cucumber is generating the JSON report which consists of all the pass and failures.

Help is much appreciated !!


#19

Hi Daniel,

Sorry I am new to this. I created a UI-Script with your code and saved it. I created a test case field of type “text” for Gherkin scenario. Then I created a new test case and added a scenario in Gherkin but i don’t see the highlighting after saving. The script should be applied to all projects.
Can you let me know what I did wrong? TestRail is on 5.3.0.3603