Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Support for Selenium


#1

Does TestRail have any support for integration with Selenium?

I’ve read the brief blurb about the HTTP-API, so imagine someone may have written some integration?

Thanks
Phil


Selenium integration
#2

Hello Phil,

Thanks for your posting. While we don’t have a ready-to-use integration with Selenium, I know that several customers already use Selenium with TestRail. TestRail’s API is very easy to use so it should be quite easy to integrate it with Selenium:

http://docs.gurock.com/testrail-api/start

Which programming language do you use with Selenium?

I hope this helps and please let me know in case you have any further questions.

Regards,
Tobias


#3

We’ve got over 300 tests integrated with selenium at the moment and growing. We use Java/TestNG and have hooked in through a custom listener and web service calls. We’ve also used the UI scripts to include logs and screenshots on failures, and just recently execute our tests based on TestRail runs and custom fields. I can say that things have been pretty extendible, although it takes some work to integrate it.


#4

Can you help me to get a better picture of what an integration between Selenium and TestRail looks like? For example: what’s the purpose? To make the results from a Selenium test run visible in the TestRail graphs and figures? Does someone have more examples?

What’s the work involved to achieve it? I imagine that somehow you have to extract all kinds of details from the Selenium database and put it into a HTTP Post request, as offered by TestRail API, to get it in TestRail.
Right, or ?


#5

@chrisk: How much effort (time, #persons, specific knowledge needed) have you spent roughly on getting those 300 test cases integrated?


#6

Many person hours to create the tests, but to integrate 300 took the same effort as integrating 1.

We’ve created a few of our own APIs to support more advanced integration. For example, we have a button we can click in TestRail that will kick off a test and report back. We also have TestNG query TestRail to figure out what its supposed to execute.

However it can be pretty simple to run a test and have it report PASS/FAIL result to TestRail:

  1. Create a java class which has code that takes a test id, and a result, and passes it to TestRail using an Http Connection object. It requires some Java knowledge, but isn’t difficult, and examples are posted everywhere.

  2. Create a TestNg listener which now listens to test results, and takes the test id and result and sends it to your Java class.

  3. Create an annotation object which you can use to mark up tests with their test ids.

That gives you a pretty easy way to do it. Of course you can do it even easier if you want to add a function to every test that reports the result. It can get much more advanced if you want it to as well. For example, we have custom fields in TestNG that match the class that executes the test, so after the test runs, it uses the class and run id to find the test and report its result.

Really it largely depends on how you’ve structured your test suite and framework. If its simple, it shouldn’t be difficult. If its advanced, it might take a little more time.


#7

Thanks Chrisk, for sketching the steps to take.


#8

another way which is not as tightly coupled as Chris’s approach would be to process your results after all the tests have run. For example, if you are (and you should) running your unit/automated tests via jenkins/hudson/bamboo/etc, you should product some junit compatible xml report. You could have a simple report processor in java/C#/etc to just parse the xml file and then use the testRail API to pump the results into appropriate testplans/testsuites

We have done just this and I think it has the following advantages

  1. You don’t have to change any of the existing test framework. Just run you tests as it via some CI and add one last step in your CI. We use Bamboo/Jenkins and have a simple java program at the end
    java -jar results_processor.jar …

  2. Since you are using TestRail, you probably have a distinct QA and Dev team. However, tests are just tests and there is no reason you should not pull development unit tests into testrail. Using approach #1, you can just pull them in TestRail

  3. If TestRail API or application fails for some reason, you won’t be bound by tightly integrated approach.


#9

Hi,
We connected TestRail to atlassian Bamboo .
We developed a Java connector between TestRail API to atlassian Bamboo.
Bamboo is executing our Selenium Web Drive tests on remote agents and the results is being “injected” into TestRail test run including status , failed reason and screen shot as attachment .

The test run is include also manual tests , for example we have Regression test run , whatever is not automated is being executed manually. So,we see the status and defect of all test result in one report.

BR
Itsik


#10

Hi, Anyone please show step by step integration of TestRail with selenium .