Subscribe Join 180,000+ subscribers and receive articles from our blog about software quality, testing, QA and security. Subscribe

Queries TestRail

1.Does it support cucumber BDD?
2.Can i import test results ?From other testing tool ?
3.Test Versioning is applicable ?
4.Traceability to Requirement/Stories?

Hi @AshMark,

the following represents just my opinion.

TOP 1
Not really - realated to the test case description.
A description consists of given and customizable fields. Usual fields for text, single/multiple dropdowns or a more complex one like separated steps. To model a Gherkin style description is achallenge. Some poeple gave UI scripting a chance to structure bigger textfields with a kind of syntax coloring.
IMHO: Not supported out of the box. All tries are more or less workarounds which I don’t wont to use myself.

TOP 2
It is NOT possible to import test results. It is only possible to import test case descriptions only.
After adding or importing yur test cases you can create results manually (not really, but just to be complete) or a kind of automated via the API.
IMHO: If you really want to mirgate from one tool to another forget about the former results or keep them elsewhere.

TOP 3
Compared to Tools like HP / Microfocus ALM the support for versioning is really limited.
Some points in addition: Test case descriptions can be versioned manually using a simple copy and a version field or similar. You can use baselining (as a repository type), where the complete repository is copied e.g. for every release - quite heavy for bigger projects.
If you’re worried about changes on already executed tests, your are safe. A test is a kind of instance of a (test) case and can have multiple results. Executed tests can’t be changed by editing the case, this will result in a new version of the test.

TOP 4
TestRail supports a lot of tools for integration.
As usual JIRA is supported as well. But you need to check the integration itself, sometimes people are wondering about the exact functionality and its limitations.
In addition using JIRA as expamle:

  • You can add a reference from a test case description to any JIRA issue like a story or similar.
  • (Negative) Results of a tests can be linked to JIRA issues like bugs and defects.
  • A test run, a list of planned tests to be executed, can be linked to an issue.
    For Jira exists a plugin which will display links to TestRail or the content from TestRail.

I hope for other opinions to enrich the big picture.

Regards
Karsten

@kwirth Thanks for the prompt reply and detailed information .Upon exploring the TestRail i have few more queries for which i didn’t find any records.Can you help in understanding that as well.

  1. Can i edit ,clone and schedule test runs ?
    2.Can i run a single testcase without adding in test plan or test run ?
    3.Can i get the execution performance for both manual and automated test cases?
    4.Does it has query language for search ?
    5.Does it support Seetest tool?

Hi,
you’re welcome…

For the additional questions:

TOP 1
You can call re-run on an existing test run, which will create a new run for you based on the existing. Hint: Assignments to users/testers are not copied. Scheduling test measures via a run is IMHO NOT supported.

TOP 2
No, not really. Technically you need a run containing tests because tests are a kind of instances of cases. BUT, you can call run for a case which afair creates a run with the single test for you, ready to be executed. You shouldn’t mix up the repository (cases in sections and suites) with the tests organized in runs and plans - you will need the structures for bigger projects.

TOP 3
I gues you’re talking about the execution time for a test…
Executing a test in TestRail means to add a result to a test. This result contains a field with a timestamp. Manual testers are assisted with a sandclock functionality. Results from automated tests are usually added via the API and you can set the duration programmatically. But, the smallestunit is afair a second, what is not really usable for Unit Tests.

TOP 4
Unfortunately NO. If you’re used to JIRA’s JQL you will definately miss it…
It is more or less a simple text search and you even can’t search within the separated test if you want to use them…
Addon: The reporting is IMHO very limited as well, if you’re not happy with the existing ones.

TOP 5
Really not sure, never used it. Several people ask for supporting tool xyz, but in general TestRail doesn’t support it itself. Usually you have to add the results of your tests via the API. Probably there is already a kind of reporter for you, but very often these solution never fit your needs and you have to create yyour own solution.

Have fun,
Karsten

  1. Edit yes but they use rerun instead of a clone feature and since Test Rail does not run any actual tests - it can’t schedule a Test Run.

  2. No - it needs to be part of a test run. A Test Run makes a copy of the test case in the repository so the result of the test case is carried with the copy in the Test Run.

  3. There is an elapsed time but it records in seconds and 8 hour elapsed time is 1 day (it goes by work hours and not 24 hour. Automated tests - you would need to see if that can be passed in by the automation tool that runs the test and then call the Test Rail API to update the test run.

  4. No - it is just a repository of test cases that holds the history of a test case in a Test Run.

  5. No direct integration but Test Rial does have an API to integrate tools:
    http://docs.gurock.com/testrail-api2/start

Thank you @kwirth @BGanger.

It helps me to understand more !