Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

How are you tracking unit test coverage using Testrail?


#1

Hi,

It is ideal to have a solid amount of coverage done at a unit test level … compared to a webdriver/slow level. In the past, I have only been concerned about tracking webdriver tests in testrail because they follow a flow, making our manual test cases easy to automate.

Now we want to track unit tests and I’m wondering what other companies out there are doing (or what testrail guru’s recommend).

Basically we have no issues adding unit tests to testrail and adding results to them, the problem is the # of test cases that would add, we will potentially slow down testrail.

My overall goal is to be able to see a picture of our features and the manual, unit and webdriver test coverage. I want to be able to flip (or delete) manual tests when they have become automated (aka maintenance).

My gut tells me that we might just need to leave testrail as webdriver + manual, extract that data to kibana or another reporting tool, have our CI tool that reports on unit test results also export to that reporting tool, link them all by feature (maybe by component) and get a view that way. The one issue with this is that when i go to testrail, I need to understand there will be test gaps because unit tests will cover them… Perhaps I would just flag that test as ‘unit tested’, perhaps have a link to where in the code this is covered and leave it at that…

Anyway, anyone have any success stories/other ideas?


#2

Personally, I would not like to include unittests into TestRail. For me they have a different purpose than the test cases in TestRail.

We use TestRail to define system/end2end tests, let them be manual or automated, that does not matter. The people working with that are Testers, Test Management and QA Engineers. They maybe also be developers, but then as a different role. Unittesting is part of development, is technical and the sole responsibility of the developer.

We start a testplan, when the development is in a state that it can be tested as a product, this includes, that all unittests (and integration tests - however you define them) are passed.

So for me there are conceptional differences, so that I would not add unittests to TestRail (and I hope there is someone out here, that a) supports my idea and b) can better reason about it).

We have sonarqube for measuring codequality, including unit test coverage, which is part of the CI pipeline.

I do not see, why there should be “gaps” in TestRail, because, as said, i think there are conceptional differences. Test cases in TestRail verify e.g. an epic are user story or what ever, unit test does not touch that at all, its soley looks at its unit…

hope this gives meaning