Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Test Runs & Results


#1

I don’t understand why someone who want to add a Test Run (which covers one test suite) vs a Test Plan (which can cover one to infinite test suites).


#2

Hello,

If you are just interested in starting tests for single test suites, starting simple test runs is usually preferred. The big advantage of test plans (besides the option to use configurations) is that it acts as an additional container with its own summarized statistics and metrics. This is especially useful if you want to group multiple test runs. But if you don’t need the benefits a test plan provides, starting separate test runs usually makes things a bit easier and cleaner.

I hope this helps.

Regards,
Dennis


#3

One possible advantage for runs over plans is quickly communicating status of a new feature or an Acceptance suite. This way, managers or scrub masters can quickly see very focused status on a specific area of change or for a suite representing the “80%” of the use cases for a release effort.

A test plan might be specific to a milestone or phase whereas a run (based on a suite) operates more like a dashboard that gets reset every build so it always indicates release readiness for that feature or meeting acceptance criteria.

So my take is to use test plans to document history… what was tested and what was failing when; and, to use runs to document status of an area… now.


#4

Hi Donald,

Thanks for the feedback and for sharing how you use TestRail. That’s appreciated and I’m sure it will help some new customers to get a productive workflow with TestRail faster.

Thanks,
Dennis