Join 34,000+ subscribers and receive articles from our
blog about software quality, testing, QA and security.

Best practice for different approaches for same test case?


#1

Hi!

I have a question about what a suggested best practice would be for having multiple approaches to the same test case.

For example:

I’m working with video game development, and for our testing we can do it either with debug options and cheats, or play the game normally.

These 2 different approaches have vastly different Estimates for completion that I want to have documented in the test cases, and I want to be able to able to estimate how long a testrun will take with either approach, depending on current testing focus.

Would you suggest 2 test suites with identical test cases, and simply name the suites after the approach? Are there better ways of doing this?

Thanks for a great tool!


#2

Hi @Flappyjacks,

Thanks for your posting! We usually advise against duplicating test cases in almost all cases as this makes it more difficult to update cases and keep them in sync.

For the estimates, I would recommend going with an average of the different game modes. The estimate for a case is only one of the values TestRail looks at when it calculates the progress, forecast and projected completion dates and the actual “elapsed” values (which can be entered on the Add Result dialog) are more important actually. TestRail uses the elapsed times to calculate an accurate forecast per case and this gets more accurate the more tests/elapsed values you add over time so using an average for the estimate should work fine.

TestRail also takes into account the so called “velocity” (how fast you are) as you go through your tests and this is another key component that affects the progress/forecasts/projected completion dates. So, using your example, TestRail will automatically project a later completion date for more time-consuming test runs with debug/cheat modes when it notices that it takes longer to go through your tests.

I hope this helps!

Cheers,
Tobias


#3

We have the same issue. The problem is that management wants to see if we are hitting target completion times and the scheduling is based on the Estimate field in the test case so we cannot have an “average” Estimate number.
Is there anyway for individual Test Runs to provide an over-ride number for the Estimate?
We have hundreds, if not thousands, of individual test cases and so there is a lot of automated scripts to try and process the data from Testrail into some readable summary for management. The individual estimates for Test Runs have to be kept somewhere but Testrail doesn’t provide that facility so we are currently resorting to different Testcases even though it is the same test.