Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Forecast not possible, more data needed


#1

Hi,

how much data we need to have in order to see the forecast ? We have background status working.

I did so far few test runs and also used ETA´s and actual times.

Regards,
Viljam


#2

Hello Viljam,

Thanks for your posting. You need more than 10 test cases with an estimate/forecast in a test suite so that the progress pages will describe the forecast as ‘High accuracy’. Estimates can be set when you edit a test case, forecasts for test cases are calculated by the background task based on the Elapsed values you enter for test results.

You can see the actual estimates/forecasts for test cases by adding the Estimate/Forecast columns to the test case/test tables on the suite and run pages:

Of course, the more estimates/forecasts you enter for the test cases, the more accurate the forecast will be for the runs/plans/milestones. This is also explained in TestRail’s user manual under Help > TestRail Help > Usage tips & tricks.

Just let me know in case anything is unclear.

Regards,
Tobias


#3

Hi,

thank you for the replay :slight_smile:

Seems there is still something wrong with the timings:

We have made sanity test run which takes about 1h by our estimates. We have done more than over 20 sanity runs already in one project. If i look the High accuracy forecast then this shows 16/12/2012 which does not make any sense what so ever.

Test run was created : 05/06/2012 completed 2% so far. Our estimates are 1h completed 1m and left 59 min of work but how the ETA is 16/12/2012 ?

Thanks,
Viljam


#4

Hello Viljam,

The forecasting also takes into account the so-called velocity of your testing activity. As an example, since it took a fews days to process 2% of your test run, TestRail assume that it takes a few months to finish it.

I hope this helps!

Regards,
Tobias


#5

Thanks Tobias,

It does not make any sense if the total ETA is 51 min. Previous testing times has been entered as 21 min many times in the row.

Even if the task has been opened for about 2 days the calculation should not show 6 month longer ???

Not sure but the forecast formula you have then does not work very well so far for us and do not show any valuable information.


#6

Hello Viljam,

Thanks for the update. The forecast does not only need to take into account the approximate estimate for the test run but also how fast you are progressing on this test run. Image you are usually working with 5 testers on a test run and can process 10% of the test run per day. After one day of work, TestRail will show a forecast of about 10 days for this test run. Now image you are only working with 1 tester and can only process 2% per day. Even though the estimate as a whole is the same for the test run, TestRail would need to forecast about 50 days of work until the test run is completed.

I hope this helps,
Tobias