Join 34,000+ subscribers and receive articles from our
blog about software quality, testing, QA and security.

Milestone Summary doesn´t include reruns


#1

Hi!

Project is at the end, all Testcases are done. I want to give my customer a summary of our tests.
all organized by TestRuns, we have 3 builds, we operate with multiple test suites
1 testrun = 1 configuration/ 1 buildversion
e.g.: 135 tested, 15 issues found
rerun-> all 15 issues could be closed

I expected, that I have now an overwiew, where I got 100% passed
Instead, 79% passed, 10 % failed and 11% untested.

If I switch the report, comparison for cases the coverage rate (for each testsuite) shows me the proper results:
100% passed

The milestone report doesn´t recognize the reruns.
In my reruns all cases with status “failed” or untested
got the new status “passed”.

Please inform me, which purpose has the milestone report, if it doesn´t include reruns?

Thanks,
Philip


#2

Hi Philip,

Thanks for your post! Did you also link the rerun test runs to the milestone? This would show the data for this milestone as well. Please note that it would also report on previous test runs that didn’t have 100% pass rate, so it would still show the full list of test runs including your reruns (if the reruns were also linked to the milestone), but this wouldn’t be a 100% pass rate as you had failed tests at some point on previous test runs. Hope this helps!

Regards,
Marco


#3

Hi Marco,

thank you for your message! Yes, all rerun test runs were linked to the same milestone.

e.g.
Milestone 1 build 42 -Testcase A Status: Untested
Milestone 1 build 50 -Testcase A Status: Passed

Milestone 1 build 43 -Testcase B Status: Failed
Milestone 1 build 50 -Testcase B Status: Passed

When I create the report “Milestone-Summary” for milestone 1, Testcase A counts as “Untested” and Testcase B counts as “Failed”. It´s just wrong. What is the point of this report?

Regards,
Philip


#4

Hi Marco,

maybe there was a misunderstanding. I don´t mean the full list of test results, just the overview.

Regards,
Philip


#5

Hi Philip,

Thanks for your reply! This might also be best to troubleshoot via email so we can get additional screenshots and details. Can you please send in another email to contact@gurock.com with the options you have configured in your report and a copy of the report as well (you can click the download button in the top right when viewing a test run to get the full report)? This will help us understand the issue and troubleshoot this.

Looking forward to your email!

Regards,
Marco


#6

Hi Marco,

mail has been sent.

Thanks,
Philip