Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.

Clearer timeseries progression/latest status across multiple test runs of a milestone


What is the easiest way to see timeseries progression and the latest status across multiple test runs of multiple test suites from the same milestone?

In our example scenario, say we have 2 test suites (Test Suite A and Test Suite B) each a test case. We run each test suite multiple times. Test Suite A’s test passes each time. Test Suite B’s test case fails in the first run, and we fix the issue, making it pass on successive runs.

When I go to the Milestone tab and bring up that milestone, the percentage there is less than 100%, because the first run of Test Suite B had a failure. That seems unintuitive to me. We fixed the bug, and the latest run of that test case passed. The listing of test runs in the Milestones page isn’t sortable, and has no clear indication of time progression.

It looks like there’s a way to get this view with a “Comparison for Cases” report (the “Latest” entries), but it’s only per test suite and we would have to generate a report per test suite per milestone.