I’m having an issue getting accurate testing reports for milestones.
I have set it up so that i have one or more test runs per sprint, and have these associated to Milestones (currently through sub-milestones per sprint, but that can be changed). The test cases assigned to each test run varies, but often the same test case are repeated in several test runs to verify fixes etc.
Now, when I look at the status for a given milestone, I am interested in the latest result for any one test case, but what I get is every individual result.
As an example, I have two test runs with 12 and 13 test cases. 4 of those overlap, so there are 21 unique test cases. When I look at the result for the individual test runs, I want to see 12 and 13 results respectively. And I do.
But, when I look at the result for the milestone, I want to see the results for the unique test (21), but instead I see 25. In my specific case, 4 of those first failed, but later passed. In the status report there will now always be 4 failed tests, giving the wrong impression to the stakeholders.
Am I doing something wrong, or should I associate my tests and test cases differently?
I have added an image describing my structure so that it is easier to understand: