Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Feature Request: Rerun Test Plan (configurations wide)


#1

Hi !

At that moment it´s possible to rerun a testplan, and in that specific testplan I can use filter actions ( e.g. “untested”) regarding the chosen testplan. When I work here with configurations (e.g. WIN XP/WIN 7/ WIN 8.1), I get only “untested” testcases which were not tested in ALL configurations at the same time. So for example I tested testcase 1 under WIN XP and WIN 7. When I proceed a rerun of that testplan with all untested testcases, I get for all systems (configurations) once again testcase 1, because it isn´t marked as “tested” under WIN 8.1.

It would be a great feature, if the rerun don´t show me
testcase 1 for WIN XP and WIN 7, but for WIN 8.1 (regarding the situation above).

Thanks,
Philip


#2

Hello Philip,

Thanks for your posting. This is by design actually as it can make sense to retest such test cases with all configurations. For example, if a test case failed for Windows 8 and you then decide to rerun the plan (after the issue was fixed), it’s usually recommended to verify the case again for Windows 7 and XP/Vista as well.

Regards,
Tobias


#3

Hi Tobias,

Thank you for your reply. There is no doubt that if one testcase actually fails, this testcase should be retested on all systems. In order to guard against misunderstandings,
I meant in my feature request the following scenario:

Exploratory testing, on multiple systems. I want to rerun different testcases on different systems.
In my post before I quote the status “untested”, not “failed”.

So, when I rerun a testplan, it would be a great feature if I got for each system (configuration) and not for all systems (configurations) the untested testcases.

When I work here with configurations (e.g. WIN XP/WIN 7/ WIN 8.1), I get only “untested” testcases which were not tested in ALL configurations at the same time. So for example I tested testcase 1 under WIN XP and WIN 7 but not under WIN 8.1. When I proceed a rerun of that testplan with all untested testcases, I get for all systems (configurations) once again testcase 1, because it isn´t marked as “tested” under WIN 8.1.

It would be a great feature, if the rerun don´t show me
testcase 1 for WIN XP and WIN 7, but for WIN 8.1 (regarding the situation above).

Thanks,
Philip


#4

Hello Philip,

Thanks for the additional details. Yes, I saw that you were referring to untested tests and the failed tests were just an example. TestRail currently handles all statuses in the same way but I agree that it can make sense to use a different approach depending on the status and we will look into this. Thanks again for your feedback on this.

Regards,
Tobias


#5

any chance this will be rectified in the near future? It’s frustrating to re-run a Test Plan and find that selecting Untested is carried into all configuration, including those already run.


#6

Hello Greg,

Thanks for your feedback! Yes, we might make this configurable in the future and/or add an exception for untested tests. I’m happy to add another vote to this feature request.

Cheers,
Tobias


#7

+1 to this. We have several teams running test plans with literally hundreds of configurations. The ability to retest only those cases that have failed would be extremely helpful.


#8

Agreed as well. We have both configurations to run against as well as release versions and we can easily cherry pick the ones we want to re-run. Ging in and removing the ones we don’t, however, is awfully time consuming. An option to configure the re-run in either fashion would be great, or even just alert flags that inform you of the ones that passed in a subset of configurations but not all of them so you can quickly scroll through and manually adjust just those could also be a nice feature.


#9

Hi All,

Thanks for your feedback! While it usually makes sense to retest the full feature on all configurations as the fix/changes could’ve affected other platforms/configurations, we also agree that it would be helpful to make this configurable for certain situations as well. We still have plans to review this, and I’ve added your votes to this request.

Regards,
Marco


#10

Hi,

I would definitely love to have this feature added! At times, we don’t always use ‘Configurations’ as operating systems but also use it for Product specific tests. Either way, I think it should be up to the user to have the option to run only ‘Untested/Failed’ or ‘All’ testcases for various configurations.


#11

Hi William,

Thanks for your feedback! I’ve added your vote and feedback to this request as well.

Regards,
Marco