I am rather new to TestRail.
I have observed the feature to re-run a Test Plan, but I found the following limitation: The re-run just takes back same list of test cases, instead of re-applying the queries/filters that were used to bring the test plan scope together.
In practice it means that I prepare a Test Plan for all my auto tests (filtered based on the test type, and whatever other criteria may be relevant).
2 weeks later, I will rerun all my auto tests… But if I take the re-run option, then all the tests that have been automated meanwhile are not included…
So, every time I re-run my auto tests (and the purpose is to run them often),… I have to manually recreate a test plan (i.e. apply the right filter for each test suite separately)…
Honestly, this feels like a nightmare…
–> Anybody got a smart way of doing this? I can’t imagine I’m the first one in this situation, but I couldn’t find any answer on the forum (sorry if my search was insufficient).
–> Or should I automate the creation of the test plans through API? assuming API offers the feature to create a Test Plan with filters…
maybe I’m wrong, but I think you’re talking about to re-run a test run as part of a test plan.
Usually runs are created based on a whole suite or a selection of cases. The selection is done manually, optionally supported by filtering the suite content based on different criterias. You can use more than one filter to enrich the run with cases - one after the other.
There is not only one filter criteria and it is (they are) not stored with the run.
The re-run function is to create an exact copy of the former run, e.g. for regression test purposes - nice if you need it.
Although I can follow your need, TestRail is not working in your way. It is first of all for manual testing (sorry Gurock). Adding the results from test automation needs additional effort or a smarter solution.
By the way, check the forum for that. There are a lot of postings about it.
Of course you can automate the creation of a test plan and it runs via the API. You can use get_cases for a suite to retrieve all cases following defined filter criterias. If they won’t help you probably have to iterate through all of them and check yourself.
Or you create the run based on your automated tests. Just an idea…
by taking a look at the Beta-Tester request you should be lucky, your feature seems to be part of next version.
Thanks for your feedback !
Meanwhile, we did the perparations to automate the creation of our test plans through the API. Nevertheless I’m glad this will be in next version as I do believe this will prove convenient for many more users in future