Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.

Editing test suites / associated tests in plan has no effect

I have some automation in place to create a new plan based on previous plan when a test is executed for a new version of the software. This automation reads the time-sorted list of old plans, matches the name to test parameters, and takes the newest match. It then create a new plan and copies all test runs and associated test cases to those runs.

Generally this works fine. Except now someone had deleted some of the test case definitions that the previous test plan refers to. I think they must have re-created them for some reason, and this resulted in the old test case definitions being gone and no test case exists for the previous plan. The results are there, since it includes the test case name etc. But for example, the results have no “test case” link for those test runs.

Now the automation has read the previous plan details using the REST API, and created a new matching plan. However, requesting the plan details for a plan that references a non-existing test case gives “None” for test case id reference (Python API call). This then results in the automation sending a create plan request with test case details referencing a “None” test case id.

I now have some empty test suites in the new automatically generated plan. I go and edit this plan (which is in open state). I associate tests to the empty test suites. In test plan view, this shows no tests still. If I go back to edit, all my changes are still there. So I can edit the plan with the empty suites, but saving it has no effect. Yet it seems to be saved since coming back to edit it shows the changes made.

This makes these plans worse than useless, since they are impossible to edit. I re-created the plan manually, after which I can edit this new plan as I expected.

What could be going wrong with this automated generation, why is TestRail completely broken on editing the generated plan?

Also, is there some log where I can see who has edited plans, test cases, test runs, test suites, …? Going back to forever?