Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Using API to clone latest matching test plan

Hello,

I am trying to automate creation of test plan when a new release is made and tests are auto-triggered. To do this, I need to clone (similar to UI re-run function) the latest test plan matching a specific naming criterion. So I need to do the following:

  • Find a plan matching the parameters used to run test. If found, upload results for it.
  • If not found, create plan to upload using following steps:
    – Find the latest (created) test plan with specific strings in its name. For example, latest plan with “windows” and “firefox” in the plan name. Plan can be open or closed.
    – Create a new plan with specific name. For example, “Windows, Firefox, build 123”.
    – Clone every configuration in the previous plan (from step 1) to new plan.
    – Copy selection of enabled tests from previous plan to new plan.
    – Upload results to new plan.

I guess there is functionality in the API to implement these steps. However, a question:

What happens if multiple tests run in parallel and each one queries the API for the test plan? Will there be a race condition where multiple new test plans can be generated? Multiple tests may request for the matching plan, get response it does not exists, decide to create a new plan, and before doing so, another test also created it (resulting in two identical plans created) ? How to avoid?

Thanks,

Hi @uliuli,

TestRail’s API doesn’t have a pre-configured ‘re-run’ method for test runs and plans, however you should be able to achieve the same functionality via the API using the ideas you described.

However, if you do have multiple scripts running in parallel for the same process, there is the possibility of your script creating multiple test plans.

If two plans are created with the same details, this wouldn’t cause issues within TestRail itself, as each plan, run, test, etc. would have a unique ID. When retrieving a list of entities via the API, many responses would include the most-recently added item as the first result, so having two plans with the same details might result in your script submitting data to the incorrect plan or run.

With this in mind, if multiple similar test plans would be problematic, we would recommend ensuring your script does not create multiple plans in parallel or handles these extra plans to suit your needs.

Hope this helps,
Jon

OK, thanks for the reply. I think I will just try to create plans and upload to the latest created. The kind of race-condition that would create more than one plan should be very rare, and one can just delete the extra plan. Other kinds of tricks would just get more complex than needed.

After some API exploration, I see I need something to copy the configurations and other details from, when creating a new plan with the API. I see I can use get_plan(old_plan_id) to get all the details to copy into a new plan.

However, I do not see a way to simply request the latest plan(s). I can request all plans, sort by creation date, and use the newest. But if there are a large number of plans accumulated over time, it will cause large network requests and require me to implement pagination logic when going over 250 plans (API limit).

Is there some way to just request TestRail to give me the latest created plan for a project, or N latest plans created (with the API)?

Hi @uliuli,

It would be possible to limit the number of responses with the ‘get_plans’ method by using the ‘limit=N’ filter, which would allow you to limit the result to N results.

You can also use this filter combined with other filters to limit the number of plans received and require less processing.

Regards,
Jon

You did not answer whether N results was the latest like the OP asked. Or what, if any, order the plans are returned.

OP: I have an extensive CI/CD code to do what you want thru the TR API. So a word of advice: Use names mostly to create TR entities. Only use TR ID’s for finding and updating TR entities. All create API calls return the ID. Save it for future use. Especially when using Test Plans and not just Runs. Test Plans have Entities with Runs in them. Often the Entity ID is needed instead/including the Run ID.

1 Like

Hey there,

When using the get_plans or get_runs method it will return the plans in reverse chronological order, based on ID. If you use the limit=N filter it will limit the total number of results to the specified value, and you can also apply filters like created_after or created_before, and you can use the offset value to return plans beyond the initial 250 returned by the API.

I hope that clarifies things!

Thanks for the clarification and tips all.

Is my understanding correct now that if I ask for limit=N test plans, it will give me N latest created plans?

That is correct, generally speaking. It returns the plans in reverse order by ID, so the highest IDs will be at the top. Since IDs are issued in sequence chronologically as plans are created, this correlates to the most recent plans being returned first, and if you set it up with something like limit=10, you’ll receive the ten plans with the highest IDs, which will be the newest.

1 Like

OK, thanks. This should cover my use case.

Just if anyone comes reading, the answer is “yes, the returned plans are returned newest first”. :slight_smile:

1 Like