Subscribe Join 180,000+ subscribers and receive articles from our blog about software quality, testing, QA and security. Subscribe

Group different test runs inside a test plan

Hi!

I’m doing system integration tests for a multi-component system which have large sets of component versions inside the same test plan.

Dimensions of the test matrix:

  • environment name (some specific configuration name of hardware+software as a pre-requisite for tests)
  • component1 version
  • component2 version
  • componentN version

Each “component” have it’s own test suite with large amount of tests.
Each “environment” can be used to run different suites for different component versions (for example: component1 v0.1 + component2 v2.1 + component3 v0.331)
Each “test plan” can contain multiple environments.

I’m using API call “add_plan_entry” and “update_plan_entry” at the moment to add new test runs/update existing runs, but it don’t allow to use different suites for the testruns, even different run names inside that entry disallowed.

I wanted to group different test runs for the components under environment names, for example:

envoronment1:
- suite for component1
- suite for component2
- suite for componentN

envoronment2:
- suite for component1
- suite for component2
- suite for componentN


envoronmentN:
- suite for component1
- suite for component2
- suite for componentN

Test Plan Configurations don’t help there because the configurations are allowed only inside the same test suite.

It would be nice to have possibility to group the test runs for different suites for the same “environment1” like this:
testrail-example2