This isn’t specifically a TestRail question, more of a general process question… but it’s relevant to how I will ultimately use TestRail:
Say you have Product A (version 1.0), and you’ve created 200 tests for it. Since the product already existed, those 200 tests are presumably regression tests, because we could run them over and over any time new features are added to the product, to ensure the old features still work right.
So… let’s say a few new features are added to the product for version 1.1… and we create 100 new system test cases for that release, in the same suite as our 200 existing regression test cases. Now we have 300 test cases.
When the time comes to test… we create a TestRail test plan which has several configurations in it (multiple OS/browsers, etc.)… some for the 100 new system tests… some for the 200 regression tests. We use the test type field to differentiate between the regression & system tests so we can select them into the correct run.
All tests pass, and version 1.1 goes to production. Do we change the test type of those 100 new system tests we wrote from system to regression now?
Say the business wants to add more features for version 1.2, and you write 100 new system test cases for those new features.
For the version 1.2 test run… will we be executing 100 system tests plus 300 regression tests (200 from v1.0 + 100 from v1.1)?
Do we always keep converting our system tests into regression tests whenever the features are deployed to production? If so… what kind of heuristic do you use to select which regression test cases to run, since the regression pool keeps expanding with every new release?