Join 34,000+ subscribers and receive articles from our
blog about software quality, testing, QA and security.

Mark test suites "completed"


#1

Hi,

Just playing around with TestRail, in order to find the best configuration for our projects.
Seems that project structure of “Use multiple test suites to manage cases” is most suitable for us.

Unfortunately it seems, that test suites can only be marked as “completed” in baseline-supported projects.
Why is it like that? And is there anything else I can do about it? (Haven’t looked into the API yet)

Regards
Jens


#2

Hi Jens,

Thanks for your posting! Closing suites is only possible with the baseline feature currently. The workflow of baselines involves regular cloning/branching and completing versions (suites), so closing a baseline/suite is much more common than with regular test suites:

https://blog.gurock.com/test-management-test-case-versioning/

The traditional multi suite mode is no longer the recommended mode for most teams and you can implement the same functionality with the default single suite mode and an additional level of section + sub-sections. Using a single suite (case repository) is much more flexible and allows you to create test plans/runs and reports for the entire case repository but also a subset thereof. If you need more control and constantly test against multiple versions, you can also look into using baselines.

Cheers,
Tobias


#3

Hi Tobias

Thanks for your response.

That’s too bad. Having one test suite will still not allow to “fade out” sets of tests that are no longer, resp. currently not needed. If I pack them in a subfolder, I will have to uncheck it each and every time I create a test run. That does not sound handy to me.

I will look into baselines for a second time. Multi-suite mode is exactly what we need, except for the missing “complete” feature.

Heard you saying “is no longer the recommended mode” is giving me the impression that this feature could fall out of your maintenance support sooner or later. :frowning:

Regards
Jens


#4

Hi Jens,

Thanks for your reply. The multi-suite mode won’t be deprecated/retired and lots of teams still use this mode (the other modes were just introduced with TR 4.0). There are still use cases where it makes sense to use multiple suites, it’s just that the single suite mode works best for most teams and is therefore the recommended/default mode for new installations and projects.

Instead of closing suites, you can also just add [Closed] or similar to the title of a suite as a workaround and this would also indicate that the suite should no longer be used for new test runs/plans.

Cheers,
Tobias


#5

Hi Tobias,

Baseline supporting project template won’t work for us.

Maybe you can help me to figure out the best way for us to set up the system:

  • At the beginning of each iteration (be it a sprint, a release, etc.) we create our release specific test cases.
  • For some of these test cases there will be separate regression tests after the release, others will be abandoned.
  • In addition to these test cases we run regression tests while we are coming closer to the release date.
  • There are daily automated tests, too, which we are planning to add to the TestRail database.

This is what I thought:
Regression tests should go to one or more test suites organised by topic.
Feature tests should go to one test suite for each release, (e.g. Features Release 1.0, Features Release 2.0, etc.)
Automation tests are organised in one or more test suites by topic.

My problem is, that since I cannot close test suites, the list of suites is getting confusingly long. Do you have any suggestions?

Adding a [closed] string is just a workaround for a feature which I consider mandatory in a test case management system that claims to support multiple test suites.

Thanks in anticipation.

Regards
Jens


#6

Hi Jens,

We usually advice against managing different versions/releases with test suites and use TestRail’s release features such as milestones & sub-milestones (iterations/sprints) and versioning features such as closing test plans/runs. If you need more control (and this is usually only needed if you test against multiple versions at the same time), we usually recommend looking into adding baselines to the mix.

Closing test plans/runs archives the plans/runs so any modification to the case repository (such as adding/updating or deleting cases) does not affect those runs. So, it’s usually not needed to keep track of old case repository snapshots on the Test Cases tab (by copying test suites) and you can always look at old test runs to see previous test results. If you do need to test against older versions, we would recommend using baselines in this case.

I hope this helps!

Cheers,
Tobias


#7

Hola Tobias

I like the word “usually” in your formulations :slight_smile:
Like I said, baselines are not an option for us.
Thanks for your response. I’ll see how we gonna handle it.

Regards
Jens


#8

:slight_smile: Let me know in case any further questions come up, happy to help.

Cheers,
Tobias