Join 34,000+ subscribers and receive articles from our
blog about software quality, testing, QA and security.

Test Run: Add cases from multiple projects and/or suites


#1

I am enjoying the product and look forward to continuing my use of it. I have hit a bit of a stumbling point and would like your assistance:

  • How do I create a test run that includes test cases from multiple projects?

Today, we tested out a fix that impacted multiple areas of our product. We planned this out on our windows using ExpoNeon markers. Later, I wanted to “document” how we did this.

I decided to leverage the demo time I have with TestRail - I wanted to create a test run that essentially captured “what we did”.

However, I quickly discovered that I cannot (or cannot figure out how to) add a test case from Project X and Project Y into a single test run (which would actually represent what we did).


#2

Hello Knaledge,

That’s currently not supported as you can only start a new test run for a single test suite (and are therefore bound to a single project). You can use test plans to conveniently create test runs for multiple test suites, however. Do you use multiple projects in TestRail for your product?

Regards,
Tobias


#3

Hi Tobias -

As a bit of context, I initially created 1 project in TestRail, then 2 test suites:

  • Major product A
  • Major product B

Engineering (our developers) consider the “project” to be the overall version of the code base (XXXXX 4.1) - so I made that our “Project”.

During test run creation, I found that the section/sub-section are not retained when you drill down into a test case itself (when you “view” a test case within a test run).

So I asked… “Well wait… what if a test run has multiple instances of test case ‘Settings: Ensure settings save when edited’ ?”

Right now, it becomes difficult to tell where each test case lives in a suite, because section/sub-section are not displayed. So if we have one section called “Triggers”, then a sub-section named under “Triggers” named after each trigger, then identically named test cases within those sun-sections… none of that is displayed if those are added in a test run (if you click into the case itself to “run” the case).

But you do see the project/test suite it is in.

So I figured to create multiple Projects for each major component of our product:

  • Project A = Triggers; suite 1 “name_of_trigger”, suite 2 “another_trigger”
  • Project B = Product B; suite 1 “name_of_component” etc.

I still think adding test cases (from at least multiple suites) to a test run (if not projects). This is absolutely something that competing products do (see Contour and Testuff).

And really, I suppose printing the section/sub-section to the breadcrumb while viewing a test case is the root cause here :wink:


#4

Hello Knaledge,

Thanks for the explanation. Yes, a more complete breadcrumb would probably solve this and I agree that more context information would help on the test page. We will definitely look into this. I would recommend merging your test cases into a single project, and maybe even into a single test suite so that you can easily create test runs for your test cases.

Regarding the versions (4.1 etc.), have you looked into the milestone feature (which are TestRail’s primary feature to track and manage versions)?

Regards,
Tobias


#5

Currently, we do utilize milestones. We have major “Feature Branches” that tend to remain static for a very long time. We actually consider it our version of a “1.x, 2.x, 3.x” system.

That is all pretty standard fare.

I currently have one project with multiple test suites. Each test suite contains one of our two major product components. Each suite has multiple sections/sub-sections (as needed) to plot our product itself.


#6

I also discovered the ability to create a Test Plan and then add multiple test suites to that test plan. This then allows me to add cases from each test suite.

Still, the breadcrumbs and callouts would be much appreciated. Thanks!


#7

We will make sure to look into it.

Regards,
Tobias


#8

Understand that this is a v old thread, but curious if it’s now possible to add multiple test suites to one test run. I’ve basically got the same scenario as the OP.

Thank you in advance,

Charles


#9

Hi Charles,

Thanks for your posting. A test run is still linked to a single suite in a single project. We recommend using just a single suite per project and this is the most flexible approach when it comes to creating test runs and reports for the entire case repository or subsets of cases. Starting with TestRail 4.0, TestRail even made suites optional and now creates new projects with the single-suite mode:

I hope this helps!

Cheers,
Tobias


#10

Thanks very much Tobias, this is exactly what we decided to do. Sorry for the late response.

Cordially,

Charles


#11

Hi Charles,

Thanks for your feedback, great to hear this works for you :slight_smile:

Cheers,
Tobias


#12

This really would be a huge improvement, we want to use test suites to organize our test cases but we want them to all end up in the same runs. Is there some other level of organization that we could use that would allow us to do this?


#13

Hey Joe,

Thanks for your feedback! We would recommend using a single test suite/case repository in this case and one of the reasons for adding this mode was actually to make sure that you can include all cases in a test run or report. We wouldn’t recommend using multiple suites in most cases and you can achieve the same level of organization by using sections and sub-sections in a single suite/case repository.

Cheers,
Tobias


#14

I am doing a free trial of your software. I would like to be able to use tests from multiple projects in one test run.

We need to execute a set of tests for 2 projects. Some of those tests are unique to the projects and some are shared between the projects. We only want to maintain one set of the shared tests; we don’t want to have to create 2 sets of tests just so they can be executed for both projects.

Within those runs, we have tests that are unique to each run and some that are the same for each run
So, something like this:
TestSuite1 - unique to Run1 (project1)
TestSuite2 - unique to Run2 (project2)
TestSuite3 - shared in Run1 and Run2 (should be run for both projects)

How would you suggest these be organized using your software?