Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

New setup: Project structure


#1

I am currently running a trial of testrail and have a couple of questions about project structure in particular.

I’m working on a fairly large product that has a few distinct types of use cases, so I set up 4 separate projects for these major areas of the product, but that leads to a few questions.

Milestones are common across the product. So version 1.1 will have new versions of two areas of the product, and introduce a new area/use-case. In testrail, a milestone is a product-level entity, so I’ve just named them the same. That seems ok, but even though all projects have a “Version X.1” milestone, there are issues with tracking.

I’d like to create top-level test plans that span projects, and that doesn’t seem possible. If I decide to keep this structure, is there any way to manage test plans that span projects? If not, can I at least let the teams manage their test cases/suites/plans at the project level, and create a custom view to integrate the project-level test plan progress?

The reason I’ve chosen this structure is that one project for the entire product seemed too granular. I’ll have 40 people in a few different geographies working in a few discrete areas of the product, and that seemed to be a natural logical division within testrail too. It mirrors our boundaries in issue tracking, source code control, other tools, etc.

thanks.


#2

It looks like from the answer in this thread from a couple of weeks ago that I cannot achieve what I want: http://forum.gurock.com/topic/1487/feature-requestassigning-runs-from-multi-projects-to-single-milestone/

Hopefully I’m wrong, but if not, do people really use just a single project in testrail so that they can get cohesive all-inclusive reporting? There doesn’t seem to be enough structure within a project. I’d want to cluster by feature, or at least major use-case.

Let’s use Microsoft Office for example: One unified release, but really separate projects. But if you want unified reporting against a single release target, would you have to put Office, Excel, etc into one project?

I’m ok managing plans separately, as those are different teams. So maybe there is a way to get unified progress reporting that aggregates all the projects under one banner defined by their common (in name) milestone?


#3

Hello Tim,

Thanks for your posting. Yes, projects are separated in TestRail and you cannot create shared test plans with test suites from different projects. The same is true for other entities in TestRail (e.g. milestones or reports). There are several reasons for this. For example, this allows you to set up separate permissions and access rights for each project and also supports different field configurations and other customizations per project.

We recommend using test suites to group your features and/or different project areas and using sections in those test suites to create a hierarchy of test cases for each feature/area. This allows you to work in a single project and you can use shared milestones, test plans and also generate combined reports.

I hope this helps and please let me know in case you have any further questions.

Regards,
Tobias


#4

[quote=tgurock]Yes, projects are separated in TestRail and you cannot create shared test plans with test suites from different projects. The same is true for other entities in TestRail (e.g. milestones or reports). There are several reasons for this. For example, this allows you to set up separate permissions and access rights for each project and also supports different field configurations and other customizations per project.
[/quote]

I can live with separate test suites/plans/runs, but is it not possible to create an aggregate progress report with the custom report feature?

I suppose I could always create a custom report by querying the DB directly, but I’ve done that a lot in the past and was hoping I could get away from it with testrail. I wanted an off the shelf tool to avoid creating and maintaining my own. Although a direct DB query would solve my other issue in that I hadn’t expected to need licenses for non-QA users. But since we used the API to report details in failed tests, we’d like to point dev users at the results. (I haven’t looked at issue tracker integrations yet, but pushing all failures to Jira bugs would produce a lot of duplicates, so that’s not viable.)

If there is no way to generate an aggregate report, then that would be our only option.


#5

Hello Tim,

Thanks for the update and my apologies for the delayed response. Yes, reports only apply to one project and there are several reasons for this. For example, each project can have different custom field configurations (e.g., a different set of custom fields or even different settings for the same custom fields). As most reports can heavily be customized (including displaying or grouping by custom fields), it would be difficult or even impossible in some cases to generate reports that apply to more than one project. There are other reasons as well (for example, project permissions) and we recommend using a single TestRail project if you plan to generate reports that span all functional areas of your products/projects.

I hope this helps and please let me know in case you have any further questions!

Regards,
Tobias


#6

I reorganized my use of testrail, and consolidated all of my projects into a single project, now I have run into another problem with this constraint, where testrail does not permit (reasonably) using the same project layout that is reflected elsewhere in our processes and toolchain.

Previously in this thread I described working on a large product with a 3-4 main areas of functionality, and I had made those areas separate testrail projects. This made sense as it mirrored our organizational structure, but it also mirrored the divisions we have implemented in Jira. But I couldn’t create and report on test plans that spanned multiple testrail projects. Now that I have addressed that by consolidating into a single testrail project, I find that this seems to break the design of the Jira defect URL integration.

I must use urls that look like http://your-server/jira/browse/%id% and I can configure only a single defect URL for my project. But in Jira, my product areas are broken out into separate projects. So I really need to use separate URLs for projects FOO and BAR, like http://your-server/jira/browse/FOO-%id% and http://your-server/jira/browse/BAR-%id%

Had I kept separate projects for this one broad product, I could have configured a defect URL per testrail project. But I can’t create/manage and report on test plans that span multiple projects.

Are there any workarounds for scaling testrail up to larger products? It doesn’t seem to be possible to configure distinct defect URLs per suite.


#7

Hello Tim,

Thanks for the update. It’s currently not possible to use different defect URLs for a single project unfortunately but you could looking into writing an intermediate script that does this. Let me explain how this works:

  1. Instead of pointing directly to JIRA, you would configure the defect view URL to point to your script.

  2. The script would take as input the issue ID and additional context information that you pass in as part of the URL. This can be done by adding one of the following supported placeholders to the defect view URL (most likely %run_id%):

http://docs.gurock.com/testrail-integration/defects-urls#supported_placeholders

  1. The script could then determine the corresponding JIRA project (e.g. with the help of TestRail’s API, it could find out the relevant test suite) and redirect the user to the correct issue in JIRA.

Also, have you already looked into using our more advanced Push & Lookup integration with JIRA? This supports a more flexible project mapping between TestRail and JIRA (as you can freely select the JIRA project on the Push Defect dialog):

http://docs.gurock.com/testrail-integration/tools-jira-rest

I hope this helps and please let me know in case you have any questions!

Regards,
Tobias