Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Reporting with more depth levels


#1

Hi, I need to run a report and tabulate results across a few differet fields. Best way to explain is an example;

First, this report isn’t going to be reporting on execution, but rather on number of testcases.
In one of our reports, we want to run that divides our testcases by;
Milestone -> custom-field “Scrum Team” -> Priority

hence, you might have something such as a heading for:

Milestone1 -> Scrum Team 1 -> P1 = (total testcases that match)
Milestone1 -> Scrum Team 1 -> P2 = (total testcases that match)
Milestone1 -> Scrum Team 1 -> P3 = (total testcases that match)
Milestone1 -> Scrum Team 1 -> P4 = (total testcases that match)
Milestone1 -> Scrum Team 2 -> P1 = (total testcases that match)

Milestone 3 -> Scrum Team 2 -> P4 = (total testcases that much)

I’ve been tasked to just see how many testcases fall into the above examples, for instanced. Note; I don’t care about the actual testcases; just the numbers. This is a metrics-driven report rather than a user-informational report. We need it to see the employee count we are going to have in order to cover all the testcases; though these analytical data would be used for much more than that as well.

But I can’t figure out how I would even start to do this with the Available Reporting options. Or if it’s even a feasible thing.

Would this require me to export all of the testcases out to XML and use some external 3rd part reporting engine? Or is this something that can be done within Testrail?

I am using the Hosted version, so I don’t have access to create my own reports (something you may want to introduce the ability to do in the future). Or do you have another part to your product where we can request these things (and pay for the report) if we need them?

Thanks,
–c


#2

Hi Jim,

Thanks for your posting! If the Scrum Team field is a dropdown field, you can already use the Cases > Property Distribution report to calculate the distribution across this field. You can also add the Priority column to the tables to see the priorities. Calculating the priority groups per scrum team would be a manual approach in this case.

An alternative is to use the CSV export and include the Scrum Team and Priority fields, open the CSV file in Excel and then sort by these two columns.

Would either one of these be an option for you?

Cheers,
Tobias


#3

Yea, that’s what I felt was going to be needed so that’s what I went ahead and did. It gets a bit daunting when you are trying to do this with 15k testcases.

I guess I would recommend a feature much like Jira has where you can do custom queries based on the database values (fields); rather than the hard-coded ones included with Testrail. This would be an advanced feature, of course, but it would give us the benefit to do some queries w/o having to export to a Excel-based program to get these statistics.


#4

Thanks, Jim. Yes, a custom query engine is something that we have on our list as a longer term goal/feature we would like to add. That’s a complex feature and TestRail deals with a lot more data than JIRA usually (because of the test runs & results, JIRA only has the concept of cases/issues if you will) but it’s planned to look into this. Thanks again for your feedback.

Cheers,
Tobias