Subscribe Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security. Subscribe

Report generation via API is useless?

So I tested the new Report API endpoints:

  • get_reports
  • run_report

But at this point can not find a way to ‘update’ a template before the report is generated to allow specific details about the run the report covers.

Even the most basic like add to the Report description OR even just change the ID of milestone in the Milestone Summary report.

Am I missing something?

What good is having a Milestone Summary template that always reports on the same milestone?

Hi @phawkins,
so I’m not with Gurock and I don’t want to defend them, but I have a problem with your post (probably you will have one with mine too).

Within this forum we have asked for an api support for the reports since a longer time. For me the current status is just a first step and neesd to be extended - I think every endpoint can be entriched with features too. Probably I’m too optimistic to get additional functionality, but we will see.

For sure, I also would like to update a report before exeuting it, but I’m fine to have the opportunity to run one via the api at all.

And yes, I several times had requirements to send around a report about the status for the same milestone day by day. Similar to a burn-down, the management want to have an idea about the progress. TestRail defines a timeframe for a milestone, so I can report frequently as well.

And finally: In general I would prefer to get access to the raw data to work with them in a different way, like in Excel or real BI tool. Depending on the requirements of the management.

So far my opinion…

Hi Kwirth,

I too have asked for a long time API support for reports. So good to see Gurock is making some progress.

Your reply does provide a small use case for multiple reports of the same milestone. I could argue the noise level would be too high and the intended audience would soon come to ignore them. :wink:

In my case to meet your need I have a guest account with only Read permissions that all vested parties can use to see progress at any time they need.

(Note the feature to allow Read access without burning a seat has also been requested for a while; and I think I just saw a new request for it go by in the last few days.)

Thanks for your feedback. So the feature is not completely useless; I suspect ‘mostly’ would be correct. :grin:

Hi,
you’re welcome.
Reporting is always a hard thing :innocent:. Some people (managers) refuse to access a TM-Tool and just want to have the figures, some don’t want to be irritated by receiving mails with reports. Usually I provide different solutions. :star_struck:
Using a separated account to access TestRail for the second group is not working because we used single sign on for all systems and asking one of the managers to use a different user is …complicated… :sunglasses:

So the requested feature is nice and has been asked several times before (new people just don’t check the forum for it and are happy to request a NEW feature). So let us put a vote on it and put and put and put.

My most important feature request:
A practical, reasonable and simple feature request and management process were the community is integrated would be great.

cheers
Karsten

1 Like

+1

The report generation via the API is not completely useless, there are cases where it works well, such as sending out a weekly report. Trying to generate reports via a Jenkins pipeline to automatically send test reports for particular code commits is where the current API lack of filter options becomes useless.

rshand,

Maybe it is not useless, thus the ? in my subject but I am open to good examples. You mention sending out a weekly report. A report of what? A Milestone summary? Surly no a project report.

As far a Jenkins goes I see no issue as a pipeline just calls a utility script that uses TR api. What issue in Jenkins am I missing?

Thanks for feedback.

Any feedback Gurock?

Hard to believe this has not been addressed before but I cannot find any threads on it.

Actually, we dont even use the API to create a weekly report. This is done using the scheduled report, so due to the API reports lack of filtering, we do not use it at all.

Our use case is as follows:

We have Jenkins that clones test plans to a sub-milestone relating to the code commit (the milestone is the feature branch that is being tested. When the Jenkins job completes, we want to send a report to the user who started the job, the commit-hash (sub-milestone) that was tested and only Failed or Untested tests. Currently the Reports API does not enable any filtering and therefore everything is rigid. Our only option is to send out a report to all users, for the entire Feature when the job completes. This spams developers and the report does not give any meaningful information as its usually only the latest commit result that is of use and not the total failure and untested of all commits within a feature branch.

So currently, we would need to create an API report manually for every feature branch, manually put in the email addresses of the users who are interested in that feature, then wait for the job to complete to get an Email, then manually set the commit-hash and regenerate the report. If the API supported filtering by email and milestone, then we could just create a single API report and Jenkins sets the parameters to automate the process.