Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

A challenge with creating a test report with pass history for each individual test result.


#1

I’m generating a test report where in addition to test result status, an history of execution is also included. So, in this manner we have an information about how test behaves over time in terms of flakiness.
The issue we are experiencing is that generating a report takes around 25 min for 3000 results, as each test case have to be interrogated individually (get_results), with a set of limit=5 i.e. look back and return 5 results for given ID. Every second there is approx 5 REST API call executed, this is taking too long at this moment and it’ll only get worse going forward.

Had a look on other available API and tried out the get_results_for_run, hoping that limit also applies to each individual test case i.e. how far to look back, however this is not the case and it only applies to all tests. So this is a no go.

What would you suggest?


#2

Hello, hope you have enjoyed the festive season and you’re back all rested with fresh ideas.
Could anyone have a look at the post and throw an idea?


#3

Hi Robert,

Thanks for the post! Apologies for the delayed reply. Currently there is not a way to set up a bulk API call that would provide more than the latest result for each call, so the way you’ve got it set up is likely the most effective way to run the reports within a single script.

The best recommendation I can think of would be to store the results after each call, and each time the script is run have it append the most recent results to an existing log, eliminating the need to query the last 5 results for each test.

Hope that helps! If you have any further questions you can reply back here or reach our support team directly at contact@gurock.com


#4

Thank you for response.
Will try some work around this limitation. Cheers.