Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Reports with test history


#1

I need to generate reports after each test run / plan that shows a full history for each test case.

So I need to show if a test case was executed 3 times, failed once, passed twice.

I currently can only make the reports show the final result so I executed 12 tests and they all passed. I need the entire run history.

Thanks


#2

Hello,

Thanks for your posting. The summary reports (e.g. Summary > Plan on the Reports tab) include an Activity section which includes all test results (not just the latest) and also with the standard activity chart. You can alternatively use the print views which include the full history for each test (click on the printer icon when viewing a test plan and then switch to Details mode).

Regards,
Tobias


#3

Yes, either of your suggestions does give you the information, but not in a easily viewed/digestible format. How about a format where the result instance status is in columns, where each column is either

  1. By Day - with the latest result of that day shown
  2. By Result - with the tested on date in the status box
    Something along the lines of: https://wiki.jenkins.io/display/JENKINS/Test+Results+Analyzer+Plugin

Alternatively, a new test plan for each execution could be built, but then the Test Runs & Results tab gets unruly quickly, polluted with many instances of test plans that represent one shot runs. Aggregating and filtering would be nice on this page nonetheless.

Maybe I don’t understand the recommended approach to using testrail for automation. Documentation seems to be lacking for that use. Sure, there is lots of documentation on pushing test results, but there is a lack of recommendations on whether those should be added results to a constantly open test plan or a wholly new test plan. If it should be a new test plan then integrating manual and exploratory results becomes problematic. Anything that separates test types into different test plans seems to drive a wedge into a wholistic picture of test status. Maybe there needs to be a notion of an integrated test plan that aggregates results. IDK, but whatever way we approach this, there seems to be glaring holes in the tool.


#4

Hi Darren ,

Thank you for the post. In this case, if TestRail is not giving you data in a format / view that you are wanting, you could export your test run data and run your own custom reports against it. If you are using TestRail Server, you can also develop your own custom reporting plugin.