Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Ability to find tests not run in last n days


#1

Before a release we wish to list all tests that haven’t been tested in the past n days (60-90 days in our case) so we can identify focus areas which are more likely to have regression bugs. With constant product updates coming in we may have 2-5 results per test case.

There are filters to find tests that were “Tested On” a custom date range, however, because we pass tests multiple times we can’t use this filter to find tests not run in the past n days.

EX: We run test A on Jan 13, March 22, and May 15th. How do we rule test A out but include test B which was run on Jan 13, and January 29th. In reality we want the opposite of Tested On. We want to find tests that were not tested between March 1st - Today.

Is there a way to do this?


How to add test cases to a run based on last execution/run date
#2

Not sure where the “Tested On” field comes from, is this a custom field? If so, how did you set it up?

Assuming that you have this field with good values and you want to select everything except test cases that have a field value that meets a given criteria - try this, when creating your Test Run “Select All” (top left of Select Cases dialogue) then, in the Selection Filter, put in the criteria that you don’t want your test cases to meet (in your case it would be Tested On = last 60-90 days).

Here’s the trick - the button at the bottom of the Selection Filter section is actually a drop-down of possible actions. Drop it down and click “Remove From Selection”.

Viola - your selection now contains only test cases that didn’t meet the criteria.


#3

Thanks MPayne,

Actually I’m not looking to create a new test run, just want to filter the
one I’m working in. When looking at 2k test cases, I wish to see the 200 that haven’t been run in the last 60 days. It seems like testrail is more run based, where as, I’m looking to work with a single run for 3-6 months. Some of our tests are month long tests so in order to get 100% coverage of the entire run (with constant code tweaks coming in), we run some sections a few times as changes come in, but still use 1 run to track our tests.

Maybe the answer is lots of smaller runs but I sure hope not.


#4

Hello jsmo,

While not specifically designed for this, have you taken a look at the Status Tops and Comparison for Cases reports under the Reports tab? They can be helpful to identify test statuses over time and to identify which tests haven’t been executed across multiple runs. We are also happy to look into a specific report for this and I agree that this would be useful to have.