Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Recovery of results of test runs after adding an additional Configuration


#1

Newbie mistake here. After starting milestone test run I became more familiar with how to use Configurations.

So I had a Test Plan using a couple of configuration items. Today I realized I should have added a 3rd (OEM) configuration for completeness. BUT this was on a Test Plan that was completed.

Not realizing what this would do I added the 3rd config item and now all 274 results are gone (dumb mistake).

Is there any way to recover them?


#2

I think I can answer my own question. I ran sql queries on the DB in ‘tests’ and do not find any tests that were ‘tested_by’ with my user ID.

Surprises me that adding a Configuration can do such damage.


#3

Hello Paul,

Thanks for your posting. Adding additional configurations wouldn’t delete any existing configurations or test runs but removing the configuration from the test plan would also delete the test run. There’s one exception if the test plan doesn’t have any configuration so far and you switch to configuration mode but TestRail would always show a confirmation message/explanation in every case before applying any changes.

I’m sorry to hear that you lost test results and you can recover test results by restoring a backup (if you are on TestRail Cloud, we are happy to look into this for you if you could send us an email to contact@gurock.com). Please note that this would restore the entire database and revert any changes made since the backup was taken.

Cheers,
Tobias


#4

First this was my mistake. The GUI warns results will be deleted. (But I will argue the warning should be much larger and a better warning color. Maybe light red or yellow and not pastel yellow. See attached image.)

Also not sure your description is correct. In my case I had a Test Plan that had only one test run in it with 3 configurations assigned to it and I added a 4th. So before and after change there would on be one test run. Thus the deletion happens even with existing configuration when one is added.

As I also said in support email I would have liked that TestRail just voided the result records in the database and not delete them. Then I could have run a query to get my results and add them back manually.

Also as I said I will add a daily report to keep a record of my history. But TestRail needs to provide a way to bulk delete reports.

Thanks for the response!


#5

Hi Paul,

Thanks for the additional details! TestRail would consider these as two different configurations and would remove/readd the test run in this case (as the review dialog suggests). What I meant with adding a new configuration previously is the following. If you test against Windows 7 for example and then add a configuration from a previously included configuration group) (e.g. for Windows 8), TestRail wouldn’t remove the Windows 7 configuration/run and add a separate configuration/run instead.

Modifying an existing run and changing the included configuration groups and/or configuration selection for this run on the other hand would result in the behavior you’ve seen because TestRail can’t know if the results still apply to the original run (especially if you change multiple runs and not just one).

I hope this helps!

Cheers,
Tobias


#6

Yes, that makes sense and is the behavior I am seeing.

Since I require a bug report for every failure in a test run it was not hard to recreate the results and apply them manually. It just looks like I was supper productive for one day. :smile:

Thanks for the support.


#7

That’s great to hear, Paul :slight_smile:

Cheers,
Tobias