Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Versioning for test case?


#1

It seems that test cases are not duplicated over to my test runs. Here are the steps I am taking:

  1. Create a test run.
  2. Execute a test case within that run.
  3. Go back to the master test case and change some data in it.
  4. Go back to the test run and view the test case I had executed in step 2.

My problem is that the new test case is shown instead of the one that I originally executed. When I mark something as pass/fail I am saying that it passed or failed based on the steps that were provided at that specific time, not what they may change to later down the road.

I know there has been a lot of talk about this on the board. Are there plans to implement this soon?

Thanks,

Chris


#2

Hello Chris,

Thanks for your posting. Changes to test cases are propagated to test runs as long as the test run is still “open”, i.e. not closed. Once you have closed a test run (with the lock icon in the toolbar when viewing a test run), the test cases are copied internally and are no longer modified when you change the original test case.

I hope this helps.

Regards,
Tobias


#3

Hi,
I would like to bring up this topic again.

We have a setup such that when a product is shipped to customer, the QA team that was responsible for testing the product, hands off its test cases to the sustaining QA team. The sustaining QA team is responsible for bug fixes, patches, etc. During these sustaining releases, we just re-run the testplan that was handed over to them. As such we expect the closed plan to remain the same.

In my example, I created testplan-01 tied to milestone-1.
This test plan has only one test case (version 1)
I marked the test as passed and then closed testplan-01 and milestone-1.
I then changed the test steps for my test case. (version 2)
I then re-ran the closed testplan-01 on milestone-2

I was expecting that since closing a test run copies all tests internally, I would see the original version of the testcase in my testplan-01 for milestone-2

What is recommended approach for my situation where tests are handed off from team 1 to team 2 and team 1 can still modify the tests where as team 2 needs to run the handed off tests as is.


#4

Hello Ali,

Thanks for your posting. The close/archive feature of test runs only archives the test cases/test for the closed test run and doesn’t support re-running the test run with the archived test cases. New test runs (including re-runs) are always created for the current test suite and test cases.

To implement your scenario, you would basically need to make explicit copies of the test suites in this case. For example, you can:

a) Create a test suite with the same name and a version suffix
b) Copy the test cases from the original test suite to the new one with the Copy/Move Test Cases dialog

When you then create new test plans and/or test runs, you can choose the correct version of your test suite depending on the team.

I hope this helps and please let me know in case you have any further questions.

Regards,
Tobias


#5

hmm… seems like a very brittle approach. Seems like we would have to do massive duplication of test cases.

I am not a huge fan of versioning in test case management system but there could be good solution to this problem.

here is one approach

  • since testrail has archived the tests for a closed run, give the user the option to really re-run the testplan as it was archived

  • based on testcase modified date, identify test cases which are different from closed run and current testsuites. User can then select if he wants to use current copy or archived copy of test case. I think this is a much more efficient approach then the versioning approach used by HP Quality Center and TestLink.

Let me know what you think.

Ali


#6

Hello Ali,

Yes, we are aware of the limitations the current approach has with regards to re-runs and it is planned to look into this. There are a few ideas on how to improve this and among other options we think about introducing more explicit versioning support and/or support for simply re-running test runs against archived test suites. That said, the planning for this feature is still early and it’s not yet clear how/if/when this feature will be added.

Regards,
Tobias


#7

Has there been any further update to this request? We are experiencing the same problem. We are doing some rapid development and we have testers running through test runs on earlier versions of the software, while developers are updating the test cases based on changes made in later versions. Would be great if either the test steps in the Test Run did not change after the Test Run was created, or if there could be some kind of versioning for the test cases without having to make a duplicate set of the cases for each version of the software (we come out with a new version every 2 weeks so this would be VERY cumbersome).


#8

Hello April,

Thanks for your feedback on this. We don’t currently have an update to this feature request. Now that TestRail 3.0 is available, it is planned to look into this and other “structural” changes to make TestRail more flexible with regards to versioning etc. One simple and yet flexible option would be to add a new checkbox when creating a test run/plan which configures whether modifications to the test cases should affect the (active) test run/plan or not. We currently recommend using the explicit test suite copy approach as mentioned earlier in this thread.

Regards,
Tobias