We have started using TestRail for one of our product releases as a small pilot project evaluation… So I just wanted to share some of our working notes and thoughts.
As the job of any QA is to find bugs, the form of these notes will just be highlighting the ‘bugs’ or ‘feature requests’ that we have encountered while using TestRail:
- It is great that whenever you hover over a reference link, you see the status of that link, but would be even better is if you didnt need to hover over it to see the status. If the status could be indicated by an icon or a colorchange of the link then it is easy to get an overview of which references in a list of test cases, that are closed - ie. ready for testing.
- Internal links. So that you can reference test cases to each other instead of 3rd party bugtrackers.
2.1 Another approach could be ‘Shared Test Cases’ that can be accessed from different projects. And updated from either.
- Being able to use Issue links inside the Description field of a Test Plan. So that you can also hover over links in here to see their status.
- Being able to deprecate Test Cases instead of just deleting them would be good to enable us to do continous housekeeping without loosing test results that have already been executed and logged.
- When batch-editing test cases, it would be nice with an ability to select to use the Steps from case 1 and apply these to the other tests. Right now you need to re-write the Steps you want to apply to a batch of test cases.
- When you select test cases for a configuration, it would be nice if the popup window automatically opened in the same section as the testcases selected in the Test Run from which the configuration was drawn.
- When you have selected test cases in the selection window, it would be nice if you could just hit Enter on the keyboard to add them instead of having to mouseclick the OK button.
- It would be nice to be able to resize the Configurations window. We have a lot of configurations and scrolling can become tedious.
- When selecting and configurations and clicking OK to create all the permutations, it would be nice that NO test runs are selected by default. In most cases we only want to run 10%-30% of the possible configuration permutations.
- Dont use more than 2 Configuration groups at a time. If you need to make a 3, 4 or many-dimensional configuration, then create a new group where those are concatenated on their own lines.
- When adding configurations to a test run, start by deselecting all configurations and then add sequentially the ones you want.
- We need to define a common mandatory syntax for Version strings so that we do not confuse things. Ideally the Version string should ALWAYS include the product key.