Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Feedback after a week


#1

Here’s some feedback we captured after using Test Rail for a week:

  1. A “variations” function that would automatically spawn new test cases. For example, the ability to create a core “scenario” and a list of variations on that scenario (i.e. browser options, or operating systems) and then it would auto-generate test cases for those variations.

  2. The ability to import from Excel or other file format through the interface

  3. API to update test results and test status from automated script

  4. Way to add test cases and test suites to test runs across projects

  5. An “import test case” feature for when you want to run the same test case in two suites

  6. A “matrixify” operation that lets me specify a list of conditions that I’d like to see each test case under. The system would expand the test cases by each item in the list. Similar to item 1, except that instead of providing a single scenario plus variations, this would be to provide two independent variables (a matrix) and have it auto-generate test cases from that.

We’ll post more as we encounter them…

Overall, the team is really impressed by the usability and stability of the product. For an initial release, it is very stable – no broken links, no obvious misses, etc. And the interface is very intuitive. At this point I’d definitely recommend the tool to anybody looking for a web-based test case management system.

Andrew


#2

Andrew,

Thanks a lot for the useful and positive feedback. Some of the issues you posted are already on our feature request list and will be implemented eventually. Please see below for answers to the individual points:

We do want to include some kind of functionality to ‘multiply’ a test case / test for different configurations or environments, but aren’t sure yet what’s the best way to implement this. Would it alternatively work to separate configuration dependent and independent cases in separate suites and just create multiple runs for the suites?

E.g., if you have your browser dependent cases in a separate suite, you could create a run for each browser you are testing against (and name it accordingly). I understand that we need to make it easier to start multiple runs at once to make this easier and we are looking at different ways to accomplish this.

[quote]2. The ability to import from Excel or other file format through the interface
3. API to update test results and test status from automated script[/quote]

We will have an API eventually, but have decided not to include it in the 1.0 in order to release the product earlier and get more feedback. When we have an API (we haven’t yet decided when we will start working on it), an easy way to provide an import functionality for different formats (CSV, Excel etc) would be to offer a separate import tool.

[quote]4. Way to add test cases and test suites to test runs across projects
5. An “import test case” feature for when you want to run the same test case in two suites
[/quote]

A way to clone/move cases between suites and possibly projects is already on our feature request list and it looks like it’s an often needed feature.

This is also something that we would like to handle with the configuration / multiple run support. I’m not sure if this would be suitable in your situation though or if we really need a way to easily generate new test cases.

Again, thanks for the useful feedback and please feel free to post any additional feedback or comments.

Regards,
Dennis