Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Recommendation for multiple scenarios for a test case


#1

Often we have test cases that are defined that have multiple data points that need to be validated. For example, we have one test called “create a validate order”, and involves two different order types, for example “hardware” and “software”.

Each of these has the exact same steps, just different inputs and expected results. If it were just two points, I might think that we would just create two different test cases, but we have some scenarios where we need to test 40 different data points, and duplicating the same steps for each would be a nightmare when it came to making a change to those steps.

This becomes more important for us when we’re using TestRail to track automated data-driven tests, where results for 10 different scenarios pile up against one test case.

Is there a way to recognize multiple sets of test parameters or scenarios against one test case? In most cases these different data points would be specific to one test, so I don’t believe “configurations” would help us here.

Thanks


#2

Hello Chris,

Thanks for your posting. There are currently two recommended options to implement parameters for test cases. You can either list the parameters in a separate custom text box (‘Parameters’) or specify them as part of the structured Steps control. The second option would have the advantage that you could separately set a test result for each parameter/scenario.

It is planned to look into solving this problem as a general feature in TestRail (probably by specifying a parameter list in the test cases and then automatically creating multiple test cases on the test run side) but these two options are currently the recommended ways to implement this in the meantime.

I hope this helps,
Tobias


#3

Hi Tobias,

The separated scenarios that you describe sounds like it might be a good workaround, but it also sounds like in order to achieve this we would need to sacrifice separated steps, which may be a bad trade-off, especially if you’re looking to implement this in another way in the future.

Are we talking about a 3-6 month timeframe, or more of a 12-18 month timeframe for this type of support? (Not asking for a commitment, just balancing whether or not it would be worth the tradeoff based on time we’d need to wait to get it somehow else)


#4

Hello Chris,

Parameters/variances for test cases are currently not on our short term list so 3-6 months won’t be doable I’m afraid. While I understand that this feature is really useful, it is not being requested as often as other major features such as improved reporting or custom statuses and these features have a higher priority for us currently but we do want to look into this eventually.

Regards,
Tobias


#5

Great, thanks for being honest, it helps with our planning.


#6

I’m just getting started with TestRail. I have a bunch of test cases that have parameters. I was hoping there was a similar feature in TestRail. From this thread I guess not. Is there any word on if and when this might become a feature?


#7

Hi John!

Thanks for your posting. We currently don’t have an estimate for this feature but can recommend storing parameters in one or more separate custom fields instead:

http://docs.gurock.com/testrail-userguide/howto-fields

One approach that is often used is to store additional parameters and data in the steps field and this makes it very easy to run a test against different scenarios or with different parameters:

http://docs.gurock.com/testrail-faq/config-steps

Regards,
Tobias