Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Creating a Regression Test Suite


#1

Hi,

New user to TestRail (Hi Dennis ;-)) and I see we have various types that we can select when creating our test cases. But I don’t see a way to set up a pick/choose option for test cases that we’d like to run as a regression suite. If there was a way to cherry pick tests from the listing in the test cases that could work. But I don’t see how this can be done without changing the Type of test case. Meaning if I have a sign-on test case in a test suite, I’d also like to have it (or be able to select it and others) in a Regression Suite. The only way I can think this is doable is to export out test cases you want for a Regression Suite, create a Regression Suite, import all these test case files (xml, csv formats) into Regression Suite, create a new Test Plan (Sign In Regression) and then add the Regression Suite to it.

Is that how some of you are doing it? Seems more steps than should be necessary.

Mike


#2

Hello Mike,

Thanks for your posting. Just to clarify, is this about adding test runs and using only regression test cases for the selection? If so, you could either look into using the Type field for this or add a custom field. The custom field can be a mult-select and many customers use this to implement some kind of tagging or similar. You can then filter for those test cases when adding test runs with the Select Cases dialog:

http://blog.gurock.com/postings/testrail-2-5-now-available/2151/

(see Case Selection Filter)

Would this work for you?

Regards,
Tobias


#3

Hi Mike,

Just a user here. What you may want to try doing is to create a test plan (instead of a test run). This will allow you to create a “Regression Test Plan Template”. In this plan you can add multiple suites and filter them to a select amount of cases as you desire.

Once your regression plan template is set up, if you enter that test plan you can click the “Rerun” button. This will allow you to create a new plan containing all of the tests/ test suites/ configurations you have used before.

Hopefully that helps.

Regards


#4

Hello Declan,

A test plan sounds like a good option here, thanks for your feedback on this.

Regards,
Tobias


#5

Hi All,

I’ll look into messing around with those on my test system. I think I still have a bit of time before the trial period on it runs out.

Thanks!

Mike


#6

Started a thread as it’s not the same issue but it is related. I created a project, then a test plan then created and added 2 suites to the plan. However since there are no test cases I went to that tab and imported one into a test suite. Upon viewing I am still seeing the same issue I posted about in this forum. Either I am doing something wrong or this software isn’t doing what it is supposed to do.

Mike


#7

Hi Mike,

You may need to think about the structure of your test suites to begin with. Basically, if you create test suites based on functional areas that relate to your application, then you do not need to duplicate test cases across other suites in order to track or maintain them as regression tests or functional tests etc.

TestRail test cases have the ability to have all sorts of information attached to them. We are using various custom fields that we can use to tag our test cases with various attributes - i.e. whether the test is a Regression test, whether it needs more information, whether it is automated, and so forth.

Once you have added your test cases to the test suites and tagged them appropriately, you can then create a Test Plan. From this Test Plan, you can click “Add Run” which enables you to select a Test Suite in its entirety (all cases), or you can “select cases”. From this dialog, you can use the filtering mechanism to select specific cases (i.e. those tests that qualify as Regression Tests) based on the attributes you have applied to your test cases.

You can add as many suites as you like to a Test Plan (as Test Runs), and filter each Test Run on the same or different filters if you like.

I highly recommend you take a look at the training videos that TestRail have provided (http://www.gurock.com/testrail/support/).

HTH,

Glenn


#8

Hi Glenn,

I’ve watched the pertinent videos multiple times looking for missing tidbits. What I would like to see (and feel would be a boon IMHO) is Suites outside of a project. Just the way I look at it. I could have 100 suites, add to a project those I need (including ability to drill down to only certain test cases from each suite) but I may be looking at it differently than others/most.

Again, new to all this as I was tasked with finding a tool and this was one of 3 that I felt did what we wanted (bug tracking a but kludgy since we don’t use an open source tool - GoToAssist is what we’re using but I got it to tie into our sections/services to match up with our projects in TestRail so my boss was happy with that :wink: ) so as long as he and I are happy/comfortable we’re good on this end other than some minor stuff.

But again I am trying to utilize the install locally to mess with this so I can get things better understood and show him (if he doesn’t beat me to it) how we can work with it more efficiently.

Mike


#9

Hmm, now it seems to have decided to work. The import fails because it is looking for the Case ID which isn’t there if the test cases had been imported previously (via Add) as it increases the Case ID.

So I deleted the test suite, re-created it, imported with Add option and all of a sudden it fell into place. Not sure what I was doing (wrong) yesterday…I will look into custom tags but not sure how I’d work that to make existing Type of Functional be a Functional Regression without changing the Type throughout a bunch of cases and still will require a lot of manual work. Hopefully with the 4.0 update around the corner (hopefully) with bulk edit that would make it more palatable/easier.

So to recap, I deleted test suite, recreated, import->Add with modified file (Type Regression) it worked this time. Will do again for other suite to ensure I see the same results ( worked!).

Mike


#10

Hello Mike,

Thanks for the additional feedback. The import functionality can either add new test cases or to the suite, or when you select the relevant option in the import dialog, update existing test cases. In this case the XML file needs to include the existing test case IDs in order for TestRail to know which cases to update.

Please note that TestRail’s next version (expected in the next 2-3 weeks) will come with a powerful bulk editing feature so you can directly make those changes in TestRail. Please note that you can also already copy test cases/suites between suites and projects via Copy/Move dialog.

I hope this helps and please let me know in case you have any further questions.