Join 34,000+ subscribers and receive articles from our
blog about software quality, testing, QA and security.

Mixing automated and manual tests in the same project?


#1

What is the best or recommended practice for mixing automated and manual tests in the same suite/project? We have a project containing only automated tests currently and these tests are synchronized with our automation tool (Robot Framework) via case id’s in the test tags. We try to automate as much as possible but obviously there are some kinds of tests that are not a good candidate for automation. I have two questions:

  1. Is it typical to have a mixture of test types within one suite?
  2. Is it typical to delete manual tests from the suite after the sprint is complete and these test no longer add any value?

Thanks!


#2

Hi Tracy,

Thank you for the post. In this case we usually recommend creating a custom field that houses the type of test you are running. i.e., automated, manual. You can then set this on each of your test cases and include this column as well so you can sort based on the test type. In fact, we have quite a few customers that do this as well.

http://docs.gurock.com/testrail-userguide/howto-fields

Let me know if this helps.


#3

Hi Marty,

I’m not sure I understand why you would need a custom field for this when test cases already have a test type? My question was more about best practices for mixing test types. Are there any examples anywhere of how others organize their tests to differentiate between automated and manual?


#4

Hey Tracy,

Thank you for the follow up. There are several ways customers do this.

  • Using the Test Type field to define each test type
  • Keeping automated tests and manual tests in their own respective sections within the test case repository

These are the two most common ways of doing this that we have seen customers set up their test cases.


#5

Hi Marty,

Are you therefore saying that some customers create a custom field in order to keep Automated and Manual tests in their own respective sections within the test case repository?

This would be interesting, I hadn’t really thought about this approach to arranging test case sections.


#6

Hey Simon,

Thank you for the follow up question. I was more so saying that you could use the Test Type field to define your automated and manual tests and on top of that within your repository, you could section off your automated and manual tests using subsections.

Does that make sense?

With what you are saying however, you could do the same thing and then filter on that field.