Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

How to setup with a number of tests run against different files many times


#1

Hi
(Couldn’t see the answer on search)

We wish to run a number of test 10 - 20, against different files,say 10 a day for the test of the year. The different files are selected each week as they are a sample of files from the week.
How do I set this up in TestRail?
I’ve thought that maybe:

  • The Different files can you entered as Configurations (we normally just use this for Browsers)
  • We could maybe split up the testing by week or month, create each month as a different Test Plan. With lots of small test runs

Any ideas how we can create this without it getting messy and time consuming.

Thanks


#2

Hi Stuart,

Thanks for your posting. If the tests (of a run) are all run against the same file(s), then it might be sufficient to add the file names to the description of the test run and this would be most the flexible option. Would this work for you and your team?

Cheers,
Tobias


#3

Hi Tobias
Thanks for the quick reply.

The scenario would be.
Uploading 100+ Audio files a week.
Test a % sample to ensure the data is correct.
The file names will not be similar, so can’t do
"Test Run 1 - File 001 - 020"
“Test Run 2 - File 021 - 040”

If we are using the same 10 tests, when we create Test Run 1, Test Run 2, is there a way of automatically changing the Test Titles.
The thought is, when running reports on failures, how easy would it be to spot when tests failed to which files.

Thanks


#4

Hi Stuart,

Thanks for the additional details. Our recommendation would be to add the file names to the description of the test runs (Description field when adding/editing the test runs). This makes sense if all tests (in a run) are run against the same files and you wouldn’t need to manage the file names per test/result. Not all reports show the test run description but you can always jump to the test runs (from the reports). Alternatively, it’s also possible to use the CSV or XML exports to create custom reports outside of TestRail.

I hope this helps!

Cheers,
Tobias