Thank you for the post and in depth information.
When you automate a manual test case, what do you do with the manual case and what do you do with the automation case:
Many customers choose to keep the manual test and add a custom field that marks the case as automated. This makes it much easier to maintain your test case repository as you do not have to create an additional test case and you can always turn the automated status on or off if need be.
What do you use the reference field for:
The reference field can be used for a few different things. The most common use of the references field however is for linking a test case to a requirement ticket. i.e., JIRA ticket. When you have integration setup between TestRail and your defect/requirement tracker, TestRail will also allow you to either click and directly load the reference information or display the reference information in a popup modal window within TestRail.
Most automated tests are webdriver, api, etc.. but do you include unit tests in testrail:
You can definitely store your unit tests within TestRail. That being said, you should integrate TestRail with your automation/CI tool. You can integrate TestRail with basically any test automation tool and many of our customers integrate their automated tests. You would integrate test automation tools by using TestRail's API. You can learn more about the API on our website here:
While we don't currently have ready-to-use scripts for specific test automation tools, many of our customers use TestRail together with various test automation tools and frameworks such as Selenium, JUnit, Cucumber, many commercial tools, CI tools like Jenkins etc. It's also planned to offer more example scripts in the future. We also offer basic API bindings for various programming languages such as Java, .NET, Python, Ruby, PHP etc. via the above link. You might also find this article interesting with details of how other companies like the BBC integrate their automated tests with TestRail:
If I have a manual test that's been automated but not under each platform, what do you do:
In this case, you would need to have multiple versions of the test case, although you could always use configurations to create your test runs. You would need to go in and mark any test cases per configuration that were not automtaed as manual, but this could save you some time in the long term as well as eventual overhead within TestRail.
If you add automated tests to testrail that do not have a corresponding test case, where do you put them:
You would always need to have a corresponding test case within TestRail for each automated test. TestRail has no mechanism for storing testing information without a test case to store it in.
How do you work around being reliant on testrail ids:
TestRail is very reliant on IDs. TestRail uses IDs to help identify which test case is which, etc. You can always create a custom field that you input your own custom ID or marker data in and use that to sort/filter your test cases by.