I’ve recently worked on integrating our Specflow based functional test framework with TestRail so that it can create a test run and post results for the Gherkin scenarios to that test run for reporting. It is working okay but the suite that I create the test run from is not fully automated by Specflow. Currently when I create the test run using add_run I set “include_all” to true. This means that as I run the tests, those that are in that suite that are not automated remain “Untested” (sensible in a way I guess) while the automated tests are set to pass or fail as appropriate. Problem with this approach is that from my point of view unless I move out all non-automated tests from this suite the results and reports always a bit strange.
So I’m thinking of the approach where you add_run but set “include_all” to false instead. What I’d like to confirm is the behaviour regarding “case_ids” - if you start with an empty set “” in add_run is every subsequent update_run with “case_ids” going to be the union of all case ids from previous invocations of update_run? Or will I have to resend all previous case ids plus the new case id that I want to add when I invoke update_run? The online documentation for update_run wasn’t absolutely clear on this.
Hope I have explained myself clearly enough.