Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.

Should One Untested Test Case Cause All Test Cases To be Untested Too?


I’m developing a Jenkins-TestRail Plugin that reports automated tests to TestRail. When a single test case has a status_id set to “Untested” (status_id = 3), it causes the entire test run/all of the other test cases to be Untested as well. It is an issue for me because I’d like to have all test cases to be Untested prior to a test run by default and their values be changed depending if a test case Passed or Failed. If a test case is skipped then it would keep the default Untested. I noticed that by having the test cases set to Retest (status_id = 4) instead of Untested, a single Retest test case would not cause all other tests to have the Retest status (unlike Untested). Is this behavior intended? Will be rolling with the Retest path for now but having Untested as the default would make more sense for me.

I am using the following command to send the test results (payload is an array of objects where the objects have a case_id and a status_id):
“index.php?/api/v2/add_results_for_cases/” + runId, payload