Join 34,000+ subscribers and receive articles from our
blog about software quality, testing, QA and security.

Random Tests in Test Run cannot be updated over API


#1

I am using the C# TestRail API “TestRail API binding for .NET (API v2, available since TestRail 3.0)”

Using this, I assign the Test Run ID, and individual Test IDs over the API to update test cases as automation runs.

Random tests in the test run NEVER update. While all the tests around them update. I have quadruple checked that the test ID is correct. I also looked at the metadata between working and non-working tests. There is no difference.

The way this integration is set up, I only declare the Test Run ID once, and then supply the Test IDs as they are satisfied. Sometimes, I provide several Test IDs in one POST. The issue still occurs if I supply any number of Test IDs at once. All but one Test may get updated, but the affected test will always fail to update, whether it is supplied by itself or as part of a list of Test IDs.

I can confirm that, at least as far as the API claims to work, that this is set up correctly, and that the IDs are all absolutely correct.

Now, I randomly receive “test run id not valid” responses from the API. However, the test run id most definitely is correct. I can debug and breakpoint, and before this response occurs, I can guarantee that the test run id is part of the payload, and that it is correct. And as mentioned earlier, I can send multiple Test IDs in one POST, along with the single Test Run ID, and 9/10 tests update, but one does not - and this error sometimes is returned when that 1/10 failure occurs. Then again, sometimes I receive a 200, regardless of the failure of the test to update.


#2

Does anyone have any ideas?