We run most of our tests in the evening and, when they run, they automatically send the results to the TestRail API. The vast majority of our calls work fine, but some come back with 504 errors with no response body that might illustrate what went wrong.
If we rerun the exact same tests and send the exact same results, it works fine. We’ve added extra logging to our requests and the requests themselves seem to be formatted properly, so whatever is going on would seem to be an intermittent backend issue in the API. We have seen this error on creating test runs, uploading results to a run, and closing runs, but it may be happening in other places too. It usually happens in the neighborhood of 7-8PM MST. We see this as little as once every few days, to as often as several times in a single night. Interestingly, we never see it in the daytime, though we hit the API far less often in the daytime.
It’s becoming a bit frustrating to come in to work and see red tests only to find out that it’s because the TestRail upload experienced another intermittent failure. To the community, if any of you have dealt with this… if there’s something we can do differently, please let me know (e.g. running at a different time to avoid backend maintenance tasks, etc.).
Otherwise, we would appreciate it if the gurock team could investigate and address the root cause of these intermittent failures.
Below is an example of a request that failed with a 504:
Request method: POST
Request URI: https://etw.testrail.net/index.php?%2Fapi%2Fv2%2Fadd_run%2F1
“case_ids”: [27, 28],
“name”: “app.hermes.plans.CrudPlans - 2018-07-13T02:45:34.828309”