Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

429 Errors thrown more since 5.0 update


#1

I am using a hosted testrail version, and ever since the update i get 429 errors like no ones business.

I handle them and resend until it passes but it’s gotten to the point where it just ends up failing because i’m sending so many calls at once.

I use TestRail in a CI system that reports automation results and resets test runs so I send hundred of API requests in a matter of minutes.

I haven’t had any issues with it up until a few days ago when i got the new version. Is this expected? I feel like this is a bug since i didn’t have these issues beforehand.


#2

Hi there!

TestRail 5.0 dosn’t change the 429 behavior and this is expected if you send too many requests in a short time. You would need to wait with additional requests as explained in our documentation here:

http://docs.gurock.com/testrail-api2/introduction

This is a standard behavior of APIs to protect the performance of your TestRail instance and you would need to process the Retry-After header correctly.

I hope this helps!


#3

It’s hard for me to think that it hasn’t changed since the new update, i wasn’t even accounting for 429 errors before a few days ago (before 5.0) and it performed perfectly. Now I do the wait retry but I have lots of tests running parallel and some of them never ending up reporting results because they reach the limit of my set retries.


#4

Hello Ryan,

Thanks for your feedback. This hasn’t changed really and the 429 behavior is actually independent of TestRail and a specific TestRail version but part of the SaaS/Cloud platform we built for TestRail. Have you recently changed anything on your side? For example, have you increased concurrency/parallelism for your tests?

Regards,
Tobias


#5

Interesting, I’ll have to take a look, adding the retries solved my problem though. i guess it was just a coincidence I started running into it after the update.


#6

Thanks, Ryan. Please let me know in case you have any further questions, we are happy to help!

Regards,
Tobias


#7

Hi there,
I’m getting 429 errors as well. We have a number of apps integrated and generally this has not been a problem. It is odd that we do not consistently get 429 errors when we run our tests at the same time. Is the rate limit applied per customer or across the board? Is it source/ip based?
Thanks


#8

Hi Nelson,

Thanks for your feedback! The rate limiting is per TestRail instance/customer and not across all instances (so, customers cannot affect each other).

The rate limit is quite high actually and is not triggered very often according to our monitoring/logs. You can either handle the 429 responses (and wait before submitting the next request according to the Retry-After header), or prevent the 429 responses by slowing down your requests a bit. The rate limit also depends a bit on the type of requests you send and write requests are handled a bit differently than read-only requests (you can send more read requests than write requests). This would explain why you see a different behavior from time to time (assuming that your request behavior differs).

I hope this helps and please let me know in case you have any further questions.

Cheers,
Tobias