Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.

[API] 504 gateway timeout during long running job


I have a job that gets all runs for a project, and then calls get_results_for_run for each one. The job sleeps 3-5s between each call so I don’t hit the API limit. But, the job will not complete if I try to get a large amount of results.

It runs on average 1.5 hrs and then fails on a 504.

The job works if I filter to a smaller number of runs, like around 50, but does not work when the run count is 1000 or more.

I can filter down my job and run on smaller sets of runs, but if I want to pull them all it becomes very manual. Do you know why the api may be giving up after an hour of a steady drip of calls every 3-5 seconds?