Got 502 response after very few calls

Question

This looks like a duplicate of Throttle violation after 300 calls but this one is reported as fixed in 2012.

Today I made ~450 requests with an app key (no oauth token) and suddenly got a 502 response:

throttle_violation : too many requests from this IP, more requests available in 86281 seconds"

Did I do something wrong? The same API key used to work some weeks ago and I made several thousand requests per day without getting any errors.

Update:

Most probably my fault but I'll need to wait another day to confirm it :) I added backoff handling code also to the problematic requests and still failed. Then I noticed that the filter I use only includes .wrapper.total and backoff field is filtered out.

Update 2:

This is still happening after I started using a filter that adds the backOff and the quota_remaining properties.

I created a log file with every request that my app made and the responses. There is not a single backOff tag and the quota_remaining is not 0.

File available here

After 644 requests I suddenly get a throttle violation error without any warning like backoff or something.

I'll create something like a unit test reproducing the issue but it would need some more time since I also need to wait for 24 hours again (or register a new API key but I don't want to abuse).

Update 3:

MCVE available at: https://github.com/vap78/testissue7255

Travis CI integration job that reproduces the issue: https://travis-ci.org/vap78/testissue7255

Additional finding - looks like that the ban is much less than the 86390 seconds returned in the error message. I used the same app key several times this morning and I manage to make ~650 requests every time before I get this error.

Update 4:

I think this is really a bug. I implemented a logic that sleeps for 5 seconds and then retries the request when this error occurs - something similar to the backoff handling which just sleeps for 5 seconds. The retry was successful and the test I implemented passed.

So my best guess is that this error is returned instead of a backoff message for some reason or no backoff is returned in the previous response.

Answer 1

Your app is subject to IP throttling as explained in the rate limiting documentation:

Every application is subject to an IP based concurrent request throttle. If a single IP is making more than 30 requests a second, new requests will be dropped. The exact ban period is subject to change, but will be on the order of 30 seconds to a few minutes typically. Note that exactly what response an application gets (in terms of HTTP code, text, and so on) is undefined when subject to this ban; we consider > 30 request/sec per IP to be very abusive and thus cut the requests off very harshly.

When I analyze your log file I see you managed to make 1289 calls in 1 minute and 37 seconds. While you are still within the 30 request per second, you're way over what the HA Proxy allows for any connection, coming from the API or not. You can find similar experiments to find the exact rate limit parameters on MSE but from my own experiments I find that several endpoints seem to have subtle different throttle rates.

The IP rate limit is separate and not linked or related to the dynamic throttle used by the backoff parameter. Not receiving a backoff doesn't mean you can't be IP throttled.

Pausing between calls, whether a backoff is received or not is the best advice I can give to prevent running into the IP rate limit.