I'm doing some testing on my app, and I let it run on a machine this afternoon while I was away. When I got back, I was geting 503 throttle violation errors from my API calls.
I was calling the "questions" endpoint at a rate of once per minute, passing a "min" date value to limit the returned questions to just those that have been submitted since my last poll. In the case where "has_more" was true in the response, I was retrying (with the page incremented) at a rate of once every 5 seconds. To my understanding, this is far less than the established maximum number of requests per unit time. I believe I was at about 300 requests total when I hit the throttle.
I've also got code in place to honor the "backoff" attribute if it was present, but that code was untested and it's possible it doesn't work.
I lost the console output, so I don't know exactly what happened when the feces impacted the high-speed rotating blades. Now, all of my requests return 502 errors, even though it's been about 2.5 hours since the "event."
Request url:
https://api.stackexchange.com/2.0/questions?sort=creation&pagesize=5&min=1325870128&site=gaming&order=asc&key=XXXXXXXX&page=1
Where XXXXXXXX is my API key. I can't get the quota since all endpoints are returning the same throttle violation error. In earlier testing, it was reading a max quota of 10,000.
Specific data that is being returned:
{u'error_id': 502, u'error_message': u'too many requests from this IP',
u'error_name': u'throttle_violation'}
Here's the API usage graph for my app, I can't grab the entire runtime graph (resolution issues), but there should be enough of it there to ensure that there's nothing odd going on:
What's a good way to debug this issue? I haven't found any good tools for showing a spike in my API requests at any higher resolution than daily. The data I can get back from the API is pretty much just "you were bad, go away" which isn't terribly helpful either.
UPDATE: At about 19:10 UTC Jan 6, the condition cleared and API requests are succeeding again. I'm going to let it continue to run to see if there's further issues, but I'd still like to root cause why this happened so I can avoid it in the future.