3

Update:

We've confirmed the issue... fixing it is taking more time than anticipated. – Kevin Montrose

Color me happy.


The throttle is now applied to api.stackoverflow.com so we have an undefined max requests per minute with an undefined burst allowance before 503 errors begin to be thrown.

It appears that the threshold has been raised significantly and interestingly it seems that the api.stackoverflow.com threshold is still higher than the other endpoints.

see What is the throttle interval or threshold? for current test results


Hey guys, we seem to be getting some love on the 503 throttle issue but need some help with some repro reports.

I have noticed just one other dev getting reports of 503 here, but trust me, if you haven't run into them yet it is just a matter of time and you will find yourself very frustrated and hacking in compensation code with something that does not resemble a smile on your face.

This is a critical issue and your help is needed to verify and prioritize this problem.

So please take a few moments and run this script while monitoring the requests in a tool like Fiddler for Windows or Charles proxy for Linux, MacOs and Windows to verify that the 503 is reproducible.

You don't need to download anything, simply paste this into a file and view it in a browser while monitoring via Fiddler or Charles.

An answer with your observations would be welcome but all that is really needed is a comment of repro/no-repro.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
    <title>NGINX Throttle Violations</title>

    <script src="http://soapi.info/code/js/stable/scripts/Soapi.js" type="text/javascript"></script>

    <script type="text/javascript">
        var apiKey = 'qgAq_KfDu0KYzlNG-qaTuw';

        // the error is easy to reproduce, but JSONP cannot handle http errors so
        // you will have to run this script and watch in Fiddler.
        // This behavior is quite consistent and if you are not seeing it then
        // it may be likely that your IP is excluded from the throttling.

        // set the queue interval from the semi-reliable default and painfully slow rate of 300ms to 10ms
        // to expose the 503 errors to Fiddler

        // comment this line to avoid 503 errors by throttling requests to the api to 1 every 1/3 second.

        Soapi.RequestQueue.setInterval(10);

        //Soapi.RouteFactory("api.stackoverflow.com", apiKey) // this endpoint behaves and dutifully returns 50 pages
        Soapi.RouteFactory("api.stackapps.com", apiKey)  // fails after 130 request burst
            .Tags({ pagesize: 1 })
            .getPagedResponse(null, null, function pageCallback(data)
            {
                // stop paged request after 50 pages or no more items
                return data.page == 500 || data.items.length == 0;
            });


    </script>

</head>
<body>
</body>
</html>

Results

Reference:

What is the throttle interval or threshold?

all api endpoints except api.stackoverflow.com are failing paged requests with HTML 503

stackauth throwing 503 (in html format) after just a few /users/{id}/associated requests

15
  • Hmmh, just gave it a shot with Chrome and IE and received in average ~54 responses with a 200 before it stopped, but I didn't receive a 503 next! I`m not exactly fluent fiddling with Fiddler options though, is there anything I need to to be aware of to surface a 503 in the UI, e.g. similar to 'Hide 304' (would be s strange default though)? Jul 16, 2010 at 7:38
  • @steffen - nope, if you can see 200's you will see 503 as shown in the image above. Interesting, so a no-repro from germany. I consistently dump out at ~30 (used to be 10) here in arizona. I wonder if there are multiple load balancers that service specific geo regions. Jul 16, 2010 at 8:03
  • @steffen - i know you have an eye for detail, but let me confirm with you that you are getting no 503 with a 10ms interval against stackapps? could you try upping the page count to 200 return data.page == 200... just to see what happens? Jul 16, 2010 at 8:24
  • @code - ah, have been on morning commute auto pilot still, you are stopping after 50 pages deliberately of course: 'Use the source, @Steffen, use the source' ;) Jul 16, 2010 at 8:28
  • @code - as I just typed while you've answered already: my eye for detail can be pretty weak early in the morning, else I wouldn't have asked the stupid question ;) You are spot on indeed, I'm getting 503 after ~75 requests now; and commenting the request queue interval does solve it as expected. Jul 16, 2010 at 8:45
  • @code - likewise the api.stackoverflow.com endpoint behaves correctly as you described and dutifully returned me 256 pages with throttling disabled again. Jul 16, 2010 at 8:50
  • @steffen - thanks much. interesting variance in throttle tolerance that might be significant. Jul 16, 2010 at 8:52
  • @code - indeed; it's kinda weird in the first place with api.stackoverflow.com behaving different: this should in theory make it much easier to pinpoint the issue, as the Stack Exchange team should now about any such difference; of course it might turn out the exact opposite with this being a strange corner case due to their entire network setup treating the high volume site differently, be it explicitly or implicitly. Would be nice to get an official acknowledgment/comment regarding this important issue at least for a start ... Jul 16, 2010 at 9:01
  • @code poet - think we've found the root of this issue. Should have a resolution tonight, maybe tomorrow. I'll post details once its in place (provided it works of course). Jul 16, 2010 at 19:02
  • @kevin - great. comment to @steffen here when it is in place and we will both bang on it. Jul 16, 2010 at 19:46
  • @code: Charles actually costs money! Let's say it all together now, "We want free stuff!" Jul 18, 2010 at 13:40
  • How about WireShark for Mac / Linux? Scratch that, WireShark is just fine for Windows too. Jul 18, 2010 at 17:30
  • We've confirmed the issue... fixing it is taking more time than anticipated. Jul 19, 2010 at 13:31
  • @geo - good catch on WireShark, completely slipped my mind as I don't use it much. Jul 19, 2010 at 14:06
  • @Kevin - thanks for the update and acknowledgement of the issue. Jul 19, 2010 at 14:08

1 Answer 1

0

We've published guidelines, and believe enforcement is working.

See this post for guidelines.

1
  • it does indeed seem to behave as expected and the rate is better than nominal. good work. Jul 20, 2010 at 3:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .