6

Update:

I generally try to back any claims of bugs and shortcomings with evidence.

I have prepared a test page which you, dear Stack Overflow staff and my fellow developers, can use to ascertain for yourselves whether the api throttle is performing to spec.

You can find these tests here: http://soapi.info/Code/JS2/latest/Tests/ThrottleTests.htm

If you choose to run the tests please use the 'Upload Results' button to upload your results into the aggregated dataset which can be viewed/downloaded here: http://soapi.info/Code/JS2/ThrottleTestIndex.aspx


As of 9/5/2010 I am still not satisfied that the throttle is performing to spec. It seems that we may be edging towards the published specification of 30 requests per 5 seconds window, but the results are inconsistent, with the most reliable rate @ 30 requests per 6.5.

I raise this issue not because I am obsessed with making the most possible requests per second as has been repeatedly implied. I raise this issue because as software developers we deal with specifications. We spend valuable time writing code to specification to ensure the best possible quality of code and user experience. If the upstream data source our apps rely upon does not perform to specification we all suffer - our users are denied the reliable awesomeness of our efforts and are likely to judge those efforts and find them wanting regardless of true merit.


original post

The throttle is acting up again.

Request rates are well within published guidelines.

see https://stackapps.com/questions/1433#1448

GET /1.0/users/moderators?key=my-key&jsonp=Soapi._jsonp133 HTTP/1.1
Accept: */*
Referer: http://localhost:17742/AlphaQuickStart.htm
Accept-Language: en-US
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; Tablet PC 2.0; .NET4.0C; .NET4.0E)
Accept-Encoding: gzip, deflate
Host: api.stackapps.com
Connection: Keep-Alive


HTTP/1.0 503 Service Unavailable
Cache-Control: no-cache
Connection: close
Content-Type: text/html

 
  
    Too Many Requests - Stack Overflow
   
  
     
          We're sorry...
          

There are an unusual number of requests coming from this IP address.

To protect our users, we can't process any more requests from this IP address right now.

We'll restore access as quickly as possible, so try again soon.

If you believe you have reached this page in error, contact us.

The nginx proxy seems to have been removed so either there is no api throttle or there is a new throttle that does not work.


reports of successful tests are puzzling.

01 throttle: 2010-07-23 23:17:31.458 - request dispatched: interval: 1282630651458ms - http://stackauth.com/1.0/sites?key=xxx
02 throttle: 2010-07-23 23:17:31.680 - request dispatched: interval: 222ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&key=xxx
03 throttle: 2010-07-23 23:17:31.696 - request dispatched: interval: 16ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=1&key=xxx
04 throttle: 2010-07-23 23:17:31.706 - request dispatched: interval: 10ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=2&key=xxx
05 throttle: 2010-07-23 23:17:31.715 - request dispatched: interval: 9ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=3&key=xxx
06 throttle: 2010-07-23 23:17:31.724 - request dispatched: interval: 9ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=4&key=xxx
07 throttle: 2010-07-23 23:17:31.733 - request dispatched: interval: 9ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=5&key=xxx
08 throttle: 2010-07-23 23:17:31.741 - request dispatched: interval: 8ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=6&key=xxx
09 throttle: 2010-07-23 23:17:31.749 - request dispatched: interval: 8ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=7&key=xxx
10 throttle: 2010-07-23 23:17:31.756 - request dispatched: interval: 7ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=8&key=xxx
11 throttle: 2010-07-23 23:17:31.765 - request dispatched: interval: 9ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=9&key=xxx
12 throttle: 2010-07-23 23:17:31.775 - request dispatched: interval: 10ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=10&key=xxx
13 throttle: 2010-07-23 23:17:31.786 - request dispatched: interval: 11ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=11&key=xxx
14 throttle: 2010-07-23 23:17:31.796 - request dispatched: interval: 10ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=12&key=xxx
15 throttle: 2010-07-23 23:17:31.807 - request dispatched: interval: 11ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=13&key=xxx
16 throttle: 2010-07-23 23:17:31.820 - request dispatched: interval: 13ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=14&key=xxx
17 throttle: 2010-07-23 23:17:31.833 - request dispatched: interval: 13ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=15&key=xxx
   throttle: 2010-07-23 23:17:31.846 - active queue full
18 throttle: 2010-07-23 23:17:31.871 - request dispatched: interval: 38ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=16&key=xxx
19 throttle: 2010-07-23 23:17:31.885 - request dispatched: interval: 14ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=17&key=xxx
20 throttle: 2010-07-23 23:17:31.897 - request dispatched: interval: 12ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=18&key=xxx
21 throttle: 2010-07-23 23:17:31.910 - request dispatched: interval: 13ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=19&key=xxx
   throttle: 2010-07-23 23:17:31.922 - active queue full
22 throttle: 2010-07-23 23:17:31.999 - request dispatched: interval: 89ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=20&key=xxx
23 throttle: 2010-07-23 23:17:32.012 - request dispatched: interval: 13ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=21&key=xxx
24 throttle: 2010-07-23 23:17:32.028 - request dispatched: interval: 16ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=22&key=xxx
25 throttle: 2010-07-23 23:17:32.042 - request dispatched: interval: 14ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=23&key=xxx
26 throttle: 2010-07-23 23:17:32.055 - request dispatched: interval: 12ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=24&key=xxx
27 throttle: 2010-07-23 23:17:32.068 - request dispatched: interval: 14ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=25&key=xxx
28 throttle: 2010-07-23 23:17:32.078 - request dispatched: interval: 10ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=26&key=xxx
   throttle: 2010-07-23 23:17:32.095 - active queue full
29 throttle: 2010-07-23 23:17:32.116 - request dispatched: interval: 38ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=27&key=xxx
30 throttle: 2010-07-23 23:17:32.128 - request dispatched: interval: 12ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=28&key=xxx
   throttle: 2010-07-23 23:17:32.136 - waiting 4822ms
31 throttle: 2010-07-23 23:17:36.964 - request dispatched: interval: 4836ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=29&key=xxx
   throttle: 2010-07-23 23:17:36.998 - waiting 182ms
32 throttle: 2010-07-23 23:17:37.181 - request dispatched: interval: 217ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=30&key=xxx
33 throttle: 2010-07-23 23:17:37.203 - request dispatched: interval: 22ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=31&key=xxx
34 throttle: 2010-07-23 23:17:37.225 - request dispatched: interval: 22ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=32&key=xxx
35 throttle: 2010-07-23 23:17:37.247 - request dispatched: interval: 22ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=33&key=xxx   
36 throttle: 2010-07-23 23:17:37.272 - request dispatched: interval: 25ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=34&key=xxx
37 throttle: 2010-07-23 23:17:37.299 - request dispatched: interval: 27ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=35&key=xxx
38 throttle: 2010-07-23 23:17:37.321 - request dispatched: interval: 22ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=36&key=xxx
39 throttle: 2010-07-23 23:17:37.342 - request dispatched: interval: 21ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=37&key=xxx

error: /tags failed http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=34&key=xxx --> @ 23:17:37.272

throttle: 2010-07-23 23:17:37.375 - request dispatched: interval: 33ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=38&key=xxx
error: /tags failed http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=35&key=xxx
error: /tags failed http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=33&key=xxx
error: /tags failed http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=36&key=xxx
throttle: 2010-07-23 23:17:37.443 - request dispatched: interval: 68ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=39&key=xxx
error: /tags failed http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=37&key=xxx
throttle: 2010-07-23 23:17:37.473 - request dispatched: interval: 30ms - http://api.meta.stackoverflow.com/1.0/tags?pagesize=1&page=40&key=xxx

11
  • The API throttle is being looked into, the sstatic throttle is [status-bydesign]. That's been said elsewhere. Chill, this only affects server [app]s running right up against the wire; its not the end of the world. Aug 23, 2010 at 22:08
  • @code: Whoa! Just for kicks, I ran your test suite again at 6.5 req. / sec. Suddenly I could no longer access StackApps! Are they throttling access to the site now?!?!? Sep 4, 2010 at 21:45
  • @geo - yes, that has been covered in another question, which was denied, and is a slightly different issue than the throttle simply does not perform as advertised. By the way, I have checked the tests in all major browsers on linux and osx and they appear to operate as expected when the instructions are observed so feel free to carefully run the tests and post your results here. Sep 5, 2010 at 3:15
  • @code: I feel bad for you - here you've designed this awesome library and I can't get it to work quite right. Maybe this will help: would you be able to try my app? Just visit this page and flip through the tabs to see if any of the sites are failing. Then we'll have a better idea of whether it's just me or there really is a problem. Sep 5, 2010 at 4:25
  • @geo - your app works just fine, as far as i can tell, on all windows browsers, ff,opera,chrome and safari on osx, opera, ff and chrome on linux. I don't see any failures. Sep 5, 2010 at 7:42
  • Well, I guess it's just me... though I can't imagine why. Maybe it's because I'm using Chrome 6 beta. Who knows. Thanks for trying it. And in a few hours you'll be able to list StackCenter as an example of a Soapi2.JS app. Sep 5, 2010 at 16:12
  • @geo - run the throttle tests and put your name in the upload message box so we can check the version. I am getting good results on both the unit tests and the throttle tests running chrome 6.0.472.53 on ubuntu. p.s. you should know better than to develop using beta browsers. Sep 5, 2010 at 16:23
  • @code: No, I developed it on Firefox :) It's just that I use Chrome for day-to-day browsing. I'm glad to hear my app's working, though. Sep 5, 2010 at 21:29
  • @code: I ran your test in Ubuntu 10.04 64-bit and got quite a bit further than last time. I also ran it in my Fedora 11 VM. Sep 5, 2010 at 22:15
  • @geo - i see you uploaded with annotation - perfect. The soapi.js2 software throttle is definitely working as expected, not the wait intervals of more than 7 seconds between any n+0=>n+30th request. thank you. Sep 6, 2010 at 2:41

3 Answers 3

1

API throttling has been fixed.

"30 reqs per 5 seconds" is now strictly true again, rather than just true for common use cases.

0
1

My results from your test across 3 browsers with different JavaScript implementations:

  • Like George I made sure to wait 30 sec. between tests
  • I've repeated tests here and there with quite different results already, but only started to document them later on, hence might add more over time
  • The results are obviously not exactly deterministic, thus pretty irritating
  • Likewise there are more failures than expected even (including complete failure on IE), hence something somewhere appears to be unaccounted for somehow still

Windows 7 / Firefox 3.6.8

Req./Sec. | Failure point(s)

30/5.0 | 34;32;32
30/5.5 | 41;34;36
30/6.0 | 46;38;40
30/6.5 | 47;--;957 (!)
30/7.0 | --;--;--

Windows 7 / Chrome 5.0.375.127

Req./Sec. | Failure point(s)

30/5.0 | 35;32
30/5.5 | 36;31
30/6.0 | 37;31
30/6.5 | --;--
30/7.0 | 179;929 (!)

Windows 7 / IE 8.0.7600.16385

Req./Sec. | Failure point(s)

30/5.0 | 34;32
30/5.5 | 41;65
30/6.0 | 46;101
30/6.5 | 47;256 (!)
30/7.0 | 179;415 (!)

7
  • so, the variances per browser are consistently inconsistent? I guess I am going to have to write a log parser that can summarize the actual numbers. Note that the timestamps written to the log are created at the moment that the request is sent, so even if the javascript implementations are variant enough(?) to introduce a temporal shift, the log does not lie. but it can get a bit tricky, as you introduce the waiting times, to calculate the actual interval between any set of requests. Thanks for running the tests. I may build a phone-home feature into that page that will send me the logs Aug 30, 2010 at 9:56
  • 2
    Yup, looks like they are consistently inconsistent! A phone-home feature sounds like a good idea for this use case indeed (if feasible effort wise on your part), as manually saving and posting/mailing logs would be both tedious and noisy over time. Aug 30, 2010 at 10:04
  • results upload is ready when you have time. Sep 5, 2010 at 16:26
  • @code - cool, just uploaded one iteration as above with pretty similar results; please note that Chrome behavior is particularly puzzling in that all tests fail if started from 30/5.0 and working up, but it works as expected if starting from 30/7.0 and working down - I witnessed this back then already and suspected a driver error on my part, but apparently there is some oddity involved indeed. Sep 6, 2010 at 12:28
  • that is kinda weird. I will give that a look. Sep 6, 2010 at 13:20
  • ok, so you are totally right about the order of execution affecting results. i am pretty sure it has to do with caching. where? i am not sure. when a test is started, i completely delete and recreate the throttle which contains the cache so i don't think it is in javascript. i think that multiple runs on a single page load do not present viable tests. when you load the page, i disable the ui for mandatory 8 seconds to prevent a quick refresh-test start from tripping the wire. if for each test you start with a fresh page load, 5-6 typically fail, 6.5-7 pass more often than not. Sep 6, 2010 at 23:44
  • another reason that i think caching is happening is that i have made 7 or 8 full runs and my quota is still 8000. so - thanks for pointing that out. yet another anomaly to track down. Sep 6, 2010 at 23:45
1

I just tested the upper limit (30 requests per 7.0 seconds) where test should run to 1000:

Mac OS X Version 10.5.8 - Firefox 3.6.8

30/7.0 | 175;60;161;52;114;60;60;292 (test never passed)

Mac OS X Version 10.5.8 - Chrome 5.0.375.127

30/7.0 | 660;passed;262;261;261;261;261;120

12
  • 1
    that wide a variance with a passing test on chrome indicates that the js throttle is working in some capacity on MAC. But the failures in Firefox seem abnormal. George reported odd results, e.g. failures @ 7 on linux as well. It looks like I need to fire up some VMs and check the JS. Thanks for running the tests Aug 30, 2010 at 22:08
  • @code: It would be wonderful if you would create a 'phone-home' script that we could run and have it send the results to you :) Aug 31, 2010 at 5:26
  • @code: Hmm... I tried Safari (using Wine), but I get an error "Failed to load resource: An SSL error has occurred and a secure connection to the server cannot be made." when trying to load the Firebug JS file. Weird. Aug 31, 2010 at 5:29
  • @geo - thanks for trying. I am busy with soapi.notify2 today. I will give this some more attention tomorrow and figure out all of these platform issues as well as a phone home. Aug 31, 2010 at 20:52
  • @code: Great! I've kinda got my own problems to deal with right now :) Aug 31, 2010 at 20:54
  • hey, i have checked the tests in linux and osx VM using current major browsers and they appear to operate as expected. I will formalize the output and implement an optional results upload tonight to make the efforts of those running the tests contribute to a dataset that can be used to determine the veracity of the tests. Sep 5, 2010 at 3:18
  • @code have you bought a MBP for this purpose :)? Or did you rig up an hackintosh? What about the phone home feature? Sep 5, 2010 at 6:32
  • just dusted off an old vmware image I cobbled together last year for another project. other than being 10.5.4 thus limiting the version of safari, it seems to do the job. and all but one of my integration tests run. the one that doesn't is not a valid test anyway as it tests for a real 404, e.g. bad api url, which is not possible with soapi. unfortunately it is the second test in the suite. lol. so, anyway, I am removing that test and giving the lib a thorough once over on osx and linux this weekend. I expect that there will be no issues. we will see. Sep 5, 2010 at 7:29
  • oops, i was wrong about being wrong. somehow in the test I upped the request count to 60 per xxx seconds (?!) I must have been distracted. It looks like the throttle works perfectly in Safari, FF and Chrome on OSX , FF and Chrome on Linux - Opera is the redheaded step child of course and needs special case code. But you should be able to run the tests and upload results. Sep 5, 2010 at 15:30
  • results upload is ready when you have time. Sep 5, 2010 at 16:26
  • @code phoned to your home; if you have stolen all my cookies with your js, go easy with my credit card, remember that i have two sons now :). Sep 5, 2010 at 18:49
  • 1
    mwuhahahaha - i have one that is the size of two. I will just feed him your cookies. thanks. Sep 5, 2010 at 18:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .