I am being told that this is by design.
I could not disagree more.
The inconsistent 503 throttling issue aside, the API is JSON based. HTML errors serve no purpose in response to an API request.
Regardless of the whys, api.stackoverflow.com behaves as has come to be expected, while the others do not.
I encourage library and app developers to switch the target site for any test suites from api.stackoverflow.com to any other endpoint, especially for consecutive page requests to understand the purpose of this post.
While paging tags, all api (that have enough tags) fail with a 503 html error after just a few consecutive page requests.
I don't know that it is limited to tags, but the behaviour seems to indicates a systemic oversight as explained below.
api.stackoverflow.com on the other hand, behaves as expected and dutifully services consecutive page requests until the end of the response.
A related, but critical, concern is that any error raised as the result of a JSONP API call needs to be JSON with a 200 status, not HTML. We covered this in this issue: API Suggestion: suppress http error code when jsonp is specified
It is this aspect of the errors that leads me to assume that the throttle code on the other api endpoints is not recognizing API requests.
As a guard against what I have to expect, the request URL are unique and consecutive and as such do not constitute any sort of API abuse.
notice the page, 298.
GET /0.9/tags?key=[mykey]&page=298&pagesize=100&jsonp=Soapi.jsonp._callback354 HTTP/1.1 Accept: */* Referer: http://localhost:23616/src/staggregator.htm Accept-Language: en-US User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; Tablet PC 2.0; .NET4.0C; .NET4.0E) Accept-Encoding: gzip, deflate Host: api.stackoverflow.com Connection: Keep-Alive HTTP/1.1 200 OK Server: nginx Date: Sat, 03 Jul 2010 22:26:47 GMT Content-Type: application/json; charset=utf-8 Connection: keep-alive Cache-Control: private Content-Encoding: gzip X-AspNetMvc-Version: 2.0 X-RateLimit-Max: 10000 X-RateLimit-Current: 7106 X-AspNet-Version: 2.0.50727 Content-Length: 196
Typical response from api endpoints !=
GET /0.9/tags?key=[mykey]&page=7&pagesize=100&jsonp=Soapi.jsonp._callback36 HTTP/1.1 Accept: */* Referer: http://localhost:23616/src/staggregator.htm Accept-Language: en-US User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; Tablet PC 2.0; .NET4.0C; .NET4.0E) Accept-Encoding: gzip, deflate Host: api.serverfault.com Connection: Keep-Alive HTTP/1.1 503 Service Temporarily Unavailable Server: nginx Date: Sat, 03 Jul 2010 22:01:14 GMT Content-Type: text/html Content-Length: 650 Connection: keep-alive <html> <head> <title>Too Many Requests - Stack Overflow</title> </head> <body style="font-family:Arial,Helvetica,sans-serif;"> <div style="margin: 0 auto; width: 960px;"> <h2 >We're sorry...</h2> <p>There are an unusual number of requests coming from this IP address.</p> <p>To protect our users, we can't process any more requests from this IP address right now.</p> <p>We'll restore access as quickly as possible, so try again soon.</p> <p>If you believe you have reached this page in error, <a href="mailto:[email protected]">contact us</a>.</p> </div> </body> </html>