How many simultaneous queries may I make of the same Stack Exchange API?

Question

I've been playing with some of the APIs that have paged results.

I've been doing a first call with pagesize=0 to determine how many pages there are then running a loop querying each page.

In my case I've never needed more than 8 pages at once so far, but I'm wondering if I were playing with a bigger SE site or when my site matures and I needed to request many more pages, are there some limits I must respect?

How many simultaneous requests is OK? Should I put in occasional delays, doing smaller batches at a time? If so how long a delay, etc?

I'm also interested in looking how any Stack Exchange API frameworks might have already implemented methods to query all pages in such "friendly" ways.

Answer 1

The official rate limits are posted here.

One thing you mentioned in your post does strike me as a little odd:

"I've been doing a first call with pagesize=0 to determine how many pages there are then running a loop querying each page."

Why the need to fetch an empty page to determine the total? Why not fetch the first page (which contains the total) and then continue from there?

"I'm also interested in looking how any Stack Exchange API frameworks might have already implemented methods to query all pages in such "friendly" ways."

I wrote a PHP wrapper for the API that does this in a rather unique way. Instead of exposing pagination to the user (although pagination data can be retrieved if needed), a stream of items is provided:

$site = API::Site('stackapps');
$response = $site->Questions()->Unanswered()->Exec();

// The $response variable takes care of pagination for us.
// We simply call Fetch() over and over until it returns FALSE.

while($item = $response->Fetch())
    // Do something with $item

The Fetch() method automatically retrieves new pages as items are requested. It also takes the rate limit into account, pausing to comply with the limit if necessary.