Best strategies for calling the API with full throttling support?

Question

I know this would be subjective on a normal SE site but as the relevant chat rooms are mostly dormant, we have no meta site, and nobody noticed when I asked if this kind of question is OK here, I'm just gonna ask m'kay? (-:

I want to make a little tool which needs to read all the tags, all the synonyms, and a bunch of the tag wikis. I'm just hacking for now and I'm happy with doing small requests with no app key. I also find it an interesting exercise. I'd like to know how some of the library and wrapper developers here did it and bounce some ideas around.

I'll be using JavaScript and probably jQuery but I'm more interested in abstract algorithms.

• How many requests are OK to send simultaneously?
• How to wait between requests, or how to know when to wait?
• Send all requests through a "manager"? Or do them manually with some guidelines and watching response feedback fields such as "back off"?
• How much of the per-IP limits to use? You never know when other stuff is running on the same IP.
• Handle the different limits for with vs without key.
• For long lists requiring multiple requests, one request first to find the total? Or first iteration of the loop reads the total?
• Retry lost portions? I can't be the only one with an often flaky connection.

Answer 1

You might consider looking at StacMan for inspiration, a C# client that deals with some of these questions.

Off hand it does push everything through a "Manager" (called a Client in that code) and automatically enforces backoff and simultaneous request throttling.

How many requests are OK to send simultaneously?

While there is a strict cutoff... Practically, you don't want to try and hit that as a matter of routine; timing over a network isn't precise enough. 10 requests a second is a sane target, though you should always try and vectorize your requests so you make as few as possible.

The right way to think about any Web API isn't "how many requests can I make" but "how can make as few requests as possible". We document our throttles (because it can be hell to develop without knowing them) but you shouldn't write code that relies on them being at a certain level. It's against the spirit of the thing, if you will.

How much of the per-IP limits to use

Similarly, the answer to this is "as little as is necessary". Trying to guess how much of the quota is meant "for you" is the wrong way to go about it, you should be trying to leave as much as possible around for whatever else may need it.

For long lists requiring multiple requests, one request first to find the total? Or first iteration of the loop reads the total?

Use has_more for this if possible, it's much cheaper than fetching total so you're less likely to cause a backoff response.

Retry lost portions? I can't be the only one with an often flaky connection.

Automatic retries are a bad idea as a default (you have to remember the case where the request gets to us, but the response is lost; you'll still consume quotas, be given backoffs and all that jazz but you won't know it). Providing them as an option is fine, but you'll want to make sure the retries are subject to the same throttle respecting logic as everything else (in so much as is possible).