Why am I being told to backoff when I'm well within my API limit?

Question

I've been running my API fine for some time now and when I added another function I realized I was being told to backoff. I only run my API once a night and don't really pull that much data. I have a registered key that I use as well. It returns the proper counts and I was less than 1000 out of 10000 attempts in when it initially told me to backoff. Now, it tells me every single time to backoff for 54 seconds, regardless. I also noticed that my APP statistics shows no data, even though I have confirmed my API key is working appropriately. Not sure if this is related or not.

I guess my question is, how do I find out why I am being throttled on my API?

Below is a sample url request and result:

https://api.stackexchange.com/2.1/questions?order=desc&page=$page&sort=creation&site=stackoverflow&key=".urlencode($key)."&filter=".urlencode($filter);

["quota_remaining"]=> int(8099)
["quota_max"]=> int(10000)
["backoff"]=> int(54)
["has_more"]=> bool(true)

I'm following the backoff rules and sleeping for ( backoff + 5 ) seconds. Adding +5 just to be safe. But it doesn't reduce or change on each and every API query now. Almost as if I've been tagged as spamming or an internal blacklist.

Answer 1

backoff isn't used to indicate when you're breaking api limits (we start returning errors when you break contracts).

The typical cause of a backoff is a request that takes unusual resources to run. These are normally either complicated queries or high page values; however, backoff is applied dynamically so the exact definition of "complicated" and "high" can vary depending on load.

For example, if we're being DOS'd you'd probably see backoff in cases when you usually wouldn't due to the unusual load.