For analyzing large chunks of data, don't use the API. Use the Stack Exchange Data Explorer (SEDE), that's what it's there for. SEDE data lags real-time by 1 to 5 weeks, but that's no big problem for most statistical purposes.
For intense "Big data" analysis beyond even SEDE's capabilities, use the quarterly data dumps. (Thanks, Tim Stone, for the reminder.)
If you really must use the API, follow these guidelines:
- Register your appRegister your app.
- Make sure you do not bust your quotas.
- Monitor and respect the
backoff
flag. - For anything more, contact the API team and negotiate for special access.