I wrote a small program querying the SE API to grab the public scoring data of all SO answers (excluding the last 3 days).
Basically the request I do gets me this:
Questions[]:
Id
Title
Tags
CreationDate
Owner:
Id
Name
Answers[]:
Id
CreationDate
Owner:
Id
Name
Score
Here's an example of the queries I do (API key removed):
The problem is: This processus is too slow to get me everything. I receive 30 questions per response of the API. I'm allowed 10,000 requests per day. So it would take me a long time to get up with the site and have all questions.
I'm also asked to backoff for 10 seconds after every request, so I suppose it's heavy for the server.
Here are my questions:
- Is some part of my request too heavy ?
- Is there a better approach ?