3

I'm using the Questions Advanced Search API to retrieve Stack Overflow questions with a specific tag that were created in the last 15 minutes.

I've written a script which runs every fifteen minutes and mostly retrieves the newly-created questions that I expect. However, I've noticed that the API doesn't reliably include questions that were created without my tag, but were then edited within the grace period to include my tag after creation.

How soon should I reasonably expect the Questions Advanced Search API endpoint to report these types of ninja-edited questions? I'm considering deliberately putting a 15-minute fromdate/todate delay in my script to give the API time to incorporate ninja edits, but can I safely use a smaller threshold instead?

Improve this question
2
  • 2
    When we do a tag clean-up it takes a minute or two until we don't get stale search results, so I assume a job runs every minute or so. I don't know if the API endpoints are as much live as the sites. Needs a dev to confirm.
    – rene ♦
    Commented Jan 19, 2022 at 18:39
  • 1
    Reporting back: a fifteen-minute delay has been working well for my script.
    – Adil B
    Commented Feb 3, 2022 at 21:54

1 Answer 1

Reset to default
3

tl;dr: I think you can reduce the delay to 8-10 minutes. Grace period edits are available for 5 minutes after post creation and search results can be stale for a few minutes afterwards, especially during system maintenance or outages.

And now the details:

The advanced search API ultimately runs the same search as would happen if you searched the site manually. This means that grace period edits need to be re-indexed before they become available. There's also a bit of result caching happening on API's side, but I don't think it's relevant here.

Search index updates happen roughly every minute, so I think you could get away with 5 minutes for the grace period duration itself + a couple minutes for the reindex.

Indexing runs on its own schedule and system load could throw a wrench in as well, so you could potentially have an edit come in at the very end of the grace period that won't get picked up until a full minute + the time to actually update the index after that.

If the 15-minute delay is working well for you (which it sounds like it is) and is tolerable, I'd stick with that, but otherwise I think you could drop it down to 8 or 10 minutes without a loss of data outside of any unusual circumstances like an outage.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .