0

In developing the firehose service, I found that the /events API often "pauses" for a couple of minutes before resuming. Is this a known issue?

Specifically, I am polling /events every 10 seconds using a custom filter with all fields turned on. I'm using the since parameter to ask for only events that are newer than the most recent one received so far. I'll always get at least one previously seen event back (and ignore it because I've already seen it). The numbers I'm getting every 10 seconds look something like (count of new events):

..., 8, 1, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 42, 4, 1, 3, ...
..., 2, 5, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 38, 4, 4, 2, ...

It appears that the events that were not sent during the quiet period are all sent after the flow resumes.

I do recognise the backoff element but I'm not receiving any of those.

Improve this question
3
  • 2
    i just read this in the docs "the Stack Exchange employs heavy caching and as such no application should make semantically identical requests more than once a minute. This is generally a waste of bandwidth, as more often than not the exact same result will be returned." the since filter is being applied to the cache contents and you are 'overtaking' the cache?
    – russau
    Commented Apr 20, 2012 at 14:02
  • @russau: That's a good point. If the caching is done at a layer above the API request handler, and during some 10-second period there is no new activity, then subsequent requests will use the same since field. That would explain why I never see single 10-second periods with 0 new events.
    – Greg Hewgill
    Commented Apr 20, 2012 at 19:51
  • @russau: I think you're on to something. I tried adding a random nonce= parameter to the query, but that didn't seem to help. Then I tried decreasing the since= parameter by 1 every time I got no results, and that does seem to avoid this problem (and I now get individual periods with no new events).
    – Greg Hewgill
    Commented Apr 20, 2012 at 20:45

1 Answer 1

Reset to default
3

The comments are more or less correct; caching applies to the /events endpoint just like every other method.

This is by design for now. We have looked at differing caching policies for /events and /inbox/* routes, as the minute lag on those methods is a bit heavy handed; though we still need to protect ourselves from abusive requests.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .