16

I've just read Kevin's answer to Do I need API key to use StackAuth API? and been quite surprised that the API doesn't seem to facilitate proper HTTP caching already:

You really shouldn't be hitting it more than once a day though, for site information, and in accordance with general API guidelines for associated users.

That's understandable and perfectly fine with me, but why don't you tell me (and especially my toolchain) this via the well established HTTP cache control headers then?

Both of which can be safely cached quite heavily.

Indeed, and I'd happily do so out of own interest, but preferably without the need to reinvent the wheel on my part for this well explored use case - if proper HTTP cache control headers would be served, any decent toolchain component wouldn't even think about violating your API guidelines/policies.

Granted, caching might not provide much benefit (if at all) for many volatile API routes and/or be not exactly easy to implement on the server side in some cases, but still it would for others, and the StackAuth routes are a good example, e.g. for /sites you could do:

  • Expiration-based caching:
    Send an HTTP 1.1 Cache-Control: max-age=... header for one day into the future, if that's what the API guidelines ask for right now.
    • Of course you could get even fancy here in case you actually happen to know when you are going to update the sites, e.g. if you'd only ever publish a new site on a Friday you might as well indicate this right away.
    • See HTTP 1.0 Expires: ... header too for the complete picture.
  • Validation-based caching:
    Send an HTTP 1.1 ETag: ... header computed from a hash of the result.
    • This might not seem to be too useful with the short list of sites right now (though it would still reduce bandwidth considerably), but this is planned to change, isn't it ;) Even more important though is that you are still communicating valuable information, because otherwise I need to compare the current and former result locally myself in order to know whether there has been any change in this resource.
    • See HTTP 1.0 Last Modified: ... header too for the complete picture.

Or any variation thereof - the chosen cache strategy largely depends on the particular route of course, but some basics applied would get us a long way without risking premature optimization, IMHO.

I'm reluctantly marking this as a feature request rather than a bug, but please consider applying these RESTful principles in order to take the cache control burden off the Stack Apps developer community shoulders and onto those of the Stack Exchange giants ;)


Further Reading

0

2 Answers 2

4

There are two, rather divergent, answers to this question.

First and foremost, its not in the API because nobody (us included, woops!) thought of it before the API interface was frozen... so it didn't get implemented, even as a trial feature. Perhaps we'll revisit it for a subsequent version.


Alright, onto the real meat of the issue.

Headers (Unintentionally) Suck

The sad truth of the internet today, especially for sites with as much uptake amongst corporate (firewall-ed) users as StackOverflow and ServerFault, is that you really can't rely on HTTP Headers. There's a reason we treat Accept-Encoding as a suggestion, at best.

Basically, anything that is HTTP/1.1 specific has a good* chance of not making it through a proxy; so Cache-Control and ETag are right out. This sucks and we know it, but its also beyond our power to change.

I'm being optimistic and hoping that HTTP/1.0 "compliant" proxies actually pass all relevant headers through, but I wouldn't be at all surprised to find a popular counterpoint.

The Difference Between the Sites and the API

The API could go the route of the sites themselves, and make use of features we know won't actually be available to everyone; but consider the difference in "targets".

The sites only care about the handful of popular browsers out there, all of which have extensive testing regiments and often times histories extending back into HTTP/1.0 days. We can be certain that if you're using IE, Firefox, Chrome, Safari, Opera or whatever that the sudden disappearance of - say - Cache-Control isn't going to cause a crash in some seldom used code path.

The API, however, is catering to a potentially much larger and more diverse set of [app]s and [library]s. Most of which are products of hobbyists with limited resources, and naturally the code-bases themselves are very young. Instructing developers they can make use of things that will often - but not always - be there is just asking for all sorts of nasty crashes in untested code paths.

Once you consider that most [app] developers probably aren't behind proxies when developing their [app]s that path is even less attractive given the implied difficulty in properly testing an [app] under such a scheme.

As I noted above, we'll revisit this - maybe not this particular solution, but something similar in purpose anyway - in future versions of the API. Making it easier for developers to not spam us with spurious requests is very attractive, naturally.

*I don't have hard numbers to throw around, but we've seen it in closed beta's which are pretty small. I'd use ~1% as a planning number, maybe higher.

10
  • Re 'nobody thought of it': that's a bit disturbing I have to say, thanks much for being honest here though! Still I don't see how this is affected by the API freeze? As hinted it could as well be treated as a bug and the API 1.x could start sending those headers with zero change in the API, except for extending the documentation. All those headers are just opt-in optimizations from a client perspective, i.e. any client not using them for whatever reason will simply GET /what/it/is/asking/for HTTP/1.1 200, no? Jul 8, 2010 at 11:00
  • Re 'Headers Suck': so you are going to sacrifice an efficient, well understood and easy to implement RESTful design for 1% of your API users behind outdated corporate proxies (based on your soft numbers), while playing much more hard ball compatibility wise in the browser arena? Wow … As outlined already cache control headers are opt-in optimizations from a client perspective - you are effectively saying that e.g. none of those bazillion Twitter clients do work for many corporate workers due the Twitter API serving HTTP/1.1 cache control headers, are you? I'd be stumped, but what do I know … Jul 8, 2010 at 11:06
  • Re 'Cache-Control and ETag are right out': but you do send the HTTP 1.1 Cache-Control: private header already anyway, and it doesn't seem to break anything, why should adding max-age=… possibly change that? I think you are contradicting yourself here, which leads me to the next point … Jul 8, 2010 at 11:07
  • Re 'Difference Between Sites and API': … after acknowledging this topic to be an oversight on your part this argument sounds more like you are starting to make excuses that just don't make sense, which is really unnecessary after being honest in the first place, IMHO. You are (understandably) limiting browser support to popular ones, and this will be the main platform your users are going to use for rich interaction with Stack Exchange sites. [... continued in the next comment, I'm trying to avoid the need for submitting an answer myself just for commenting, sorry!] Jul 8, 2010 at 11:14
  • [... continued from the previous comment, sorry!] But for those using it via an obscure hypothetical 3rd party client you'll rather stick to outdated technology and custom workarounds, reducing performance and scalability as well as increasing development cost and the likelihood of bugs in custom code for all others who'd simply use their frameworks instead? Come on … Let's please just GET /over/it HTTP/1.1 202 and implement cache control properly in an upcoming point release :) Jul 8, 2010 at 11:21
  • @Steffen Opel - the problem is not hypothetical. We've already been down this route with Accept-Encoding and I have no desire to repeat a mistake of the past. In this particular case of purity versus pragmatism, pragmatism wins. Igor Zevaka answer outlines some additional arguments as to why caching headers wouldn't be that much of a net gain. Do note that the server already does substantial caching behind the scenes, we're just not pushing those headers as documented parts of the API for the outlined reasons. Jul 8, 2010 at 19:41
  • 1
    @Kevin Montrose - you are comparing apples and oranges, because Accept-Encoding requires active participation from the client (in 'pure' mode), while Cache-Control does not (opt-in). It's not about purity, to the contrary, it's about a proven best practice design pattern for RESTful web services with a track record for performance and scalability. Thanks for answering at all, but I can't help but notice that you haven't answered a single specific question of mine (why can Twitter do it, why should Cache-Control break with max-age=... etc.). Alas, looks like a lost case at the moment … Jul 8, 2010 at 21:23
  • @Steffen - I see the disconnect here. You think opt-in behavior is a good idea, and I disagree. Opt-in API behaviors tends to lead to brittle code; especially where the opt-in behavior works most but not all of the time. Consider: we document that we set some header on every request (which would be true), how many libraries are just going to read the header without checking if its actual arrives at the client? A depressing number, historically. How many subsequent bug reports are getting [status-norepro]'d by [app] developers? Another depressing number. How do you solve that problem? Jul 8, 2010 at 21:24
  • 1
    @Kevin - while I can finally see what you mean with not all the time at least (thanks!), I think you are overstating the possible impact (and sacrificing a lot for this assumption): developers coding/using headers manually in a way that could crash their app (instead of reusing proper framework components in the first place) are going to make you trouble anyway. Well, I'm going to sleep over possible workarounds now in order to disconnect my application architecture from your surprisingly unique API design indeed - good luck with version 1.0 still! (I'm not trying to be cynical here ;) Jul 8, 2010 at 23:18
  • 1
    @Kevin: Incidentally, I am behind a proxy at work and implemented proxy authentication in StackLINQ. If/when you decide to try this out, I can test it, for my particular proxy at least. Jul 9, 2010 at 0:52
3

Perhaps I can offer an opinion on cacheability of the API. Please don't downvote before reading through.

Caching is hard

Caching and cache invalidation is hard. Often, if not always, you need to intimately know the data to be able to cache it effectively.

In order to enable a generic Etag caching ability the server will need to execute the request and then get a hash for it to compare with the requested hash. So, from server's point of view, this kind of caching is pointless.

Take a look, for example, at this request (which is the heaviest I could think of - requesting 100 answers with body):

http://api.stackoverflow.com/0.9/questions/9033/answers?pageSize=100&body=true

Answers request http://img641.imageshack.us/img641/6236/answerrequest.png

It takes the server nearly twice the transfer time to process the request. So, a trivial implementation of Etag caching would be at most 50% efficient.

Validation using time

You could possibly have an even more trivial caching that simply caches by URL and, for example, if the URL matches exactly something the server did before than it would either serve cached data or reply with Not Modified. However, I would think that cache hits on exact API URLs would be pretty rare.

Expiration-based caching

Again, without actually getting the data from the DB, the best the server can do is give you an estimate of how long it thinks the data will be valid for. This is the kind of thing that an app/wrapper can actually do for itself. E.g. do not request the answers for a particular question more than once a minute. Which brings me to the next point:

Wouldn't it be easier for the API to provide data "freshness" intervals for different kinds of requests as an API call?

This way the app has the option of requesting data more frequently than the server thinks it's valid for and giving the app the ability to cache with possibly better freshness confidence than the app would guess.

Regardless,

Caching at the application level trumps all

Let me give you a half example, half plug. Stack Tagz does two levels of caching.

One is completed timeseries - this is the graph that you see. I consider that questions don't really get retagged all that often, so once it calculates a timeseries, it's persisted to the DB. Requesting a graph for Jon Skeet takes about 400 requests (which would be smaller if the vectorized requests are fixed, hint hint). It'd be crazy to make those requests every time someone wants to look at his graph, especially considering it won't look any different next week.

Another is caching of individual questions. There is significant overlap if multiple people answer the same question, no good reason to request it again. Even if a question is retagged/deleted, it's OK, there are plenty of other data points for it to be statistically significant.

So, I guess what I am trying to say, is that different apps have different tolerances for the data freshness, and hence can improve on the most pessimistic guarantee that the API can provide. Individual app makers should really think about caching, not delegate it all to the API.

1
  • 1
    Re 'Caching is hard': it sure is - thanks for taking the time to explain your argument. I still think you are missing the point here and there, e.g: ETag might not even need to get recomputed, see Jeffs answer for an example); CPU time can simply be reduced by big iron, while network bandwidth/latency can not; Cache-Control is providing exactly the freshness interval your are suggesting, albeit in a standardized way every framework understands already - you are free to ignore Cache-Control in your apps cache strategy as you please ;) Jul 8, 2010 at 10:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .