Could/Should the API evolve more agile rather than being blocked by v2 writing (and authentication) features?

Question

The activity on Stack Apps has dwindled noticeably; this might be a topic for its own discussion, however I'd like to revisit the API roadmap beforehand to set a proper stage and expectations in this regard.

Status

I'll try to summarize the API status from my perspective, which might be a little biased regarding my own interest in the Stack Exchange platform, which, despite being a very happy Stack Overflow user myself, has always been integration with other platforms I'm involved with (partially with commercial interest, i.e. turning a solution into a product I could live from developing); so please take my statements with a grain of salt.

Bugs / Feature Requests

There have been 141 bugs posted, with 100 completed, 23 bydesign, 7 norepro and 2 deferred, leaving 11 unaddressed so far, which isn't nice, but all in all bugs have been handled properly and promptly so far.

On the other hand there have been 118 feature-requests posted, with only 23 completed, 3 bydesign, 20 declined, 12 deferred and 4 planned, so a whopping 56 are not even acknowledged/classified officially as of today. (Note that a couple of feature requests are covered in API V2 Musings: What would you like to see as well.)

API / App Promotion

After announcing the Stack Exchange API Contest Winners there haven't been much promotional activities, as far as I know; it seems particularly worrying that respective inquiries do not even get any comment from the authorities, see Is it possible to have a StackApps Advertising - Sidebar contest?.

Legal Issues

There are still no API Terms of Service and I haven't gotten any kind of feedback regarding this on several channels, see e.g. Where can I find the “API Terms of Service” referred to from the legal pages?

Summary

The outlined partial (or rather selective) lack of feedback can be seen on MSO as well, but I think it is even more of a problem for API/App developers, who need to decide whether and how to deal with their own app user base and roadmap in this regard; this is a rather huge impediment for anybody considering investments into App development from a professional perspective (i.e. time/money), I'm afraid.

But enough regarding orphaned topics, let's move on to those where official statements/hints are available at least and try to figure out some ideas on how to accelerate API/Apps development from here, eventually:

Official Statements/Hints

The design goals for the API v1, specifically the read only restrictions are covered in Jeffs blog post Stack Exchange API Public Beta Starts:

Version 1 is read only. Coming up with a solid API is hard enough without adding writing and authentication to the mix. For the initial release, it’s a read-only API. We’ll take on the much more challenging problem of writing (and authentication) in v2.

Likely it has been this statement which introduced the theme of tying the API v2 with writing (and authentication).

A more recent and granular update regarding this has been Kevins answer to Timeline for v2.0 of the API?, which added bugs and improvements to the API v2 theme:

[...] defects in v1.0. Part of v2.0 would be addressing these issues [...]

Another part of v2.0 would be bringing the API into line with the improvements made on the sites. [...]

Finally, since we've implied - though not quite promised - write access in v2.0 the exact mechanisms for that need to be worked out. [...]

Back then I already asked have you considered a more agile approach of progressive improvement instead, but haven't gotten any response, hence the revisit here ...

Jeffs Code Camp Video Interview

Recently I've fast forwarded through Jeffs Interview to get some clues eventually, and actually did - please see Selective API oriented transcript fragment of Jeffs Code Camp Video Interview for a bit more context, but essentially Jeff is expressing serious doubts about the feasibility of a write enabled API:

Yeah, the UI is helping [...] there's a lot of ambient information present on the website, how would that even be presented in an external app we can't control? I'm fine with reading [...] but one of the quickest way to ruin a Q&A system is to flood it with bad questions - now I'm not sayin having a write API would cause that, but anything that risks that is a very very serious risk.

I actually agree with his observations, he definitely has a point in that a writable API could exhibit quality issues and a rich ajax app offering much better UI options to deal with this (an interesting variation of the same old argument between rich clients vs. web apps in general btw.).

Still, this puts a serious bump into the current expectations among Stack Apps developers, I'm afraid, in so far writing (and authentication) have been the number one feature request, obviously. However, this shouldn't provide a full stop to API development forever, or should it?

I'd rather say it is a strong hint towards a necessary shift in pace and scope in order to let the API evolve more agile with the platform as best as possible under the current read only restriction. I'd personally pretty much prefer an API keeping track of essential and stabilized platform improvements (e.g. Linked Questions, Tag Wikis, Tag Info Pages etc.) over waiting for the golden age of writing (and authentication), which apparently isn't going to happen anytime soon, if ever.

What do you think?

Answer 1

"Agile" API releases are a really bad idea, frankly.

The StackOverflow code base undergoes rapid iteration already, just watch the build numbers. On average we deploy at least once a day. Oftentimes these are just bug fixes, but a non-trivial amount of the time we're introducing new features or revising existing ones.

To take one feature for example:

• The "interesting tab", announced November 9th
  • initial revision deployed (for testing) on the 3rd
  • formula revised the 11th

There are 3 different dates on which the API could have been revised, and the only certain one (the 3rd) is also the one almost invariably incorrect in very short order (8 days later in this case).

Remember, once its in the API (baring exceptional circumstances) it has to stay there until the whole version† is deprecated. This is promise #3 on our API announcement.

The right way to expose new features to an API is to let them bake on the site until we're satisfied with them, and then expose them. Which is basically what we're doing.

You'll note that nobody really does agile APIs. Twitter goes months only updating documentation and fixing bugs, and Facebook is notorious for not even bothering with the "fixing bugs" part of that.

---

Now, all that aside I do think its about time for another API version. We froze version 1 a little less than 6 months ago, and I guess a biannual API revision (or beta, or something) is just as good a schedule‡ as any.

I also don't think its worth blocking on writing (which has actually gotten harder, now that a few sites have custom post renderers). I'd like for the next version (perhaps just a point release) to be a refresh, exposing new features, fixing all the little issues with v1.0, making it easier to tweak [app] performance, etc. Maybe add user authentication, as while there are some tricky issues there in the absence of writing I'm more open to experimentation.

Its just a matter of finding the time.

---

As to Legal and Promotion...

Legal usage of the API is governed by the same agreement as the sites (subject to change at any time, etc. etc.) which is linked as legal in the footer of every page. Some additional guidelines (likewise, subject to change) are linked from my answer to the question you linked.

This ad (and a similar one, in a sidebar slot) has been running on SO (and possibly SU and SF, I forget) since the end of the contest. Its a link to the apps tab. Given that neither you nor anybody in the linked thread noticed, I doubt yet another ad (which would presumably replace some of the OSS ads*) would do any good.

_{†I suppose we could start deprecating methods instead of versions, but that'd put alot of burden on developers... for basically no gain.
‡Note, there will never be an official release schedule. Its a pointless constraint. I'm speaking in broad vagaries.
*The number of ad slots being finite, something would have to go.}