31

We've started working on the next version of the Stack Exchange API.

The current draft specification is available, and we'd like to hear your feedback!

The headlining feature for this release is authentication, v2.0 will remain read-only.

To be clear, this is meant to solicit feedback not to facilitate development against the version 2.0 API. For that, you'll need to wait for the eventual beta.

We've discussed this draft internally and privately solicited feedback from many of our [app] developers, so we're fairly confident this draft is acceptable.

When reading the spec, remember that all methods that exist in 1.1 will be carried forward to v2.0 unless explicitly noted otherwise. The document was long enough already without re-spec'ing all of 1.1's methods.

If you have feedback, please leave it on this post (as an answer).

Improve this question
10
  • Will any 1.0 methods be carried over, specifically the stackauth routes (starting to read the spec now so I apologise if it has been answered in it)
    – Jonathan.
    Commented Sep 17, 2011 at 21:55
  • @Jonathan - there are 1.1 equivalents, so yes. 1.x isn't getting shut down when 2.0 launches anyway, so all 3 versions will be available (at least for a while).
    – Kevin Montrose
    Commented Sep 17, 2011 at 21:57
  • 2
    /my/comments would "feel" so much better than /me/comments.
    – badp
    Commented Sep 17, 2011 at 21:57
  • 3
    @kevin, 1.1 is ok, but the pagination of sites is quite annoying, as typically you get all sites at once, rather than questions etc where you only want a few at a time.
    – Jonathan.
    Commented Sep 17, 2011 at 22:11
  • @kevin, oh and I never said thanks for all the hard work you and the SE team put into this, so thanks :)
    – Jonathan.
    Commented Sep 17, 2011 at 22:21
  • 2
    @badp - I was considering /you/* or /authed/* before I (re-)discovered Facebook's use of /me. Figured it was better to use something dev's would be more likely to guess from familiarity with another API than be different for the sake of being different.
    – Kevin Montrose
    Commented Sep 17, 2011 at 22:42
  • 3
    Looking forward to beta!
    – Adam Davis
    Commented Sep 17, 2011 at 23:41
  • Partly related to all of the fields being HTML-safe, will /users/{id}/timeline continue to return Markdown instead of HTML?
    – Tim Stone ♦
    Commented Sep 27, 2011 at 14:48
  • @Tim - no, that's one of those bugs we got stuck with since people rely on it.
    – Kevin Montrose
    Commented Sep 28, 2011 at 18:21
  • Makes sense. I sometimes wish the other routes could return Markdown too, but I guess that's more than likely a narrow userscript-related usecase.
    – Tim Stone ♦
    Commented Sep 28, 2011 at 18:25

6 Answers 6

Reset to default
9

Some comments on the new features:

Authentication

I really dislike an authentication flow which mandates a call out to a browser. It makes command line applications basically impossible. It makes consistent UIs for specific platforms very hard (e.g. iPhone), as you basically have to dump the user into a web browser with an entirely different UI. It also adds very little practical security. The idea is that the user is conditioned to only enter their authentication details into the StackExchange UI, but there's nothing stopping malicious applications chroming a fake login page, presenting that, then proxying the entered data onto the "real" OAuth endpoint (or, having cached the illegal data, just telling the user that it was the wrong password, and redirecting them to the real site).

If I can't turn the tide on this, and I doubt I can, I would like to lobby heavily for an alternative authentication flow, with normal "POST user/pass to secure endpoint, get token" XAuth style process. This flow might be restricted to trusted applications of some form, as is provided by Facebook for certain partners. Failing that, the ability to various parameters to the authentication system stylesheet would be useful - enough so that we can make the transition less jarring for people.

The expiry of access tokens after 24 hours is also a huge pain. Imagine your smartphone Twitter client if it had to prompt you to login every day - it would be unusable. It also eliminates e.g. push notification systems that check your inbox on your behalf, and send messages to your phone. This is probably enough of a problem that it would make most of my use cases unusable.

Filters

I like this, if either:

  1. Filters created programatically exist forever. Or, more pragmatically, application owners can create filters that are associated with their application tokens via some web UI, and these exist forever.
  2. Enough useful pre-defined filter tokens are created.

What concerns me is the complexity of having to bake in a filter existence check, and filter creation process to every single request. This would be a big pain in the otherwise natural workflow of "Send request. Get response or error", which would mutate into "Send request; Did filter work? Create filter. Resend request. Get response or error". It also adds the certainty that many copies of the same filter are created - every client app instance will end up having to create its own copy, which might make optimisations on the server side harder. If filters were created on a web interface, and a token given that lasts forever, it would make it trivial for you to associate that ID with a query strategy optimised for it (should that become needed).

Edit: Based on comment answers about filter longevity, perhaps there could be an easy route we could call to check a filter's validity? Then the "Startup, check filter, recreate" setup might be practical with a "Bomb out if filter happens to die in mid workflow" handler case.

Improve this answer
10
  • Auth token expiry will be removable with a scope of no_expiry. It's really important IMO that apps are not given perpetual access to a user by default. There are, like you say, good reasons to have it sometimes; so it's an option.
    – Kevin Montrose
    Commented Sep 20, 2011 at 15:51
  • 2
    While I can, with great reservations, understand an objection to browser based flows for authentication I don't really think it's applicable to us. We already have to have a browser for OpenID logins, after all.
    – Kevin Montrose
    Commented Sep 20, 2011 at 15:54
  • As I mentioned on Jonathan's question, filter's should never expire or be invalidated during normal operation. There will be some provisioning for if we ever have to invalidate them, but an [app] can safely treat that as an exceptional "burn everything to ground" case. A "was filter bad, if so retry" flow should not be necessary.
    – Kevin Montrose
    Commented Sep 20, 2011 at 15:56
  • 2
    @KevinMontrose But apps, in principle, must have a fallback "Filter creation" system? For iPhone apps, if there's any possibility that they might break, you have to handle them to avoid the bricking of the app.
    – Adam Wright
    Commented Sep 20, 2011 at 16:07
  • 1
    @KevinMontrose Ah, I missed "no_expiry". Thats grand. The lack of a non-browser auth route is a bit sad - it does render the API unusable (or very difficult to use) from some environments (e.g. command lines).
    – Adam Wright
    Commented Sep 20, 2011 at 16:09
  • in principle yes, though I think a "shutdown and re-check on startup" approach will be more than adequate. I'm going to put serious effort into making filters as stable as possible. An app wasting requests (or a dev wasting time) dealing with filter invalidation greatly diminishes the value of filters IMO, so getting it right is officially a "big deal".
    – Kevin Montrose
    Commented Sep 20, 2011 at 17:27
  • @Kevin, it will be possible (and allowed?) to upload a user's token to a server, so that a server can check routes needing auth-ing on the user's behalf. (e.g. for push notifications for just about every mobile platform currently available)
    – Jonathan.
    Commented Sep 20, 2011 at 20:00
  • @Jonathan - yes, that's a pretty common behavior w.r.t. OAuth.
    – Kevin Montrose
    Commented Sep 20, 2011 at 21:24
  • 1
    @Adam - to address your edit... There's always the ghetto case of "just call something with it and see if it bombs", but there will also be a /filter/read (or similar... debating exacting naming) that could serve the same purpose. /filter/read is envisioned as being for development mostly.
    – Kevin Montrose
    Commented Sep 28, 2011 at 18:19
  • @KevinMontrose Ah, excellent. I only ask because I didn't want to use an "expensive" query just to throw away the results (expensive both in bandwidth used, and for you to compute).
    – Adam Wright
    Commented Sep 28, 2011 at 21:45
8

I would like to see fixed paging. And yes, that means I think the current system is broken.

What do I mean by broken? When you want to get 500 newest questions and the page size is 100, it's impossible to do so without the possibility of duplicates (which are easy to ignore) or (much worse) missing questions.

This happens because you can access page 1, then page 2, etc. But in-between the fetches, new answers get added and old ones get deleted. So, at the time when I access page 1, some question is at the top of page 2. But when I access page 2, the same question might be at the bottom of page 1 now (because some question from page 1 got deleted in the meantime). Because of this, I will never see that question, even though I should.

I don't expect fully consistent view of the data. If a question was added during my fetching, I might or might not see it, that's okay. What's not okay is if there is a chance that I might or might not see a question that existed before I started, and still exists after I finished.

To fix this, I propose a system similar to one MediaWiki uses in its API. When you do a query, you get back a query-continue value. For you, it's some identifier of the next page and you don't care about its contents. If you pass it back to the API, you get the next page, as you would expect. Internally, it's something like the sort key of the item that would be on the top of the next page.

Improve this answer
11
  • For the paging issue, a cumbersome workaround seems to exist (I haven't tried it).
    – Gilles 'SO- stop being evil'
    Commented Sep 22, 2011 at 0:23
  • @Gilles, that won't work if another question gets undeleted at the same time. It's not likely, but certainly possible.
    – svick
    Commented Sep 22, 2011 at 0:48
  • 2
    This can be dealt with completely by specifying the current timestamp for the 'fromdate' parameter. This will ensure consistency.
    – Nathan Osman
    Commented Sep 22, 2011 at 17:43
  • @GeorgeEdison it would be much easier to have a system as svick suggested
    – Jonathan.
    Commented Sep 22, 2011 at 18:04
  • @GeorgeEdison, I don't understand. Wouldn't that mean I get only items created since I started the query? If you meant, todate, that wouldn't help either, because items can be deleted and undeleted.
    – svick
    Commented Sep 22, 2011 at 19:15
  • @svick: Yeah, sorry - I meant todate. That won't cover deleted items, but is that a concern? Once you are done enumerating, the item would still be deleted regardless - at least this way you would know that something was deleted because the total field returned would be smaller.
    – Nathan Osman
    Commented Sep 22, 2011 at 19:29
  • @GeorgeEdison, I'm not worried that I won't see an item that was deleted while I was enumerating. I'm worried that deleting that item changes paging and so I won't see some item that wasn't deleted. And if you mean I should start enumerating anew when I detect deletion, that could easily lead to basically infinite cycle. And it's not reliable either. If some item is deleted and another is undeleted, it looks like nothing changed.
    – svick
    Commented Sep 22, 2011 at 19:31
  • @svick: If an item is deleted, you will still see all of the rest of the items in the set.
    – Nathan Osman
    Commented Sep 22, 2011 at 20:36
  • @GeorgeEdison, no, I won't. Imagine that there are items 1 – 100 on the first page and I fetch that. Then item 1 is deleted. But now, page 1 is 2 – 101 and so when I fetch page 2 now, I will get items starting with 102. This means I will never see item 101. That's what I've been talking the whole time.
    – svick
    Commented Sep 22, 2011 at 20:46
  • @svick: Ah, you're correct - it took me a minute to wrap my head around it. I see where your solution comes into play now. Sorry about the confusion.
    – Nathan Osman
    Commented Sep 22, 2011 at 20:58
  • @svick: I might have solved part of the problem here: stackapps.com/questions/2147/…
    – Nathan Osman
    Commented Oct 26, 2011 at 3:06
7

I would be interested in seeing a /me/votes route to get a list of all my upvotes and downvotes. Note that this would only apply to my voting history and not anyone else's.

Improve this answer
1

Will authentication as a moderator/10k let us lookup deleted questions? If not, is this likely to be added in the future?

Improve this answer
1

(Should this be an answer?)

I don't understand why there is an /users/{ids}/inbox. Surely there only should be /users/me/inbox?

And it is nice to see the /events route (just wish there was webhooks or something as well.

About the filters, we must send a list of fields we want to a filters route, receive an id (I'm not sure what an opaque value is, sorry) which must then be passed to the actual route we wanted in the first place? That sounds very slow, especially on cellular connections. Couldn't it be possible just to send a list of fields we want back in the request header?

Improve this answer
3
  • There will be a /me/inbox route, as well as a /users/{ids}/inbox. The filter use case is "at start up, build the filters you need (or load them from a cache); then put the opaque return on each subsequent query". It's a constant number of extra queries over the lifetime of an app, not an extra query per "normal" query. It's also much more compact than sending a list of fields on each query, so it's actually ideal for the mobile case.
    – Kevin Montrose
    Commented Sep 17, 2011 at 22:34
  • Oh right that makes a lot more sense. So the filter stay on the server side for a while, or "forever"? When a user first uses the app it runs the filter routes, then saves the opaque values to disk (to keep them between app runs) and then never has to use the filter route again?
    – Jonathan.
    Commented Sep 17, 2011 at 23:13
  • a filter's lifetime will be "as long as we can possibly make it". There will be some provisions for if we have to invalidate a filter for whatever reason (bugs, infrastructure outages, whatever); but generally caching to disk will be safe.
    – Kevin Montrose
    Commented Sep 17, 2011 at 23:19
0

As a user of the API, rather than a library developer, the filter mechanism doesn't make me particularly happy as it adds a few lines of code that does not add immediate value to a program. I imagine however that the better libraries will come with a few 'inbuilt', 'precached' common opaque values to default with, so the end result is something that in the common case wouldn't be any harder to use.

I have no comment on the other changes, which mostly make me happy. Especially the /events route sounds fun to tinker with.

Improve this answer
2
  • 1
    The default returns (without any filter specified) will be reasonable, plus there are 10 or so built ins for common cases; so it's really not extra code (any more than ?body=true is with 1.x). However, excluding fields via a filter will let us tune the backing DB queries; making the end user experience better, since the API will return results faster.
    – Kevin Montrose
    Commented Sep 17, 2011 at 22:37
  • Rest assured that my PHP library will do everything :) I've been furiously planning it and it's really going to make development easier.
    – Nathan Osman
    Commented Sep 18, 2011 at 17:03

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .