So it was written, so it is done.

Go see the real deal!

var questionWithStuff =
    ctx // soapi context
    .Official // convenience properties for known sites
    .FromDate("january 1, 2001")
    .ToDate(new DateTime(2020, 10, 10))

Assert.AreEqual(6, questionWithStuff.Count);

// issues the following requests

// http://api.stackoverflow.com/1.0/questions?answers=true&body=true&comments=true&fromdate=978307200&max=100&min=1&order=Asc&page=1&pagesize=2&sort=Votes&tagged=asp.net%3bc%23&todate=1602288000
// http://api.stackoverflow.com/1.0/questions?answers=true&body=true&comments=true&fromdate=978307200&max=100&min=1&order=Asc&page=2&pagesize=2&sort=Votes&tagged=asp.net%3bc%23&todate=1602288000
// http://api.stackoverflow.com/1.0/questions?answers=true&body=true&comments=true&fromdate=978307200&max=100&min=1&order=Asc&page=3&pagesize=2&sort=Votes&tagged=asp.net%3bc%23&todate=1602288000

The question is, would anyone be interested in using code that looks like this? Just like this and does what it says it will do?

Before I go any farther in extending Soapi.CS and Soapi.JS, I need to know if there is interest. Otherwise I will spend my time writing some apps.

I am actively begging for constructive or informative feedback, positive or negative. Please. Sometimes it feels like I am writing in the wilderness. ;-(

// what you see is what you would get. Imagine similar in JavaScript
// while it looks like LINQ, it is not quite. I am simply planning to 
// use dynamic proxies (ala NHibernate) to lazy load from the API.


// initialize simply fetches all sites and prepares them with
// the necessary data and functionality to serve as the root for
// all queries for that site. 

// initialization is done on a static method and happens only once
// for each appDomain. (appdomain is a scope of execution, 
// an application for example, NOT a web domain)

ApiContext ctx = new ApiContext();
// an ApiContext instance maintains an objectGraph cache
// ensuring that when a new object is brought in from an http
// query, if it is already in the graph, update it in place,
// otherwise add it to the graph at the appropriate location

User codePoet = ctx.Sites("api.stackapps.com").User(14);

foreach(Question q in codePoet.Questions())
  foreach(Answer a in q.Answers())
    Console.WriteLine(a.Owner.DisplayName + " " + a.Owner.Reputation));

// so you noticed the lazy loading in the  expressions above - this is where it
// gets more interesting....

IEnumerable<Question> codePoetsQuestionsOnStackOverflow 
   = codePoet.UserAssociation.Users

// what you see is a navigation from my stackapps User up into
// it's UserAssociation object (stackauth /users/{id}/associated), 
// finds the user who's site is stackoverflow and then returns all
//  of my stackoverflow questions. 

// Again, there are no strings - this code will run and give you exactly 
// what it says it will.


The lazy loading proof of concept is working just as you see above and will be ready for release in a few days.

The idea of layering a LINQ provider on top of this is on hold. See Luke's answer.

And this is without LINQ. When/if I accept the task and finish implementing this, Dave and I have been talking about merging this with StackLINQ to provide the kind of client library that will let you stop worrying about how to get data and let you concentrate on doing something interesting with it.

// real live linq

var someInterstingQuestions = ctx.Sites("api.stackoverflow.com")
.Users.Where(u=>u.Reputation > 20000)
.Questions.Where(q=>(q.Title.Contains("foo") || 
q.Title.Contains("bar")) && q.UpVotes > 10).Skip(5).Take(10);

// the Skip is a multiple of the take, so get the 6th page of 10


Can we get some pre-orders please?

  • @kevin - yeah you are right, my bad. real tired but I have to keep an eye on the kid for a few more hours - sleep posting. Jul 25 '10 at 20:56

The following is simply my opinion so take it for what is is worth:

Whenever I see a LINQ provider built on top of a service, I become very interested to know how and when it is actually executing queries and if the abstraction is really saving me anything.

For example the statment:

var users = ctx.Sites("api.stackoverflow.com")
.Users.Where(u=>u.Reputation > 20000);

How is this query actually going to be executed? There is no way that I can tell from the API that you can query users by their reputation. So the best thing the LINQ interface can do for me here is to grab a list of users sorted descending by reputation and then just keep grabbing pages of users until it comes to a page which contains user which do not meet that reputation bar. So how many actual web calls need to be made to do this? Can it even be known before hand?

So I guess my point here is as a developer I need to have enough advantages from the abstraction of the LINQ provider to offset all of the little small things I have to worry about when using the provider.

In the case of LINQ to SQL I see a clear advantage because it can have enough smarts built in that it can actually write very good queries for you. In the case of a rest API like the SO API which are not really query based, I think the abstraction start to leak alot.

So the LINQ API you show above really becomes syntactical sugar over an abstraction that isn't exactly straight forward to understand. I guess it would remain to be seen if such a LINQ provider could provide enough value over just straight API calls to be worth it. Since I know how tricky it is to write a LINQ provider, I have to say I am skeptical at best.

provide the kind of client library that will let you stop worrying about how to get data and let you concentrate on doing something interesting with it.

In summary, I think what you propose is interesting but it would really come down to the implementation to see if the abstraction is worth it. Maybe I am just too concentrated on "how to get the data" ;)

  • luke, i really cant argue wih your points, they are valid and I often wonder why i want to do this. i usually end up agreeing with myself in that the ability to invoke a query from the from an arbitrary object, using it as the context of the call compelling. I will talk a little more about that. util thenas for the rep query, api.stackoverflow.com/1.0/… Jul 26 '10 at 1:23
  • Ok, after some sleep I find myself sharing your concerns. Certainly, the parametric considerations of querying each route would make providing an intuitive linq abstraction a challenge. the way I have thought about it is that the linq layer must be consistent and that a 1:1 mapping of api<->linq is not possible. But a solid subset, however limited, that is can be easily understood in both what is possible/allowed and what is not could lower the cover charge for a majority of the kinds of queries one might make. and the underlying documented parameterized per route library is always there Jul 26 '10 at 5:59
  • for when you need to be explicit, cannot express the query using the supported subset of linq, or simply cannot determine how to construct the expression. These scenarios would all seem to be heavily supported by a interceptor based fluent lazy/eager loading strategy layered on the domain model. I am going to get a POC for the first scenario in the Q tonight and hopefully will have good news. i sure do hate it when i get a harebrained idea and i am the only one who doesn't see immediately that it is harebrained. thanks for your time. Jul 26 '10 at 6:03
  • 1
    @lfo the more i think about it, the more i agree with you. LINQ is nifty, but the cost, for an implementation worth using, would be far too high. The reduction in resolution of the kinds of requests that could be made intuitively just doesn't would make it, at best, a novelty. I am going to pass on the linq but I have implemented lazy loading on the domain objects using the same parameter usage as on the route maps. And for stubs, e.g. partial nested objects, I just flag them and when a call to that property is intercepted I fetch it. I am really happy with the POC. Thanks for the dialog. Jul 26 '10 at 15:39
  • @code - yeah, I think the lazy loading is a very useful feature. I have it in the plans to add that to stacky at some point.
    – lfoust
    Jul 26 '10 at 17:01
  • luke. i finally settled on a custom IList<> implementation that provides a lot of linqiness for free from Enumerable extenstion and simply layered parameterized queries on top. results in an abstraction that lets you get as close to the metal as you like while retaining the promise of a '3 lines of code to get some data' abstraction. check out the soapi announcement if you are interested. Aug 5 '10 at 2:46

So would Soapi.js get something like this? If so, sure. I would love to see something like that.

I don't (and probably never will) use C#, so my opinion extends only to JavaScript.

  • yes. the javascript won't be as robust, but in many more ways it will be much easier to approximate this. I spent the last 2 weeks focusing on getting the js and cs base libraries stable, mostly the js, so i am going spike this in cs and then approximate as closely as possible in js. the library as you know it is not going to change an your code and knowledge will still apply. Simply view this as the 'wrapper for the wrapper' i alluded to in the beginning. Simply a higher level of abstraction for those who find value in it. Jul 25 '10 at 23:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .