Take the 2-minute tour ×
Stack Apps is a question and answer site for apps, scripts, and development with the Stack Exchange API. It's 100% free, no registration required.

Soapi.js


If you are looking for a full featured, compact JavaScript client library that provides complete end to end coverage of the API, Soapi.JS is a good choice.

You may also want to check out it's big brother, Soapi.JS2, which offers a rich relational query based metaphore.

Soapi.JS is not deprecated nor is it obsoleted by Soapi.JS2, each library has it's strengths and intended audience.


As easy as...

Soapi.RouteFactory('api.stackapps.com', 'key')
.UsersById(14)
.getResponse(function(data) {
    alert(data.items[0].display_name);
});

As expressive as...

Soapi.RouteFactory("api.stackoverflow.com", "key")
.Questions({
    // no unix timestamps here!
    fromdate: "1 Jun 2010 00:00:00", // use a string
    todate: new Date("1 Jun 2010 12:00:00"), // or use a Date. up to you.
    sort: "creation",
    pagesize: 10
})
.getResponse(function(data)
{
    alert(data.total);
});

About

Soapi.js is a compact single file, self-containted, self-documenting, intuitive and easy to use fluent JavaScript wrapper with full coverage of the Stack Overflow API.

Soapi.js is unobtrusive and has no external dependencies so it plays quite well with other libraries/frameworks.

Soapi.js is packaged with Visual Studio intellisense -vsdoc files and full documentation in html format.

Soapi.js includes advanced features including a configurable request caching buffer and throttled request queue.

Demos

A few online demonstrations of Soapi.js usage can be found here.

The demos are also contained in the zip download.

License

You may use Soapi.js under the terms of either the MIT License or the GNU General Public License (GPL) Version 2.

See http://soapi.info/code/js/license.aspx for more information.

Download

Current Version: Soapi.js 1.0 release 1.0 2010.07.09.00

The downloads and source are hosted on codeplex:

http://soapijs.codeplex.com/

You may view the documentation online here

While soapi.info is not a CDN, the latest version of the script will always be available at the following locations.

While discountasp.net is typically trustworthy, I make any guarantee up uptime other than what they provide me.

Platform

Soapi.js has been tested on all modern major browsers. Reports of platform related issues are welcome.

Contact

You can contact the author, me, Sky Sanders (code poet) through my gmail account: sky.sanders

Code

Soapi.js was developed leveraging the great JavaScript tooling and debugging capabilities of Visual Studio 2008.

Usage and Example Index

In the answers to this Question are reference information, usage examples and demo applications developed with Soapi.js.

Introduction To Soapi.js

How Do I....?

Apps created with Soapi.js

Any known app, great or small, written using Soapi.js will eventually be listed here.

Online Unit/Integration Tests

If you would like to help test the xbrowser compatability of Soapi.js, you may run a suite of tests in your browser or on your device. Thumbs up/Thumbs down reports in comments along with the platform and browser would be greatly appreciated.

The test suite will always be here.

share|improve this question
    
"soapy"! <!-- comment --> –  Maxim Zaslavsky Jun 10 '10 at 7:29
    
@maxim - yup. I know and I like it. ;-) –  Sky Sanders Jun 10 '10 at 8:56
    
@code, maybe I'm going blind, but how do you retrieve just a question? I see AnswersById, but this is only valid for answers, not questions, and I'm not using VS so I can't see the intellisense hints, and I've scoured the soapi.generated.js file to try and find the parameter and maybe I'm just tired... sigh –  Mark Henderson Jun 14 '10 at 1:03
    
@farseeker - see the answer A: How do I get a question? stackapps.com/questions/494/… –  Sky Sanders Jun 14 '10 at 1:29
    
@farseeker - I am finishing up the beta3 of soapi.cs and soapi.sl and getting that documentation generated. Then I will have to break out my old proof of concept javascript documentation generator, get it up to date and generate .chm and html docs for Soapi.JS as well. FWIW the method names are generated directly from the route path with a few formating conventions. With a little examination the pattern will become clear. –  Sky Sanders Jun 14 '10 at 1:33
    
@code - thanks, that's very helpful :) –  Mark Henderson Jun 14 '10 at 1:52
    
Turns out I was just being dense. I was using singlar instead of plurar (getQuestionById instead of getQuestionsById) –  Mark Henderson Jun 14 '10 at 2:02
    
@farseeker - the method names are not the prettiest, but as I intend all of the Soapi libraries to serve as solid core foundations upon which higher level libraries can be built I felt it would be best to generate all of the identifiers directly from the api specs. Once I have the core libraries up I will post examples of extending Soapi with lazy loaded object graphs. say, start with a Users, dot Questions will load questions if empty (Users.Questions.load/refresh are explicit directives) and so on throughout the object graph enabling fluent dot navigation across the entire api. –  Sky Sanders Jun 14 '10 at 2:30
    
@code: Wouldn't Doxygen be more what you're looking for? IIRC, it can generate CHM, HTML, LaTeX, etc. from Javascript code. –  Nathan Osman Jun 14 '10 at 3:18
    
@george - last time I checked it did not support javascript xml comments. I will take another look. I certainly do not look forward to managing this myself. I have other things I would like to do. Thanks for the heads up. –  Sky Sanders Jun 14 '10 at 3:27
1  
I just realized that Soapi.JS is licensed under the GPL v2 only. That's generally fine, appreciated and your choice anyway, however: I'm currently in the prototyping phase regarding some larger projects and have to decide which components to build these upon. While I intend to open source most derived components too (hard to avoid with JS anyway ;) the assembled apps might not be for various reasons (think e.g. GAE/GWT here). Are you considering dual licensing Soapi.JS in the future, e.g. (ideally ;) with the MIT License like jQuery or (understandably) a commercial license option? Thanks much! –  Steffen Opel Jul 1 '10 at 10:58
1  
To add some context: GPL licensing with JavaScript is obviously a complex matter, highlighted pretty well e.g. in Is this scenario in compliance with GPLv3?. A more concise summary might be GPL and Javascript. See Richard Stallmanns position regarding the The JavaScript Trap too, eventually, as he refers to input from John Resig of jQuery fame specifically. –  Steffen Opel Jul 1 '10 at 11:27
1  
@steffen - thanks for bringing this up and adding some well researched context to your question. Yes, initially i released soapi dual licensed ala jquery. I don't know when/why this changed in my mind but it just seemed to propagate throughout without questioning it. I will make it clear that it is (again) dual licensed and that until the next release where this will be codified, feel free to amend any license notice accordingly with my blessing. –  Sky Sanders Jul 1 '10 at 13:57
1  
@steffen - licensing has been updated here and in the package. thanks. –  Sky Sanders Jul 1 '10 at 14:35
1  
@code poet - Awesome, thanks so much! It would have been a real pity not being able to built upon your excellent work, which isn't just extremely productive, but really opened new design options regarding my project architecture, very inspiring! So thanks again, I really hope there's something in it for all of us in the long run, aside from all the fun, anyway :) –  Steffen Opel Jul 1 '10 at 19:34
show 22 more comments

15 Answers 15

Usage Example: Stack Exchange Site Monitor

JavaScript Site Monitor in 20 lines of code or less....

Demonstrates the use of Soapi.js to query stackauth.com to get an up-to-date list of Stack Exchange sites and then polling /stats for each one on a revolving timer.

alt text

$(document).ready(function siteMonitor() {

    var apiKey = '';
    new Soapi.RouteFactory("", apiKey).Sites().getResponse(function(data) {
        var sites = [];
        for (var i = 0; i < data.items.length; i++) {
            sites.push(data.items[i].api_endpoint.substring(7));
        }

        var siteIndex = 0;
        var delay = 10; // get it started
        var pollingInterval = 20000; // 20 seconds per site, > 1 minute between identical requests

        $.each(sites, function(idx, itm) {
            $('<div/>').addClass('stats').attr('id', itm.replace(/\./g, '')).appendTo(document.body);
        });

        function update() {
            var client = new Soapi.RouteFactory(sites[siteIndex], apiKey)
            .Stats()
            .getResponse(function(data) {
                var id = sites[siteIndex].replace(/\./g, '');
                $('#' + id).hide().fadeIn(100).jqotesub("#template", data.items[0]);
                siteIndex++;
                if (siteIndex == sites.length) {
                    siteIndex = 0;
                    delay = pollingInterval;
                }
                window.setTimeout(update, delay);
            });
        }
        update();
    });
});
share|improve this answer
add comment

Soapi.JS Usage Introduction

To use Soapi.JS, simply include soapi.js in your document.

e.g.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
   "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
<head>
    <title></title>
    <script src="scripts/Soapi.js" type="text/javascript"></script>
    <!-- the rest of your document -->

Soapi.JS is designed to be very simple to use. The only 'class' you will instantiate is RouteFactory.

RouteFactory

The RouteFactory requires 2 parameters:

  • targetSite (Required) - the full name of the API you wish to query. e.g. api.stackoverflow.com or http://api.stackoverflow.com
  • apiKey (Optional) - your application's registered apiKey. If you have no api key, API queries will be limited to 300 (or the latest limit) and requests will fail upon usage of that threshold. The JSONP mechanism provides no ability to access response headers and subsequently the x-rate-limit headers that provide usage metrics, so you will simply need to watch for timeouts and respond accordingly.

RouteFactory's raison d'être is to hold a reference to the target site and the api key and build Route objects via factory methods. NOTE: you do not have to, although you may, use the new keyword when instantiating a RouteFactory.

Soapi.RouteFactory('api.stackapps.com', 'api key if you gots one')
.UsersById(14)
.getResponse(function(data) {
    alert(data.items[0].display_name);
});

Next: Routes: what are they and how to use them

share|improve this answer
add comment

Routes: what are they and how to use them

Soapi.js defines Route 'classes' for each and every route, or url, in the StackOverflow API.

A Route is primarily a 'class' off of which to hang the url path and define the parameters for the url. Specifying required 'id' parameters in the factory method call and the application of parameters via explicitly defined parameter objects or json literals will be covered later in this post.

A Route, once constructed via the factory methods on RouteFactory, may be re-used as often as desired.

Route exposes two functions

  • getResponse(success, error, timeout).
  • getPagedResponse(success, error, pageCallback, timeout).

Route.getResponse parameters:

  • success (required) - function(data) - data will be a JavaScript object (jsob) built from the response json as defined on the route's help-page.
  • error (optional) - function(err) - err will be a jsob shaped like this: { code: number , message: "String", url: "String"}
  • timeout (optional) - number - The default timeout value is 10,000, or 10 seconds. If this is not acceptable, simply provide a timeout value in ms.

example:

var route = factory.Users();

route.getResponse(
    function success(data) {
        alert("response contained " + data.items.length + " users");
    },
    function error(err) {
        alert("Error: " + err.code + " " + err.message);
    },
    2000);

Route.getPagedResponse parameters:

  • success (required) - function(data) - data will be a JavaScript object (jsob) built from the response json as defined on the route's help-page.
  • error (optional) - function(err) - err will be a jsob shaped like this: { code: number , message: "String", url: "String"}
  • pageCallback (optional) - function(data) - data will be a JavaScript object (jsob) built from the response json as defined on the route's help-page. You may examine this object to determine if you would like to terminate the paged request. Simply return true if you wish to terminate and receive the current aggregated results in your success function.
  • timeout (optional) - number - The default timeout value is 10,000, or 10 seconds. If this is not acceptable, simply provide a timeout value in ms.

example:

var route = factory.Users();
route.page = 1;
route.pagesize = 100;

route.getResponse(
    function success(data) {
        alert("response contained " + data.items.length + " users");
        // should be 500, as we fetched 5 pages of 100
    },
    function error(err) {
        alert("Error: " + err.code + " " + err.message);
    },
    function pageCallback(data) {
         return data.page == 5;
    },
    2000);

RouteFactory Method Parameters

Routes that require an id, typically a semi-colon delimited list of identifiers, the factory method will accept id values as a param array, e.g. (id1,id2,id3,...) and construct the 'vectorized, semi-colon delimited' string for you.

example:

var route = factory.UsersById(1,2,3,14, { pagesize: 5});

resulting in a url resembling the following (url-encoded, of course):

http://api.xxx.com/0.8/users/1;2;3;14&pagesize=5&key=myApiKey

The other parameters of the route may be supplied as an object literal in the factory method, as shown, explicitly set on the Route object itself or appied via the Route.applyParameters function with an object literal.

'semicolon-delimited' parameters, which can be set in the factory method call, can be set as a string, ('1;2;3'), a param array, (1,2,3) or an array, ([1,2,3]), and are properly encoded by getResponse.

'unix timestamp..' parameters are set as JavaScript Date and are properly converted and encoded by getResponse.

Setting parameters directly on the Route object will, I think, be of more interest to Visual Studio users who may take advantage of the intellisense and code completion provided by soapi.generated-vsdoc.js. This topic will be covered in detail in the Intellisense Support post.

example in the form of a QUnit test:

asyncTest("AnswersByIdSanityCheck", function() {
    expect(1);
    var factory = Soapi.RouteFactory('api.stackoverflow.com', apiKey);
    var route = factory.AnswersById(2917525, 2701766, {
        //id is set in factory method call
        // these are default values that do not result in a 
        // parameter being added to the url
        body: false,
        comments: false,
        sort: "",
        order: "",
        min: 0,
        max: 0,
        fromdate: 0,
        todate: 0,
        page: 0,
        pagesize: 0
    });



    route.getResponse(
      function (data) {
          ok(data.items.length==2, 'async call succeeded, count inaccurate');
          start();
      }, 
      function(err) {
          ok(err.message);
          start();
      }, 5000);

example direct member set (with code completion and inline documentation) in the form of a QUnit test:

asyncTest("AnswersByIdSanityCheck3", function() {
    expect(1);
    var factory = Soapi.RouteFactory('api.stackoverflow.com', apiKey);
    var route = factory.AnswersById(2917525, 2701766);

    route.body = false;
    route.comments = false;
    route.sort = "";
    route.order = "";
    route.min = 0;
    route.max = 0;
    route.fromdate = 0;
    route.todate = 0;
    route.page = 0;
    route.pa


    route.getResponse(
      function(data) {
          ok(data.items.length == 2, 'async call succeeded, question count inaccurate');
          start();
      },
      function(err) {
          ok(err.message);
          start();
      }, 5000);
});

alt text

Next: Complete Route Listing - suprisingly simple mastery of the API via Soapi.js lies here

share|improve this answer
add comment

Enumerated Constants, or enums, in Soapi.js

The advantages of using enums include the elimination of magic strings scattered about your codebase and the ability to update all references to an enum value in one location.

Typically, in a statically typed language, enums are readonly or immutable values. While this concept is alien to the dynamically typed JavaScript language the benefits of enums are not mitigated. You simply need to treat them as immutable and not assign values to them.

Soapi.js defines 'enums' for all of the 'choice' parameters and return types in the API. Using enums may seem like more work in that they are more verbose than using string constant but in reality they make your code more robust and much easier to read and maintain.

For example:

var $sd = Soapi.Domain;

// as a parameter

route.Sort = $sd.UserSort.reputation;


// identifying values in return types

switch(user.user_type){
   case $sd.UserType.anonymous:
      // do something 
      break;
   case $sd.UserType.unregistered:
      // do something 
      break;
   case $sd.UserType.registered:
      // do something 
      break;
   case $sd.UserType.moderator:
      // do something 
      break;
}

Enums used in parameters

SortOrder

  • desc
  • asc

PostSort

  • activity
  • views
  • creation
  • votes

CommentSort

  • creation
  • votes

QuestionSort

  • activity
  • votes
  • creation
  • featured
  • hot
  • week
  • month

TagSort

  • popular
  • activity
  • name

UserSort

  • reputation
  • creation
  • name

FavoritesSort

  • activity
  • views
  • creation
  • added
  • votes

Enums used in return types

SiteState

  • normal
  • closed_beta
  • open_beta
  • linked_meta

UserType

  • anonymous
  • unregistered
  • registered
  • moderator

PostType

  • question
  • answer

PostTimelineType

  • question
  • answer
  • comment
  • revision
  • votes
  • state
  • accepted
  • unaccepted

RevisionType

  • single_user
  • vote_based

UserTimelineType

  • comment
  • askoranswered
  • badge
  • revision
  • accepted

Users of Visual Studio will get extra benefit from the use of the defined enum types as they have the luxury of intellisense code completion, which will be covered in detail in the Intellisense Support post.

alt text

Next: Studio JavaScript Intellisense and Code Completion Support

share|improve this answer
    
Does this work with the Express editions of Visual Studio? –  Nathan Osman Jun 12 '10 at 23:36
    
@George, intellisense you mean? sure, I don't see why not. Just pay attention to the Intellisense Support instructions and you should have no problems. Let me know in a comment under the Intellisense Support answer. –  Sky Sanders Jun 12 '10 at 23:45
add comment

Visual Studio JavaScript Intellisense and Code Completion Support

Overview

Included in the .zip is soapi-vsdoc.js. This file contains JavaScript code documented following the Visual Studio JavaScript Intellisense Documentation format.

This file, while of special interest to users of Visual Studio, contains complete documentation of the API, including all enumerated constant values and route parameter specifications.

For Visual Studio users: When using Soapi.JS in Visual Studio, the -vsdoc.js file simply needs to reside in the same directory as the soapi.js and soapi.generated.js files. Visual Studio will

If you are using Visual Studio 2008, you may need to apply the JavaScript Intellisense patch. The patch specifies X86, but I can assure you that the same patch is applicable to both x86 and x64.

Enums

As described in the Enumerated Constants post, soapi.js define enums that can help you write stronger code against the Soapi.JS library.

In addition to the interesting 'enum' construct provided by soapi, the soapi-vsdoc.js file enhances the value of the enum types by adding Visual Studio Intellisense and Code Completion via the use of XML Documentation Comments.

alt text

Factory Methods

The factory's intellisense describe the route to be built:

alt text

The factory method's intellisense describe the route's 'id' parameters:

NOTE: The documentation is generated directly from the API, so it is assumed that the programmer will know that the 'semicolon-delimited string' description, in the context of Soapi.JS, describes, in the factory method call, a param array, and when setting parameters directly on the route, an array, which will be properly converted and encoded by getResponse.

alt text

The route's fields describe themselves:

NOTE: The documentation is generated directly from the API, so it is assumed that the programmer will know that the 'unix timestamp' description, in the context of Soapi.JS, describes a JavaScript Date, which will be properly converted and encoded by getResponse.

alt text

Convenience Constructors

When becoming acquainted with a new library, it is sometimes helpful to have code completion assistance and intellisense documentation for the objects returned from API calls.

By leveraging the vsdoc file and a special constructor function, we can build a 'casting' constructor that, if the object passed in is an instance of itself, simply return it, otherwise apply the object passed in to a new instance. This provides intellisense, code completion and method chaining via type inference.

The properties of the response type are enumerated:

alt text

The fact that users is an array is demonstrated by the type inference provided by Visual Studio.

alt text

Casting an element of the users array to Soapi.Domain.User results in the same effect.

alt text

While this added functionality may seem trivial, in a scenario where one is dealing with a new/beta/constantly shifting API, this type of assistance can be a great productivity boost. Especially on objects that have many many members and nested objects.

It is Visual Studio's type inference functionality, driven by the -vsdoc.js file, that enables complete intellisense code-completion throughout the compact, or 'fluent', example that is given at the head of the question to this answer.

Next: Date Handling in Soapi.js - no more greasy unix timestamps.

share|improve this answer
add comment

soapi.js vs soapi-vsdoc.js

While the -vsdoc.js is quite large, it is never served. It is, if you are using Visual Studio, used to provide intellisense services, otherwise you may either delete it or keep it around as reference.

The file that actually defines the domain types at runtime, soapi.js, is quite small, 9.5k packed, in that it only contains meta data that is used at 'compile' or more accurately parsing time to dynamically build the domain objects.

There is a slight performance hit to be taken when using new Function() when compared to inline code but I have not seen that to be an issue, especially when you are dealing with network requests. It is my observation that an app using Soapi.js will be network bound as opposed to CPU bound.

share|improve this answer
add comment

A: How do I get a question?

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
    <title></title>
    <script src="scripts/Soapi.js" type="text/javascript"></script>
    <script src="scripts/Soapi.Generated.js" type="text/javascript"></script>
    <script type="text/javascript">

        function run() {

            var apiKey = '';


            var route = Soapi.RouteFactory('api.stackapps.com', apiKey).QuestionsById(494);
            route.answers = true;
            route.body = true;

            var response = route.getResponse(
            function success(data) {

                alert(data.items[0].title);
            },
            function error(err) {
                alert("Error:" + err.message);
            });


            // or
            Soapi.RouteFactory('api.stackapps.com', apiKey)
            .QuestionsById(494,{
                answers: true,
                body: true
            })
            .getResponse(
            function success(data) {

                alert(data.items[0].title);
            },
            function error(err) {
                alert("Error:" + err.message);
            });
        }
    </script>
</head>
<body onload="run()">
</body>
</html>
share|improve this answer
add comment

Complete Route Listing

The RouteFactory's factory methods are consistently and intuitively named by convention to make banging out code without intellisense code completion as painless as possible.

Here is a complete list of the RouteFactory methods and the API equivalent. Starting to see a pattern? ;-)


Api Routes

  • AnswersById = /answers/{id}
  • AnswersByIdComments = /answers/{id}/comments
  • Badges = /badges
  • BadgesById = /badges/{id}
  • BadgesName = /badges/name
  • BadgesTags = /badges/tags
  • CommentsById = /comments/{id}
  • ErrorsById = /errors/{id}
  • PostsByIdComments = /posts/{id}/comments
  • Questions = /questions
  • QuestionsById = /questions/{id}
  • QuestionsByIdAnswers = /questions/{id}/answers
  • QuestionsByIdComments = /questions/{id}/comments
  • QuestionsByIdTimeline = /questions/{id}/timeline
  • QuestionsUnanswered = /questions/unanswered
  • RevisionsById = /revisions/{id}
  • RevisionsByIdByRevisionguid = /revisions/{id}/{revisionguid}
  • Search = /search
  • Stats = /stats
  • Tags = /tags
  • Users = /users
  • UsersById = /users/{id}
  • UsersByIdAnswers = /users/{id}/answers
  • UsersByIdBadges = /users/{id}/badges
  • UsersByIdComments = /users/{id}/comments
  • UsersByIdCommentsByToid = /users/{id}/comments/{toid}
  • UsersByIdFavorites = /users/{id}/favorites
  • UsersByIdMentioned = /users/{id}/mentioned
  • UsersByIdQuestions = /users/{id}/questions
  • UsersByIdReputation = /users/{id}/reputation
  • UsersByIdTags = /users/{id}/tags
  • UsersByIdTimeline = /users/{id}/timeline
  • UsersModerators = /users/moderators

StackAuth Routes

  • Sites = /sites
  • UsersByIdAssociated = /users/{id}/associated

NOTE: the stackauth methods/routes are available on every RouteFactory instance and are not tied to a particular target site. In fact, you may simply instantiate a RouteFactory with an empty string for target. This is useful in that you need to fetch /sites (Sites) in order to get an authoritive list of api_endpoint to use as target.


So, to recap, the basic hello world for Soapi.js looks like this:

var factory = Soapi.RouteFactory("api.stackapps.com","mykey");

var route = factory.UsersById(14);

route.getResponse(function success(data){
   alert(data.items[0].display_name); //code poet!
}, function failure(error){
   alert(error.message);
});

Remember: the data returned is the JavaScript object returned from the API method call. If you are in doubt as to what is being returned, simply look at the API help page for that route.

The one exception that I have made is to abstract the array property of all 'returns' types to items. This enables more powerful application patterns.

With an understanding of this simple code snippet, you can now easily and freely explore the Stack Overflow API with Soapi.js.

Next: Enumerated Constants, or enums, in Soapi.js

share|improve this answer
add comment

Date Handling

As of RC2, Soapi.js is Dates in, Dates out. No more dealing with unix timestamps.

This includes input parameters, todate, fromdate, min and max (when applicable) and all response object date properties e.g. creation_date is converted to a JavaScript date object before it is passed to your success function.

todate, fromdate may also be specified using string, e.g. "1 jun 2010 00:00:00".

min and max are special 'variant' parameters and when dates are appropriate you must explicitly set a Date object.

Example

<script src="scripts/Soapi.js" type="text/javascript"></script>

<script type="text/javascript">
    // demonstrate dates in / dates out
    // no need to deal with unix timestamps in Soapi.js

    // here are some date objects
    var fromDate = new Date("Tue, 1 Jun 2010 00:00:00");
    var toDate = new Date("Tue, 1 Jun 2010 01:00:00");

    // get some questions
    // use min/max parameters with date objects
    // dates are converted to unix timestamps to build the url

    Soapi.RouteFactory("api.stackoverflow.com", "").Questions({
        min: fromDate,
        max: toDate,
        sort: Soapi.Domain.PostSort.creation,
        pagesize: 1
    })
    .getResponse(function(data) {
        // unix timestamps are converted to Date in response before you get them
        alert("min/max\r\nThis date should be between Tue, 1 Jun 2010 00:00:00 and Tue, 1 Jun 2010 01:00:00 \r\n\r\n"
            + data.items[0].creation_date + "\r\n\r\n Dates In - Dates Out.\r\nNeat, huh?");
    }, function(error) { alert(error.message); });

    // use fromdate/todate parameters with date objects
    // dates are converted to unix timestamps to build the url

    Soapi.RouteFactory("api.stackoverflow.com", "").Questions({
        fromdate: fromDate,
        todate: toDate,
        sort: Soapi.Domain.PostSort.creation,
        pagesize: 1
    })
    .getResponse(function(data) {
        // unix timestamps are converted to Date in response before you get them
        alert("fromdate/todate\r\nThis date should be between Tue, 1 Jun 2010 00:00:00 and Tue, 1 Jun 2010 01:00:00 \r\n\r\n"
            + data.items[0].creation_date + "\r\n\r\n Dates In - Dates Out.\r\nNeat, huh?");
    }, function(error) { alert(error.message); });        

</script>

Next: Building Soapi.js

share|improve this answer
    
+1 for this extremely useful improvement! Dealing with Unix timestamps in an application layer is always annoying and a perfect job for a library/wrapper as such, much appreciated! –  Steffen Opel Jul 3 '10 at 14:51
1  
@steffen - if you would like to submit any other "i wish soapi did xxxx" as individual comments on the main question you just might see them in the next release. –  Sky Sanders Jul 3 '10 at 17:46
add comment

Packaging Soapi.js

The process of packaging Soapi.js for distribution may be of interest to users of Soapi.js as well as others interested in using Visual Studio to automate the build and packaging of JavaScript resources, including documentation.

Requirements:

Generating the documentation

The documentation of Soapi.js is built from Soapi-vsdoc.js comprised of two parts:

  1. Visual Studio Xml Documentation Comments to support intellisense and code completion
  2. JavaDoc style comments for use in generating HTML documentation with JsDoc Toolkit

example:

/**
* Global success handler. Replace (override) this method. This event is raised before the instance success function.
* @function
* @static
* @param {Object} data The response data
* @param {Object} [context] An arbitrary user defined value/object
*/
Soapi.onSuccess = function(data, context)
{
    /// <summary>Global success handler. Replace (override) this method. This event is raised before the instance success function.</summary>
    /// <param name="data" type="Object">The response object</param>
    /// <param name="context" type="Object">An arbitrary user defined value/object</param>
};

There are a few reasons for the duplicated effort.

  1. There is currently no public means of generating vs doc xml from POJO code. I have an abandoned and partially implemented project but have not taken the time to revive it for reasons that will become clear.
  2. The JsDoc format is far more expressive than is the Xml Doc format and better than mix those concerns, I find it a much cleaner, albeit labor intensive, approach to use JsDoc Toolkit for generating hard copy docs.

There are a few features that JsDoc Toolkit does not support, so there are some post processing that needs to be done.

  1. update the build number in both the doc file and the script
  2. run JsDoc Toolkit to generate the doc site from the vsdoc (jsdoc) file into /build/docs
  3. post process the output to clarify enums and create linked func parameters
  4. run a search and delete of JsDoc comments to output a smaller vsdoc file.
  5. write a vsdoc for Soapi.js and Soapi.min.js to the /build/scripts directory

Packaging The Distribution

  1. copy all of the content and third party demo support scripts to build directory
  2. copy the scripts and tests to the /build/scripts directory
  3. use ajax minifier to compress the script and copy to the /build/scripts directory
  4. gzip the minified script and copy to the /build/scripts directory
  5. build 3 versioned dowload files in the /download
    1. zipped docs only
    2. zipped scripts only
    3. all content including docs, scripts, tests, demos and examples
  6. manually review tests and deploy to public sites

All of this could be built into a custom MSBuild task, but for now it is being performed by the included console application, Soapi.JS.Deploy, that is run on successful builds.

Next: Request Caching

share|improve this answer
add comment

Request Caching

In the interest of application performance and to prevent inadvertent api 'abuse', Soapi.js includes an integrated request cache that stores the results of each unique request url for a default (and minimum) period of 60 seconds.

The caching duration may be increased for individual routes as per your application's needs.

e.g. perhaps your application requires that the stackauth site list be current to only 10 minutes you may set an increased cache duration as follows:

Soapi.RequestCache
.setDuration(Soapi.Routes.SitesRouteMap.routeFormat, 60000 * 10);

After a successful response, subsequent requests to /sites within the cache duration will be returned from the in-memory cache. Upon expiration of the cache duration, the data for the particular route is purged from memory and the next request will be forwarded to the API.

Next: Throttled Request Queue

share|improve this answer
    
that's a deep and detailed work, good job. I promised to myself that i will study this bunch of great stuff soon :). –  systempuntoout Jul 20 '10 at 6:44
    
@system - thanks dude, it is a labor of love/hate and I am pleased to be nearing a point when I can concentrate on bringing soapi.cs to the same level and then maybe actually finish my apps. –  Sky Sanders Jul 20 '10 at 7:49
add comment

Throttled Request Queue

To provide transparent compliance with the throttle limite, Soapi.js includes an integrated throttled request queue.

The queue is integrated into the library and is transparent, requiring no additional coding or consideration.

The current default queue interval is 170ms.

This means that, with the request queue and request cache, you may make requests with impunity and without concern for 503 errors or API abuse and all requests.

Requests that are cached are not subject to queue delays.

The request queue throttle interval can be adjusted from the current default of 170ms via the static Soapi.RequestQueue object e.g.

Soapi.RequestQueue.setInterval(intervalInMS);

Care must be taken to self-control bursts so as to comply with throttling rates, otherwise unexpected behavior can, well.. be expected.

As a convenient side effect of implementing a request queue, throttled or otherwise, is the ability to provide feedback when your application is busy requesting data. This capability is implemented via global 'event' handlers and can easily facilitate UI blocking or manipulation e.g. ajax loading mask or other visual cues.

You can find a simple example of this in StackUsers Lite

/******************************************************
* A cheap auto loading mask by overriding the queue events
* Whenever the queue goes active or complete, wait 100ms
* to make sure the event is not transient and then check
* the queue length and show/hide mask as indicated.
* could use jQuery/BlockUI
*/

Soapi.Events.onQueueActive = Soapi.Events.onQueueComplete = checkMask;

function checkMask()
{
    // want a little buffer to eliminate mask flicker
    window.setTimeout(function()
    {
        if (Soapi.RequestQueue.getLength() == 0)
        {
            unmask();
        }
        else
        {
            mask();
        };
    }, 100);
};

another sample:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
    <title>NGINX Throttle Violations</title>

    <script src="http://soapi.info/code/js/stable/scripts/Soapi.js" type="text/javascript"></script>

    <script type="text/javascript">

        var apiKey = 'qgAq_KfDu0KYzlNG-qaTuw';



        // default will be set to 170 in next release
        Soapi.RequestQueue.setInterval(170); // 30 per 5 sec = 6 per sec = interval 166.6 ms

        // see http://stackapps.com/questions/1143/request-throttling-limits


        // get a bunch of associations from stackauth
        window.setTimeout(function()
        {
            Soapi.RouteFactory("api.stackoverflow.com", apiKey)
            .Users({ pagesize: 100, page: 1 })
            .getResponse(function(data)
            {
                for (var i = 0; i < data.items.length; i++)
                {
                    var id = data.items[i].association_id;

                    if (id)
                    {
                        Soapi.RouteFactory("", apiKey)
                        .UsersByIdAssociated(id)
                        .getResponse();
                    }
                };
            });
        }, 100);

        window.setTimeout(function()
        {
            // get a bunch of tags from meta at the same time - all should be queued and executed with no error

            Soapi.RouteFactory("api.meta.stackoverflow.com", apiKey)
            .Tags({ pagesize: 1 })
            .getPagedResponse();

        }, 10);
    </script>

</head>
<body>
</body>
</html>

Next: Paged Requests

share|improve this answer
add comment

Paging Operations

The majority of operation against the API involve paging operations.

Most of the work and complexity of dealing with paged results have been encapsulated by the Route.GetPagedResponse() method.

Soapi.RouteFactory("api.stackoverflow.com", apiKey)
.Tags({ pagesize: 1 }) // contrived page size
.getPagedResponse(function success(aggregatedPages)
{
    // when the paging operation completes, whether due to completion of
    // paging or arbitrary termination in pageCallback, the aggregated data is
    // returned to this, the success handler.

    // should be the aggregated total of 10 pages of 1 item or simply 10
    alert(aggregatedPages.items.length);

}, function failure(error)
{
    // handle the error

}, function pageCallback(currentPage)
{
    // you can handle the data page by page in this handler or 
    // aggregated in total in the success handler

    // simply omit this function parameter if you want all 
    // pages returned to the success handler

    // return true from this function to terminate paging operation 
    // and return currently aggregated data to success handler.

    // we will report progress on the paging operation and constrain
    // the paging operation to 10 pages

    reportProgress(currentPage.page);

    // return true to terminate operation
    return currentPage.page == 10;
});

Next: VectorizedIdList - reliably maximize request payload

share|improve this answer
add comment

VectorizedIdList

Many routes accept a 'vectorized list of identifiers'. This allows you to retrieve data for multiple items and helps maximize efficiency and performance.

The problem is that the 'vectorized list' is part of the url path, which much remain below a certain length (around 240) to avoid 400 (Bad Request) errors.

the VectorizedIdList allows you to add an arbitrary number of values and will emit batches that will result in a url that is withing length tolerances.

You can then simply iterate the batches and retrieve your results as shown below

var apiKey = 'qgAq_KfDu0KYzlNG-qaTuw';
var factory = Soapi.RouteFactory("api.meta.stackoverflow.com", apiKey);
factory.Users({ pagesize: 100 }).getResponse(function(data)
{
    // 240 is a safe value for max path length otherwise you risk 400 (Bad Request) errors
    // See http://stackapps.com/questions/889/dev-tip-how-to-batch-up-multiple-id-requests-to-maximize-economy-and-still-avoid/1021#1021

    var idList = new Soapi.Utilities
        .VectorizedIdList(240 - Soapi.Routes.UsersByIdReputationRouteMap.routeFormat.length);
    for (var i = 0; i < data.items.length; i++)
    {
        idList.add(data.items[i].user_id);
    }

    // so now we have a VectorizedIdList loaded with 100 user id values
    var batches = idList.getBatches();

    // batches:            
    //    [0]: [1, 811, 22656, 22164, 23354, 658, 9314, 18393, 98786, 130154, 3043, 2915, 115866, 1288, 12950, 132636, 59303, 1782, 13295, 22459, 136558, 17174, 52443, 16587, 18511, 14048, 130127, 59808, 27414, 132480, 23921, 2598, 13531, 4660, 52738, 1069, 2509]
    //    [1]: [131779, 95589, 893, 33708, 55747, 138112, 146126, 130914, 745, 131932, 139541, 130090, 101371, 140478, 2961, 141911, 132632, 75852, 135200, 1337, 54262, 8435, 76337, 137798, 69307, 3, 56338, 138822, 89334, 4, 91, 133733, 3724, 572, 102937, 47529, 137184, 5190]
    //    [2]: [130109, 74022, 84237, 419, 131713, 42471, 132223, 9345, 44765, 132073, 6309, 14148, 56555, 6258, 16417, 20654, 44330, 16487, 68727, 2, 146719, 133301, 140171, 63550, 130494]
    //    length: 3

    for (var i = 0; i < batches.length; i++)
    {
        // each batch will be a list of user id that will contain the
        // maximum number of items and still remain under the 240 length limit

        factory
        .UsersById(batches[i], { pagesize: batches[i].length })
        .getResponse(function(data)
        {
            // do something interesting with your batch of users here
            alert(data.items.length);
        }, function(error)
        {
            alert(error.message);
        });
    }


});
share|improve this answer
add comment

I was a little too lazy to read through everything on this page...

Does Soapi.JS slow down requests to prevent 503 errors when necessary?

If not, would you add that code if I wrote it?

share|improve this answer
    
seriously!? you only have to read as far as the question. stackapps.com/questions/494/… –  Sky Sanders Aug 11 '10 at 20:21
    
@code: Well, you have to admit that this page is quite large... and I figured it would take you all of 2 seconds to tell me. –  Nathan Osman Aug 11 '10 at 20:55
    
I am way ahead of you.. In any post that I make that contains help answers you will find an index of topics. You see, I was not too lazy to index so that you, my dear user, would not have to read every answer. or the bold header to every answer post. lmao. –  Sky Sanders Aug 11 '10 at 21:06
    
@code: Sorry, one more question - does Soapi.js take timezones into consideration? –  Nathan Osman Aug 14 '10 at 15:16
    
yes. to/from utc is handled by the lib. dates are relative to your local time. –  Sky Sanders Aug 14 '10 at 15:33
    
@code: Thanks - also, is there a way to get the original timestamp? (Not the JavaScript Date() object one.) –  Nathan Osman Aug 14 '10 at 17:34
    
@geo - Date.getTime() is UTC, so just take the local date that soapi gives you, call .getTime() and divide by 1000. e.g. originalTimestamp = question.creation_date.getTime()/1000; –  Sky Sanders Aug 16 '10 at 18:07
    
@code: Thanks! I updated StackCenter so that it now informs you of new comments that have been posted since your last visit. –  Nathan Osman Aug 16 '10 at 18:45
add comment

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .