24

alt text

About

http://stackusers.com

All Stack Exchange users at your fingertips.

The core of this application is a robust data acquisition and maintenance platform based on Soapi.CS that reliably and economically maintains a full user info data set.

Features

Current features include:

  • Find Stack Exchange Users By UserName OR Email
  • Query Stack Exchange users via Filtered Sortable grid and/or Entity SQL statement
  • CSV to Clipboard export
  • Drill down functionality on any user
  • Optional client side live update via Soapi.JS
  • Generate Ranked and Aggregated Custom Flair
  • JSON API
  • your idea here...

What's Next?

It is up to you!

  • Your feature requests are welcomed.
  • A planned feature, for registered users, is local hosting of JavaScript, both server and client side, that can access the underlying dataset maintained by Soapi and present it in any way you choose.

Cost

Free for everyone. Always.

If you would like to help mitigate hosting costs and possibly upgrade the hosting plan for faster response and perhaps a more open API, means to do so at no cost present themselves to you on every page of the site. ;-)

Platform

Browser Based

Contact

Sky Sanders

Code

Stack Users is the first application deployed upon a robust server side background processing engine creating using the Soapi.CS/Soapi.JS2 client libraries.

NOTE: This site is me learning MVC2 as you watch, while breaking in a new web/data host and dog-fooding Soapi all at the same time. Expect the unexpected ;-)

31
  • 2
    @Sky that's a huge project Sep 14, 2010 at 10:48
  • Wow. I likes. +1. Small I18 note - any chance the dates could be formatted YYYY-MM-DD? Makes it easier for those of us outside the US to write dates where the M and D are < 12 Sep 14, 2010 at 12:12
  • 1
    Usage note: To filter on a site use Contains and a unique portion of the site name. To exclude a site you need to specify the full URL including a trailing slash: http://example.com/. Sep 14, 2010 at 15:10
  • @dennis - yes - i am right now stepping through the client side JS so that I can let base on an entity sql statement. Also - r.e. existing grid filter et al - The grid is just a stock grid I threw on top of the data store - the ui will be customized to the use cases presented by interested users such as your self. I intend to modify the filter popup for the site field to enable things like exluding linked meta etc. I suppose a wiki or feedback section for each page would be a good idea to collect and make discoverable user feedback.. Sep 14, 2010 at 16:10
  • @dennis - not sure what you mean by stable sort? would sorting on multiple columns be what you mean? Sep 14, 2010 at 16:16
  • Would you provide an API so that we can get a user_id based on email? Sep 14, 2010 at 17:09
  • I get an error when I try to save my account details. Sep 14, 2010 at 17:19
  • @geo - r.e. api - that is being considered but i first need to decide how to manage many of the same concerns regarding acceptable usage that we have had here. I am just a little guy paying for bandwidth myself so this will have to be considered closely. r.e. error - give me a proper description of the error so that i can respond.... Sep 14, 2010 at 17:25
  • @Sky: Just the generic "Error in / application" error. Sep 14, 2010 at 18:42
  • @geo - not sure the problem - i see valid info and verified email address for you. Sep 14, 2010 at 19:08
  • @dennis check the esql pre-filter in addition to the column sort/filter and see if you can get desired results. let me know. Sep 14, 2010 at 19:09
  • @Sky: Well I'll try it again. BTW, my website has been down for the last half-hour :P Sep 14, 2010 at 19:37
  • 1
    @dennis - i initially had multi-sort enabled, but the latency between the app server and the sql server combined with 360,000 records made it unbearable. The new host eliminates this problem so I re-enabled the multi-sort after your reminder. thanks. Sep 15, 2010 at 5:30
  • 8
    I can't get this to work anymore. The home page is 500-ing. May 17, 2011 at 4:02
  • 1
    The stackusers.com domain name has expired... Jul 3, 2013 at 19:01

7 Answers 7

4

Querying Stack Exchange Users

alt text

You may leverage the Filtering and Sorting capabilities of the Ajax grid to shape results based upon the entire Stack Exchange user base.

You may export the contents of the current grid page to CSV.

1
  • Just got a chance to try this... very flexible. +1 Sep 17, 2010 at 21:22
3

Find Stack Exchange Users By UserName OR Email

alt text

You may find any user in the Stackiverse by the email address they supply when registering on any Stack Exchange site or by full or partial user name (display_name).

JSON API

{
    "email_address": "[email protected]",  // yes - this is a real account
    "items": [{
        "site_url": "http://stackoverflow.com/",
        "user_id": 370536,
        "association_id": "00000000-0000-0000-0000-000000000000"
    }]
}

Programmatic access to this data may be granted by IP address. Please email [email protected] with a brief explaination of the intended usage and expected traffic pattern.

Once you are authorized, you may access this data via /users/email/[email protected]

This endpoint will be discontinued shortly. Please see the announcement of the StackUsers JSON API

3
  • I'm now using this in the global StackImage wizard. (There's a link to it at the bottom of this page.) Sep 19, 2010 at 17:08
  • @geo - glad you find it useful. your wizards are smooth. Sep 19, 2010 at 19:34
  • As good as it is, I use my Gmail account to log on to this site, but it isn't public... how/why does it link to my account? Is the login email address public some how?
    – wilhil
    Feb 11, 2011 at 23:12
3

JSON API


Note:

I am just one developer and I do not have the time/money/resources to maintain an industrial grade public data source. If it is determined for any reason that you are not using the API in the capacity and terms indicated, your IP will be banned with a quickness.

I will shortly implement a private key system. Those interested in using the API now should monitor this post for the announcement of the process to get a key. There will be a 1 week grace period once the key system is in place for you to get a key. After which non-keyed requests will be denied and abusive IPs will be banned.

When and if a need for more resources and a way to finance it presents itself this policy will be reconsidered.

If you would like to help mitigate hosting costs and possibly upgrade the hosting plan for faster response and perhaps a more open API, means to do so at no cost present themselves to you on every page of the site. ;-)


User Search

   http://stackusers.com/1.0/users/{term}/search

Where term is an email address or a full or partial display name.

Example:

http://stackusers.com/1.0/users/[email protected]/search

Results:

{
    "term": "[email protected]",
    "items": [{
        "site_url": "http://stackoverflow.com/",
        "site_name": "Stack Overflow",
        "endpoint": "http://api.stackoverflow.com/",
        "user_id": 370536,
        "association_id": "00000000-0000-0000-0000-000000000000",
        "email_hash": "f3ada405ce890b6f8204094deb12d8a8",
        "display_name": "Cale Gibbard",
        "reputation": 111
    }]
}

Results are limited to the first 50 matches. There is no paging. If you cannot find what you are looking for in 50 responses, be more specific.

How It Works:

If the incoming term is an email address, all of the users with either matching email_hash or association_id are returned. This will catch the case of a user that registers on multiple sites with different email addresses. (not sure if this is a valid case, i would have to check with jeff but it is covered anyway)

If the incoming term is not an email address, a simple if(DisplayName.Contains(term)) is performed.

TERMS:

This route is provided as a service to developers of applications and websites that would like to streamline the user lookup/identification process.

Again, this route is meant to be used for interactive processes as a means for your application to identify a user in response to an interaction with your end user.

You are encouraged to persist the results of a successful match.

You should not treat this, or any StackUsers endpoint, as a bulk data store. You may build your own data store from the Stack Exchange API.

If you have any questions about these terms please email [email protected].

1
  • Thanks for letting me know about the move. I fixed StackImage and it's using the new URL now. Keep up the good work! Sep 21, 2010 at 21:30
2

Client-side Live Updates


With 370,000 rows of user data to maintain, it is an impossible task to keep every record updated up to the minute.

To mitigate relatively stale data on volatile fields such as last_access_date, StackUsers offers optional client-side updates of results using Soapi.JS2.

You must opt in by either clicking the Enable Live Update button or ticking the appropriate check box in your account info page.

How it works

When Live Updates are enabled, as data is fetched for the grid, it is preprocessed and fresh data is pulled from the api by your browser using Soapi.JS2.

Differences are indexed and the dataset is presented in the grid.

Both datasets are available for viewing and fields that have changed are indicated by a visual cue.

A mouseover tooltip will present the previous and current values.

You may toggle the datasets, as shown below, by clicking the Show Live Data button.

CSV export is aware and will produce data for the current grid view.

NOTE: Sorting is server-side and is not aware of the client side live updates (yet?) so if the Show Live Data mode is active it is the underlying dataset that is being sorted and the visible results may not appear to be sorted. Just an FYI.


From Database

With Live Updates

Convenient Tooltips

alt text

Details

You may monitor all requests issued by Soapi.JS2 by opening the StackUsers Client Log window.

All requests are batched to make the client side processing economical and fast.

The dot next to the Soapi icon acts as a visual cue as to the current state of client side processing.

All site data and site icons are cached and served from StackUsers to mitigate load on the API server(s?) as well as reducing rate-limit consumption.

alt text

2
  • 1
    Wow, freakin impressive: you are covering enormous ground of messy real world issues here, with noticeable attention to detail and pretty high usability already! Did I already mention that's freakin impressive? Sep 22, 2010 at 7:21
  • @steffen - thank you. thank you very much. it is nice to know that the work is appreciated. Sep 22, 2010 at 14:35
2

Drill-Down


An evolving feature of the StackApps platform is drill down on any user. The utility and polish of the features are somewhat affected by the fact that I am cutting my teeth on the completely different, and refreshing, paradigm that MVC presents. So bear with me. And email me suggestions as to what you would like to see.

Current features

Standard link out to profile page

alt text


Visual online status indicators

When backed with live update you will be able to see up-to-the-second last_access_date. alt text


Aggregated User Info PopOut

alt text

This feature needs a better head for design than mine and input from you as to what kind of data points would be interesting.

It seems that efficient use of space, which I am obviously not adept, is the key to success here.

If you would like to get involved, open Photoshop, Gimp or notepad and mock up some ideas. Your input is actively solicited.

Email requests, suggestions, rough sketched, full blown professionally designed royalty free designs and money to [email protected] ;-)

The about page 'thank you' list needs some entries.

2

CSV output

alt text

Any grid on StackUsers provides an Export CSV to Clipboard button that will copy the data from the current grid view. This includes all user fields, not limited to those shown in the grid.

The output is standard CSV and can be opened or consumed by any tabular data editor.

Anonymous StackUsers can page 50 rows, while registered users are provided larger datasets.

If you have registered and would like a record count increase to satisfy a particular usage requirement, just email me @ [email protected].

Implementation

For those interested in converting a homogeneous array of JavaScript objects to CSV, here is the code that I use on StackUsers.

It is freely offered without restriction.

jsonToCSV

function jsonToCSV(sourceRows, omitHeader, fieldsToIgnore)
{

    /// <summary>
    /// Converts an array of JSON objects to CSV.
    ///</summary>
    /// <param name="sourceRows" type="Object[]">
    /// An array of objects containing a single level of scalar fields.
    /// e.g. [{ foo: "bar", fu: true }, { foo: "baarbar", fu: false }]
    /// 
    /// The first element in the array will define the header row.
    /// </param>
    /// <param name="omitHeader" type="Boolean" optional="true">if true, header row is not emitted</param>
    /// <param name="fieldsToIgnore" type="Map" optional="true">a map of fields to ignore e.g. { field1:1,field4:1 }</param>

    /// <returns type="String"></returns>
    /// <author name="sky sanders" contact="http://skysanders.net/subtext" date="2010-09-19"/>

    function quote(value)
    {
        return '"' + value.replace(/"/g, '""').replace(/\r/g, "\\r").replace(/\n/g, "\\b") + '"';
    };
    function pad(n)
    {
        return n < 10 ? '0' + n : n;
    };

    var header = "";
    var rows = "";
    var headerComplete = false;

    for (var i = 0; i < sourceRows.length; i++)
    {
        var firstElement = true;
        var row = "";
        for (var key in sourceRows[i])
        {
            if (sourceRows[i].hasOwnProperty(key))
            {
                if (fieldsToIgnore && (key in fieldsToIgnore))
                {
                    continue;
                }

                if (!headerComplete)
                {
                    if (!firstElement)
                    {
                        header = header.concat(", ");
                    };
                    header = header.concat(key);
                };

                if (!firstElement)
                {
                    row = row.concat(", ");
                };

                var value = sourceRows[i][key];

                if (typeof value != 'undefined' && value !== null)
                {
                    if (value instanceof Date)
                    {
                        var dateResult = value.getUTCFullYear() + '-'
                                + pad(value.getUTCMonth() + 1) + '-'
                                + pad(value.getUTCDate()) + 'T'
                                + pad(value.getUTCHours()) + ':'
                                + pad(value.getUTCMinutes()) + ':'
                                + pad(value.getUTCSeconds()) + 'Z';
                        row = row.concat(dateResult);
                    }
                    else if ((value instanceof Boolean) || !isNaN(value))
                    {
                        row = row.concat(value.valueOf());
                    }
                    else
                    {
                        row = row.concat(quote(value.valueOf()));
                    }
                }
                firstElement = false;
            }
        }
        rows = rows.concat(row).concat("\r\n");
        headerComplete = true;
    }
    return omitHeader ? rows : header.concat("\r\n").concat(rows);
};
1

Just wanted to ask / mention a few things and I thought an answer would be more appropriate:

  • In regards to an API: I understand your concern about bandwidth. Would you be willing to do what the SO team did and offer a private beta while you work out a key/throttle method?
  • In regards to hosting: how will that work? Are you basically just giving registered users XX MB for JavaScript hosting or what? Could StackCenter be moved there if I removed the PHP code?
  • Regarding apps: I saw that somewhere... is there some kind of app system you're setting up?

Answering questions like this in comments is silly. We can carry on a conversation here...

[sky]:

r.e. Bandwidth: the usage itself is certainly a concern, but the salient concern is DOS, whether it be a hard shutdown via abuse or an administrative shutdown triggered by hogging the resources of the shared host. I will spend a few hours, soon, coming up with a secure method of providing access to certain data.

Currently I provide access to users/email/{address} on a per-IP authorized basis. Please see this post for more details.

r.e. Hosting: I intend to provide an environment that will support direct access to the datastore via Ajax (same domain, I do not have the resources to expose JSONP to external consumers) as well as server hosted scripting languages (and perhaps even c# using a sandboxed mono compiler I built last year) for saving state for your javascript apps. The server code will be limited and not suitable for developing apps, only for specifying connector endpoints for the browser based javascript/flash/silverlight etc.

[George]
Flash won't be a problem because it supports cross-domain policy files.

As for the API, no hurry :)

4
  • @geo - i am not emplacing xdomain access files yet. api access is per ip address only, limiting the usefulness of a javascript client. Sep 14, 2010 at 22:13
  • @Sky: Speaking of which... did you get my request email? Just want to make sure you got it - no hurry. Sep 15, 2010 at 3:47
  • @geo - see stackapps.com/questions/1573/foo/1613#1613 - the /users/email route will be discontinued soon. Sep 20, 2010 at 10:56
  • @Sky: I'll update my page soon. Sep 20, 2010 at 19:10

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .