StatsExchange - low friction, API-based statistics generator

Question

##Screenshot / Code Snippet

##About

StatsExchange is an application that's designed to leverage the filtering and data reporting features of the Stack Exchange API. With StatsExchange, you can quickly and easily generate statistics on just about any question or answer metrics that you can filter on using the API.

Sometimes you've got a question about the Stack Exchange sites you frequent. For example:

• What percent of the time is my answer accepted?
• How many views does a particular tag generate for the site?
• During a period when the site was running a promotion, how much activity did we see around a promoted tag?
• Am I upvoted or downvoted more on average than other people?

I found myself asking questions along these lines relatively frequently, and with my knowledge of the API, I could usually whip up a quick custom "throwaway" tool to answer them with real, live data. This is the result of an experiment to generalize those throwaway tools into something that might have wider appeal.

You can think of it as a similar tool to Stack Exchange Data Explorer but designed with a less-technical audience in mind, and focused more on "instant gratification." If you feel comfortable with SQL along with the Stack Exchange database schema, and you don't mind the fact that the data's a bit stale, the Data Explorer site has a wealth of information and can help you answer many different questions.

Sometimes, though, there are questions that the API could answer better, faster, and with fresher data. The purpose of this application is to make interfacing with this data easier.

###Usage

To load the application, click this link.

Start by clicking the "Add New" button to specify the name of the query you wish to create, and set the filtering options.

• If you choose to search by Questions, the query will find all questions that match your criteria, and then generate statistics based on those questions and their answers.
• If you choose to search by Answers, the query will first locate answers that match your criteria, and then generate statistics based on just those answers and the questions they were posted on.

Regardless of the search type, you can set several filter options, including the date the post was created, its owner, what tags were on the question, as well as minimum and maximum scores for the question or answer.

For instance, if you wanted to know what percent of the time your answer was accepted on the tag "string" - you'd set up a new query for "Answers" on tag "string" and user "Me."

If you wanted to know how often user 12345 got their "skyrim" questions answered, you'd set up a new query for "Questions" on tag "skyrim" and user id "12345."

If you wanted to find out how many questions a site has that have a score less than zero, you could set up a "Questions" query with "Max Score" equal to -1, and "Users" set to "Everyone."

You can specify multiple tags - they will be logically AND'ed by the API. Therefore, ALL tags in a query must appear on a question for it to show up in the final count.

Once you've set up the query, you can run it by pressing the "refresh" button, or "edit" or "remove" it if you need to make changes or are finished with the query. When a query is running, you can "Abort" it if it's taking too long or you need to make changes.

You can create as many queries as you like, and use them to compare and contrast different aspects of data. For instance, you might be interested in comparing the statistics of a set of users, or of a set of tags.

The number in parentheses indicates the "per question" average.

If you'd like to share a query or set of queries with others, you can use the "Import/Export" dialog to print the queries in JSON, which you can then copy and paste to chat or email, if you so desire. The queries are saved in your browser's local data storage until you clear them.

###Caveats, Known Issues, and Limitations

The filtering on tags only works at the API level for "Questions" by "Everyone" - so if you're expecting to do a "Answers" by "Everyone" query and filter it by tags, you'll have to load every answer and question on the site in order to do this. This is likely to take far too long to be useful in any normal situation. Limiting the query's scope by other means would be a good idea.

###TODO

I'd like to figure out a way to expose more of the data collection and reporting aspects for people to add their own statistics and tracking information. This would make it possible to add your own columns displaying custom data to the report.

There are a number of things I could do with more API options, but it's likely that I'll have to make feature requests for "API v2.1" or "v3.0" in order to make this possible.

###License

BSD 2-clause. Source code is available from this GitHub repository.

##Contact

If you run into bugs, come find me using the chat system. This is one of the more complex applications I've built, so it's likely to have its fair share of issues.

Answer 1

I just released version 2.0, which now allows you to export the data to a CSV format from your browser.

Now you can set up your "query," run it to get the "generic" stats I generate, and then export the data to the data analysis/charting/statistics package of your choice to run arbitrary other data manipulation steps on it.

For instance, you can copy and paste into a text file, and upload this to say, Google Docs, and then filter/sort/chart the data through there.

The columns available in the data exported are:

question_id, question_creation_date, tags, question_score, question_up_vote_count, question_down_vote_count, view_count, question_user_id, question_user_rep,answer_id,answer_creation_date, answer_score, answer_up_vote_count, answer_down_vote_count, is_accepted, answer_user_id, answer_user_rep

There is one row per question + answer pair (so question_id's may be duplicated if there is more than one answer). If a question is unanswered, it will have one row in the exported set, with all 0's for the answer data. (note that "answer_id" == 0 is invalid, so you can use this to filter for unanswered questions or invalid answer data).

Dates are in Unix timestamp format (which is the SE API standard). Multiple tags use | as a separator.