GrafOverflow

A metric collection script for the StackExchange API.

View Some Realtime Stats

Here is a link for you to view some stats in real* time, with 1 year of history!

GrafOverflow

Screenshot of Dasboard

*Stats are collected every 5 minutes, and smoothed to a 30 minute average ( on query ) to account for API caching.

About

Data is beautiful, so why not draw some beautiful graphs about the sites that helps so many.

This script is based around the carbon-cache metric storage backend, and is using Grafana for the UI.

To query the API, I wrote up a node.js script that will run a collection on a 5 minute interval; this then parses the JSON response from the SE API into a format parsable by the metric engine, opens a socket, and sends it.

In the above screenshot, you can see the result of simply collecting and graphing the data returned by the /info API endpoint.

The list of statistics collected and graphed:

  • Total Votes
  • Total Questions
  • Total Answers
  • Total Comments
  • Total Accepted
  • Accepted Percentage
  • Total Users
  • Total Badges

All per minute stats shown in Grafana are calculated by applying time function transformations to the total counts.

Multi Site Support

The configuration accepts a list of sites to poll. The hosted instance has the following sites available:

  • stackoverflow.com
  • serverfault.com
  • superuser.com
  • gaming.stackexchange.com
  • askubuntu.com
  • math.stackexchange.com
  • apple.stackexchange.com
  • 3dprinting.stackexchange.com

Download

The script source can be found on GitHub.

Following in the path of StackExchange, this is published under the MIT License.

Requirements

The application is built in node.js and outputs data in the carbon plaintext protocol.

This applications requires an instance of carbon-cache or equivalent metric storage engine to properly store stats; in addition, it requires a front end viewer - in this example, I am using graphite-web and Grafana.

Configuration & Usage

To configure the application, simply copy config.sample.js to config.js and adjust your desired parameters.

To start the application, simply run the main file with node.

cd $APP_HOME
node stackstats.js &>> log &

Contact

Created by:
  Matthew Clark
  matt@mclarkdev.com

Update 9/13/18 Sorry, the domain name expired for some time there, but I never stopped collecting metrics! I have updated the active link, metrics can again be viewed with 1 year of history.

  • Any chance you could allow us to change the amount of time something is counted on a certain gauge, for instance I'd like to view some of the slower sites with 'x per hour' instead of 'x per minute' gauges. – 3D1T0R Sep 17 at 7:13
  • Is this what you want? I am not too happy with it, but I don't have a ton of flexibility in allowing dashboard modifications :/ In the top left, there is a new field, Scale in seconds. Change this from 60 to 3600 to view rates as, X per Hour, change it to 86,400 to view rates a X per day. – Matt Clark Sep 17 at 7:35
  • Yeah, I see why you're not too happy with that, but it does what I requested. Thanks. It really is more useful to be able to see Ask Ubuntu's stats as 5.33 Q/hr & 3.33 A/hr than .09 Q/min & .06 A/min. – 3D1T0R Sep 17 at 16:27

You must log in to answer this question.

Browse other questions tagged .