# General

Heat detector is a stackapp running as bot, that query [comments api](https://api.stackexchange.com/docs/comments) for all comments on [SO](https://stackoverflow.com/) to catch offensive, rude, snarky comments and comments that leads to abusive content (offensive, spam posts or previous offensive comments)

The comments are output in chat for manual review.

**Example**

[![output][1]][1]


To avoid storing comments in chat transcript (unearth buried fights), the comment is automatically deleted from chat before timeout.

The bot also have both a **test function** and a **report function**, that enables possibility to test different comments and to train the classifiers on uncaught content.

# Implementation

The comments api is queried every 1,5 min-3 min depending on traffic, the objective of query interval is to get all last comments within 1 page result (max 100 comments), to reduce api calls to a minimum.

All comments are processed both by regex and multiple machine learning algorithms (NLP), they all contributes to a final score. If score is above set threshold the comment is outputted to chat.

## Pre-processing
Before regex is applied the comment is pre-processed removing username, html tags and ripetitive characters. Before NLP  classification also code blocks, links and non normal ASCII chars are removed.

## Regex
The regex system is currently divided in two external text files, high scoring regex and low scoring regex. 

## NLP (Natural language processing)
Currently 3 different NLP systems are used, the different classifiers contributes to final score with different weights.

### [Naive Bayes][2]

This is the primary classifier (currently the best performing classifier) used, contributing most significantly to the final score. 

The classifier runs with these primary settings:

    minWordFrequency: 1 (comments are to short to use word frequency)
    useWordFrequency: false
    lowercaseTokens: true
    normalizeDocLenght: false
    stemmer: IteratedLovinsStemmer
    Stopwords: Standard english stopwords
    Tokenizer: NGramTokenizer (min 1, max 3)
 
 Testing on training set it has correctly classified instances at 99.15%

### [OpenNLP][3]

Apache Open NLP standard `DocumentCategorizerME`,  with default settings (cutoff 0, iteration: 4000)

### [J48 - Decision tree][4]

The feed is filtered to a `StringToWordVector` and classified with the J48 algoritim, settings are similar as for Navive Bayes.

## NLP Feed

The classifier has a feed divided in 2 categories, the good feed that is a download of old (>90 days) comments from comment api and a bad feed that is a composition of:

1. [Feed provided by SE](http://meta.stackoverflow.com/a/327148/5292302), these comments have been reviewed manually and almost 50% removed as they did not seem to be within scope

2. [Twitter feed](http://meta.stackoverflow.com/a/326617/5292302) provided by [Laurel](http://meta.stackoverflow.com/users/6083675/laurel)

Currently the feed contains 2000 good SO comments and 2000 (1000 SO, 1000 Twitter) bad comments .

## Accounts

The bot run under same user as [SOCVFinder](http://stackapps.com/questions/6910/socvfinder-providing-enjoyable-reviewing-live-duplicate-notifications-and-cher) the [Queen](http://stackoverflow.com/users/6294609/queen). The main reason is to leverage calls to comments api from same server, hence SOCVFinder already query comments api for possibile duplicates.

## Source code

Source code is available on [GitHub](https://github.com/jdd-software/SOCVFinder).

## Status

This stackapp is currently in **testing phase**, to understand correct regex and specially to improve feed, including new bad SO comments replacing the twitter feed comments. The objective is to run on 3000 good/bad only SO comments. 


  [1]: https://i.sstatic.net/yOsWX.png
  [2]: http://weka.sourceforge.net/doc.dev/weka/classifiers/bayes/NaiveBayes.html
  [3]: https://opennlp.apache.org/
  [4]: http://weka.sourceforge.net/doc.dev/weka/classifiers/trees/J48.html