# General

Heat detector is a stackapp running as bot, that query [comments api](https://api.stackexchange.com/docs/comments) for all comments on [Stackoverflow](https://stackoverflow.com/) to catch offensive, rude, snarky comments and comments that leads to abusive content (offensive, spam posts or previous offensive comments)

The comments are output in chat for manual review.

**Example**

[![output][1]][1]


To avoid storing comments in chat transcript (unearth buried fights), the comment is automatically deleted from chat before timeout.

The bot also have both a **test function** and a **report function**, that enables possibility to test different comments and to train the classifiers on uncaught content.

# Implementation

The comments api is queried every 1,5 min-3 min depending on traffic, the objective of query interval is to get all last comments within 1 page result (max 100 comments), to reduce api calls to a minimum.

All comments are processed both by regex and machine learning algorithms (NLP), they all contributes to a final score. If score is above set threshold the comment is outputted to chat.

## Pre-processing
Before regex is applied the comment is pre-processed removing username, html tags and ripetitive characters. Before NLP classification also code blocks, links and non normal ASCII chars are removed.

## Regular expressions
The regex system is divided in three external text files; high, medium and low scoring regex. 

## Natural language processing (NLP)
Currently 2 different NLP systems are used. On the basis of the predicated value the score is increased.

### NLP Feed

The classifier has a feed divided in 2 categories, the good feed that is a download of old (>90 days) comments from comment api and a bad feed that is a composition of:

1. [Feed provided by SE](http://meta.stackoverflow.com/a/327148/5292302), these comments have been reviewed manually and almost 50% removed as they did not seem to be within scope

2. Comments captured incorrectly or correctly during testing

3. [Twitter feed](http://meta.stackoverflow.com/a/326617/5292302) provided by [Laurel](http://meta.stackoverflow.com/users/6083675/laurel)

Currently the feed contains 2000 good SO comments and 2000 (1000 SO, 1000 Twitter) bad comments .


### [NaiveBayesMultinomialText][2]

The classifier runs with these primary settings:

> **useWordFrequency**: false (comments are to short to use word frequency)<br/> 
> **lowercaseTokens**: true<br/>
> **NormalizeDocLenght**: false<br/>
> **Stemmer**: IteratedLovinsStemmer<br/>
> **Stopwords**: [Standard english stopwords][3]<br/>
> **Tokenizer:** NGramTokenizer (min 1, max 3)

The settings have been chosen on basis of cross validation and classification result on training set. Testing on training set it has correctly classified instances at 99.075%

[![Classifier result][4]][4]

### [Apache OpenNLP][5]

Apache Open NLP standard `DocumentCategorizerME`, is used with these non default settings:

> **FeatureGenerator**: BagOfWordsFeatureGenerator<br/>
> **WhitespaceTokenizer**: WhitespaceTokenizer.INSTANCE<br/>
> **CUTOFF_PARAM**: 0<br/>
> **ITERATIONS_PARAM**: 4000<br/>

Testing on training set it has correctly classified instances at 99.823%, with 7 comments out of 4000 not classified correctly

[![OpenNLP result][6]][6]

### Other classifiers

Other classifiers as [J48][7], [SMO][8] and [SGDText][9] have been test but without good results.


## Accounts

The bot run under same user as [SOCVFinder](http://stackapps.com/questions/6910/socvfinder-providing-enjoyable-reviewing-live-duplicate-notifications-and-cher) the [Queen](http://stackoverflow.com/users/6294609/queen). The main reason is to leverage calls to comments api from same server, hence SOCVFinder already query comments api for possibile duplicates.

## Source code

Source code is available on [GitHub](https://github.com/jdd-software/SOCVFinder).

## Status

This stackapp is currently in **testing phase**, to understand correct regex and specially to improve feed, including new bad SO comments replacing the twitter feed comments. The objective is to run on 3000 good/bad only SO comments. 


  [1]: https://i.sstatic.net/yOsWX.png
  [2]: http://weka.sourceforge.net/doc.dev/weka/classifiers/bayes/NaiveBayesMultinomialText.html
  [3]: https://github.com/SOCVFinder/SOCVFinder/blob/master/SOCVDBService/ini/stop_word.txt
  [4]: https://i.sstatic.net/IKIg9.png
  [5]: https://opennlp.apache.org/
  [6]: https://i.sstatic.net/Dkh39.png
  [7]: http://weka.sourceforge.net/doc.dev/weka/classifiers/trees/J48.html
  [8]: http://weka.sourceforge.net/doc.dev/weka/classifiers/functions/SMO.html
  [9]: http://javadox.com/nz.ac.waikato.cms.weka/weka-dev/3.7.10/weka/classifiers/functions/SGDText.html