Background:
This bot has been developed in an attempt to help capture possible vandalism. This includes:
- Removing all code
- Replacing all content with nonsense/repeated words
- Adding solutions to questions instead of self-answering
- Removing large amounts of text from their post
- Using certain keywords or offensive language within the edit summary
Why do we need the bot?
The point of the bot is to help identify bad edits and/or potential vandalism made to posts in real time so that the changes can be quickly rolled back.
Hippo
Hippo is a Higgs web dashboard for Belisarius. It is the place where all the posts the bot catches are sent. Most of its features are publicly available, however you need to sign in to send feedback.
See more about it on relevant page in the website.
Implementation:
The bot queries the Stack Exchange API once every minute to get a list of the latest posts. There is logic to check that the post has been edited and that it has been edited by the author.
The post_id from each post is then taken and the Stack Exchange API is again queried for the list of revisions. To limit calls we utilise the functionality of pushing multiple ids into the API and then logic is in place to ensure we are using the latest revision.
Edits can be made up of a title change, body change of a question, tag changes, or changes made to the body of an answer. Currently tags are not checked. Instead the title, question body, and answer body depending on what has been edited are run through filters, as is the edit summary.
Filtering:
In order to find if a post has been wrongly edited, titles, bodies and edit summaries are run through some filters.
Titles are run through the following filters:
BlacklistedWords; the title matches a blacklisted regex.
The question/answer body is run through the following filters:
TextRemoved; 80% or more of the body must have been removed with a Jaro Winkler score of less than 0.6BlacklistedWords; the post matches a blacklisted regex.CodeRemoved; the code has been removed with the latest edit (for questions only).FewUniqueCharacters; the body is either 30+ characters long and has less than 7 unique characters or 100+ characters long and has less than 15 unique characters.RepeatedWords; the body has been replaced with 5 or less unique words as of the latest editVeryLongWord; there a word bigger than 50 characters in the body.
Edit summaries are run through the following filters:
BlacklistedWords; the edit summary matches a blacklisted regex.OffensiveWord; the edit summary matches an offensive regex. Note: In order to reduce false positives inVeryLongWord,TextRemovedandBlacklistedWordsreasons, some HTML tags are stripped (a,code,img,pre,blockquote).
Where's the list of blacklisted and offensive words?
The bot fetches the blacklisted and offensive regexes from the database. You can find the blacklisted words CSV here and the one with the offensive words here.
Accounts:
The project is running under the user Belisarius in the SOBotics room. A more detailed presentation is at http://belisarius.sobotics.org/ including a list of commands.
Feedback:
What are the different feedback types and what do they mean?
There are two feedback:
- tp: This means that the reported edit is harmful and should be rolled back.
- fp: This means that the reported edit is not harmful and no further action is required.
How can I send feedback?
There are two ways to do this:
- Reply to the report with either
tporfp. There's a userscript which makes this process a lot easier. - Go to the respective Higgs report (click the "Hippo" link) and select the type of feedback you wish to send.
Notes:
- The feedback must be a reply to a report.
- In some cases, the report spans over 2 messages. In those cases, reply to the message that contains the link to both all the revisions and to the potentially bad one. If they are in different messages, then you'll need to send feedback through Higgs.
A sample image of a report is:
The source code is available on GitHub and suggestions are welcome via issues or pull requests.
