FireAlarm: detecting low-quality questions with machine learning

Question

^{There are two versions of FireAlarm. One version is written in C, and the other is written in Swift. This post is about the Swift version.}

---

FireAlarm is a chatbot Ashish Ahuja and I have been working on. It attempts to detect low-quality questions using a naive Bayes filter.

All reports can be seen on FireAlarm's web dashboard, Bonfire (StackApps post).

How accurate is it?

Each question is assigned a score by the machine learning filter, and a lower score means the question is more likely to be bad. Different chat rooms have different thresholds; in SOCVR the threshold is 35, in SOBotics it's 45, and in FireAlarm Development it's whatever we want it to be at the moment.

At a threshold of 35, since the bot started running in SOCVR on February 8:

• 12% of reported posts have close votes but aren't closed or deleted,
• 13.98% are closed but not deleted, and
• 40.7% are deleted,

which adds up to 66.7%. That's pretty impressive, considering only about 9% of questions from that time period are closed. (I wasn't able to get statistics on deleted posts or posts with close votes.)

• 5.8% of non-deleted reported posts have a positive score
• 31.5% of non-deleted reported posts have a negative score
• 78.4% of all reported posts have a negative score, have close votes, are closed, or are deleted.

Where does it run?

FireAlarm is currently running in SOBotics, SO Close Vote Reviewers and of course the FireAlarm Development room. Enter a room and run @FireAlarm help for a command list.

Where's the code?

The code is on GitHub. FireAlarm uses two libraries I've helped write: SwiftChatSE, a Swift chatbot framework, and SwiftStack, an API wrapper FelixSFD and I have been working on. You may reuse any of the code in your own projects; it's all licensed under MIT.