Background and History
The New Answers to Old Questions tool helps us to find all the answers added to questions which are more than 30 days old. However the issue with the tool is that it is not real time. There are requests on Stack Overflow Meta to not only make it real time but also to enhance the tool to add more features to it. However these requests have yet not been implemented.
Why do we need the bot?
This project aims to not only overcome the "not real time" issue but also help those users below 10,000 reputation to take a look and moderate the answers.
Implementation
The Bot queries the Stack Exchange API once in every 30 seconds to get the list of the latest answers. The question_id from each of the answer is taken and the Stack Exchange API is again queried for the list of posts with those IDs. Every answer that is posted after 30 days of the question is considered as a new answer to a old question and is returned by the Bot.
False Positives and Filtering
As the Bot returns all the posts, there are a lot of false positives and hence many filters are applied
High Filtering Rate — Return a lot of True Positives
- Reputation Filter: The reputation filter is set at 50, which is the reputation that is needed to comment. Hence all the answers that are posted due to the lack of reputation are caught here.
- The last punctuation: Whenever an answer ends with a question mark the likelihood of the answer being open-ended is very high. Hence in most of the cases, answers that are caught by this filter need to be
- Edited to make it an answer that does not request clarification from the user
- Flagged as NAA
- Salutations: Those answers that end with "Thanks", etc, are most likely to be non-answers. Hence those answers that are caught by this filter are potential NAAs.
- Single Line Only: Great answers are seldom single line, Hence a single line filter usually catches those posts which either need polishing or need to be flagged.
- Starting Keywords: Interrogative pronouns like What, Why, Who, etc. The only false positives with this were the rhetorical questions posted as answers, which are very few in number.
Low Filtering Rate — Return lesser number of True Positives
Code Blocks: Non Code Blocks usually imply that either the post is
- Badly formatted: Here we can edit the post to improve the formatting.
- Not an Answer or a Very Low Quality answer: Here we can flag them appropriately.
Hence many answers that need either editing or flagging are caught by this filter.
- Body Contains a Question Mark: This filter returns those answers that contain a question mark that's not present either with a link or a code block. This again returns those posts that are potential NAAs.
- Possible Link Only Answer: The total length of the body is considered and the length of the link is considered. When the link is 60% longer than the total length, the bot considers the answer as link only.
Coupling Filters
Coupling two or more filters together give a high rate of true positives.
The "Single Line, Ends with ?" filter has an accuracy of nearly 100% while reporting NAAs.
Automatic flags and comments
If the score is 7.0 or higher, Natty will automatically raise a flag. Under certain conditions, an explanatory comment will be posted. See: Can a machine be taught to flag Non-Answers and post comments on them automatically?
Future Enhancements
The future works about this include
- utilizing NLP to discern if a post is a question and to report those
Why are we focusing on old questions only?
New Answers on Old Questions receive lesser attention than the new ones. The period of 30 days was chosen as it was the same as the New Answers to Old Questions page in the 10k tools page.
Accounts
The Project is running under the user Natty in the SOBotics room. A sample image of a report is
The source code is available on Github and suggestions are welcome. The list of commands are present on the website
