Background and History

The New Answers to Old Questions tool helps us to find all the answers added to questions which are more than 30 days old. However the issue with the tool is that it is not real time. There are requests on Stack Overflow Meta to not only make it real time but also to enhance the tool to add more features to it. However these requests have yet not been implemented.

Why do we need the bot?

This project aims to not only overcome the "not real time" issue but also help those users below 10,000 reputation to take a look and moderate the answers.

Implementation

The Bot queries the Stack Exchange API once in every 30 seconds to get the list of the latest answers. The question_id from each of the answer is taken and the Stack Exchange API is again queried for the list of posts with those IDs. Every answer that is posted after 30 days of the question is considered as a new answer to a old question and is returned by the Bot.

False Positives and Filtering

As the Bot returns all the posts, there are a lot of false positives and hence many filters are applied

High Filtering Rate — Return a lot of True Positives

  • Reputation Filter: The reputation filter is set at 50, which is the reputation that is needed to comment. Hence all the answers that are posted due to the lack of reputation are caught here.
  • The last punctuation: Whenever an answer ends with a question mark the likelihood of the answer being open-ended is very high. Hence in most of the cases, questions that are caught by this filter need to be
    1. Edited to make it an answer that does not request clarification from the user
    2. Flagged as NAA
  • Salutations: Those answers that end with "Thanks", etc, are most likely to be non-answers. Hence those answers that are caught by this filter are potential NAAs.
  • Single Line Only: Great answers are seldom single line, Hence a single line filter usually catches those posts which either need polishing or need to be flagged.
  • Starting Keywords: Interrogative pronouns like What, Why, Who, etc. The only false positives with this were the rhetorical questions posted as answers, which are very few in number.

Low Filtering Rate — Return lesser number of True Positives

  • Code Blocks: Non Code Blocks usually imply that either the post is

    1. Badly formatted: Here we can edit the post to improve the formatting.
    2. Not an Answer or a Very Low Quality answer: Here we can flag them appropriately.

    Hence many answers that need either editing or flagging are caught by this filter.

  • Body Contains a Question Mark: This filter returns those answers that contain a question mark that's not present either with a link or a code block. This again returns those posts that are potential NAAs.
  • Possible Link Only Answer: The total length of the body is considered and the length of the link is considered. When the link is 60% longer than the total length, the bot considers the answer as link only.

Coupling Filters

Coupling two or more filters together give a high rate of true positives.
The "Single Line, Ends with ?" filter has an accuracy of nearly 100% while reporting NAAs.

Automatic flags and comments

If the score is 7.0 or higher, Natty will automatically raise a flag. Under certain conditions, an explanatory comment will be posted. See: Can a machine be taught to flag Non-Answers and post comments on them automatically?

Future Enhancements

The future works about this include

  • utilizing NLP to discern if a post is a question and to report those

Why are we focusing on old questions only?

New Answers on Old Questions receive lesser attention than the new ones. The period of 30 days was chosen as it was the same as the New Answers to Old Questions page in the 10k tools page.

Accounts

The Project is running under the user Natty in the SOBotics room. A sample image of a report is

Sample Image

The source code is available on Github and suggestions are welcome. The list of commands are present on the website

  • Interesting, I did get a notification when Brock edited the post. Perhaps it works only on SO. – Bhargav Rao Sep 20 '16 at 16:44
  • You did or did not get a notification? Also, Stack Apps seems to use a much older code base. – Brock Adams Sep 20 '16 at 20:06
  • I did not get a notification in my inbox. But it is present in the responses tab. – Bhargav Rao Sep 21 '16 at 3:01
  • 1
    I guess NATO is more pronounceable, but it's New Answers to Old Questions, not New Answers To Old questions. – Cascabel Oct 17 '16 at 23:28
  • 4
    Haha, @Jefromi, I first called it as NATOQ, which was to be pronounced as "NATO Queue", but everyone were confused and thought it was "Natock". Hence, I had to drop the last letter. Apart from that NATO (the guys with the big guns) helps us kill the bad guys and NATO (10k tools) helps us find the bad answers and kill them. So, it's a double sided win-win situation. As a side note, while coming up with a catchy acronym, we need to think Fast. :D – Bhargav Rao Oct 18 '16 at 8:01
  • 1
    "When the link is 60% longer than the total length, it's considered link only" -- No, that's not the definition of link-only, at least when it comes to flagging. You should clarify that in your description. – Nic Hartley Jan 22 '17 at 5:18
  • Ahrm, @QPaysTaxes, That's the definition of the way the bot flags a post and not how the flag itself. I'll add more details tonight once I'm back. Thanks for the tip. – Bhargav Rao Jan 22 '17 at 8:01
  • 1
    That's what I'm saying you should clarify :) – Nic Hartley Jan 22 '17 at 8:18

You must log in to answer this question.

Browse other questions tagged .