Summary
This is a suggestion to create one, possibly two, indicators relating to Reopen Review Queue status. The purpose is to alert users to the requirement for intervention outside the normal process to get a question reopened.
Background
The design of the Reopen Review queue and the nature of Super User create a situation where it is easy for closed questions to fail reopening in the review queue, even though they have been improved into good questions. When a question fails to be reopened in the review queue, there is no system mechanism for it to collect reopen votes, regardless of further action on the question. External intervention is required to reopen the question.
However, the fact that the question was in the review queue at all, or that it failed, is invisible to users on the question, itself. Question activity can be easily misinterpreted to assume that there is an active reopening process underway when there is not. Reopen activity just appears to stall for no apparent reason. If someone following the question investigates the history, they can discover that it failed the reopen review. But again, it is easy to misinterpret the reported activity as the will of the community, when it is not. Users unaware of the review or its failure, may edit the question or vote to reopen thinking those activities will get the question into review when they will not.
The result is that questions worthy of reopening are left sitting idle, with nobody aware that reopen votes are needed, and existing reopen votes aging away. The questions don't get reopened, and people assume that was the community's decision. In fact, intervention was required and nobody was aware of it.
Details of the underlying process, and how and why this happens, are described below as supplemental reading.
The objective here is to create status indicators to make users aware of the situation and the need for intervention.
API Source
This is out of my area, but I'm going on the assumption that at least the detail for the first indicator below is available in the API because I assume it is used to create the question timeline. Here's the timeline for a recent example (Continuous Number Generator), that got reopened by recruiting votes in a Meta site post after I discovered that the action was necessary:
Around a third of the way down from the top, it shows Reopen Review completed (2 day marker). For the second indicator below, the voting, editing, etc. is summarized along with date information, so I'm assuming the detail behind the summary might be available.
What the app or feature would do
There are two levels of functionality that could be useful.
The "simple" functionality would indicate on the question that reopen review was completed (but only if the question is still closed; it would be just noise if the question has been reopened). That's the critical information needed to stop expecting that more reopen votes will be coming organically, and it is available long before someone might think to investigate.
This has a second value. If the review ended and failed to reopen the question even before the edit was made, the edit will not trigger review, contrary to the editor's expectation. Even manually voting to reopen in addition to the edit won't trigger review. The question is already in the Phantom Zone. So the editor would immediately need to take external steps to get the question reopened, and the review completion indicator would be the alert.
Implementation: The logical place for an indicator of review completion would be at the bottom of the "Closed" message box. That would also solve the issue of displaying the message on a reopened question (if it is already open, there's no Close message in which to put the indicator; if it is currently closed and later reopens, the Close message disappears, taking this indicator with it).
Since many (most?) users aren't familiar with the implications, it could perhaps be a message like:
NOTE: Reopen review has been completed. This question will not appear in the Reopen Review queue again. If you believe it should be reopened, please use other means to have it reconsidered.
Phantom Zone pattern alert: The above functionality serves multiple purposes, so the second part would be supplemental rather than an alternative. The above alert also doesn't really address the appearance of reopening failure being the will of the community.
The second part would depend on the granularity of the detail available through the API. If the detail was available, it could look for the characteristic pattern associated with this problem. The pattern is the following sequence:
- The question entered the reopen review queue.
- It received only votes to leave closed prior to step 3 (as a practical matter, this would be either 2 or 3 votes; 3 would end the review).
- The question was edited (regardless of whether the queue was still open).
- All, or all but one, subsequent votes were to reopen, regardless of where those votes occurred (review queue or on the question), and regardless of whether the question was still in the reopen review queue at the time.
Identifying this pattern would provide a direct alert that action is required, and provide a paradigm shift for people following the question who believe there is still an active reopen process.
Implementation: The logical place for a Phantom Zone pattern alert might be under the line of buttons that contains the Reopen button; that's where people will be following the reopen vote count. It might be a message like:
Timing prevented reopening in the review queue,
no additional reopen votes will come from there.
See note in the Closed message.
Optional Supplemental Reading
Site Applicability: This could potentially be useful on a number of sites, but I suspect it is most applicable to Super User because of the nature and size of the site.
A substantial portion of the technical questions SU receives are from non-technical users. The questions are answerable only with accurate, complete detail that non-technical users don't know to provide, or how to provide. For many posters, English is a second language, and technical descriptions are ambiguous. Getting a question into good, answerable form often requires repeated attempts at clarification and editing.
What happens and why: (Simplified explanation of a complicated process.) Questions that are unanswerable as-is get closed. It takes multiple rounds of clarification and editing to turn it into a good question that can be reopened.
The first edit triggers the reopen review queue, often before the question is in a condition to reopen. Then one of two things happens, whichever comes first. It collects five reopen votes, the review is ended and the question is reopened, or it collects three leave-closed votes, and the review is ended with the question still closed (but it retains any reopen votes received).
Often, the edit that triggered the review queue doesn't turn it into a good question, a later edit does. The early reviewers vote to leave the still-inadequate question closed. If the question is still in the review queue when it becomes good, reviewers start voting to reopen.
Editing continues independent of any reviewing, and at some point, it becomes a good question. Independent of the review queue, the editor may vote to reopen. A few people who are actively following the question may see the improved question and vote to reopen on the question, itself. So soon after improvement, it collects some number of reopen votes from various sources.
Behind the scenes, the question may already have collected three leave-closed votes, so it's no longer in the queue. Or it may have collected two before the final edit, started to collect reopen votes in the queue, and one reviewer, having a bad hair day, casts the third leave-closed vote that ends the review.
On the question, there's no indication of the review status, or even that it is in the queue. People make assumptions based on visible activity. What anyone with an interest in the question sees is easily misinterpreted: the question was turned into a good one, and then started receiving reopen votes. They assume the last edit bumped it into the queue, and those reopen votes are activity in the queue.
It looks like appropriate momentum, and reopening is expected. However, for no apparent reason, reopening just stalls; those initial reopen votes are the only ones that show up. People assume there's an active process underway to finish reopening the question, when there isn't.
Questions get only one shot at the reopen review queue. So when it fails in the queue, it becomes invisible. Nobody is aware that that they need to vote to reopen it. By this time, the question is no longer fresh, so there is only a trickle of people with voting privileges, who have not already voted, who might stumble across the question.
To make matters worse, if someone gets curious and checks the question timeline, it looks like it went through reopen review and it was the will of the community to leave it closed. The timeline information doesn't reliably indicate when the edit happened in relation to the individual votes. This perception may discourage even pursuing getting it reopened.
Expected Audience
If this was a feature of the SE UI, a lot of users might benefit from it on their own questions on rare occasions (users tend not to stick around if they post a lot of bad questions). As an app or script, the typical user probably won't have much need or incentive for this. I anticipate it would be of interest mainly to two groups of users who would run into this situation with any frequency:
- Users who work the review queues and regularly try to rehabilitate bad questions so they can be answered.
- Prolific answerers who want questions open so they can answer them, and want their answers on open questions.
