9

Inspired by the Fastest Gun question on the central Meta, this script implements answer sorting based on a modified Wilson score. Installing the script adds a ‘Modified Wilson score’ sort option on question pages that attempts to sort the highest-quality answers first. One should hope this ordering method to boost hidden gems that would otherwise be buried under hastily-written popular-but-wrong answers that would otherwise hold the top spot under a naïve score tally sort.

Though initially directly inspired by the FGITW problem, ultimately the script aims to mitigate the related, but broader and more nebulous problem of popular-but-wrong (PBW) answers. As such, a number of weighing tweaks have been applied to take the specific voting structure of Stack Exchange sites into account:

  • Downvotes are counted as five times heavier than upvotes, as they are assumed a better indicator of answer quality for a number of reasons:

    • The reputation threshold to cast upvotes is very low (15), much lower than for downvotes (125 on most sites). As such, upvotes should be assumed used more often by users who are less informed than the downvoters (with respect to the topic at hand, or the Stack Exchange customs).
    • Downvotes cost reputation to cast, therefore they are used more sparingly and with more care, and presumably, usually for reasons of merit.0 Upvotes are assumed more likely to be cast for more superficial reasons (appreciation of effort, formatting, writing style, humour).
    • Some users are reluctant to cast downvotes; a few have even explicitly stated that they never cast downvotes, no matter how bad an answer is (exhibit A; exhibit B; also, discussion). The weighing factor attempts to correct for this tendency, and estimate how everyone would have voted according to their convictions about the answer quality, regardless of any other considerations.

    To mitigate the effects of tactical and revenge downvoting, the first two downvotes are only weighed 1× and 2× as much as one upvote, respectively.

    The weighing is disabled completely on Stack Apps and Meta sites, including the central Meta; on those, the script reverts to using the plain, unweighed Wilson confidence interval. The above reasons don’t apply as much to Meta sites: on regular meta sites, neither upvotes nor downvotes cost reputation to cast, it is assumed most Meta participants can downvote, and furthermore it is assumed that Meta answers represent the users’ opinions, and so upvotes indicate agreement and cannot be meaningfully considered ‘incorrect’ (there is no such things as a popular-but-wrong meta post); the best the script can do is estimate how well a given answer would have been received, had everyone read it.

  • Answers with more (unweighed) votes are judged more harshly: that is, the more votes an answer has relative to other answers, the closer the final score is to the lower end of its Wilson confidence interval. Answers with few votes have scores closer to the middle of the confidence interval; the answer with the most votes has the final score at the bottom of the confidence interval, unless all answers have fewer than 8 votes total (to avoid disturbing monotonicity where all answers have few votes overall).

  • Bounties are counted with upvotes; to avoid high bounties biasing the scores too much, bounty awards are scaled and square-rooted. A bounty of 25 is worth 1 extra upvote; a bounty of 500 is worth 10 upvotes. each 25 points of bounty earned beyond the first is counted as one upvote. Bounties do not count towards harshness adjustment.

  • Answer acceptance is not taken into account. In my experience, it tends to be a poor proxy for quality relative to other factors. Since askers usually upvote simultaneously with accepting the question anyway, I just did not bother accounting for acceptance at all.

If the question has answers that weren’t displayed on the current page, the script will generate stubs linking to the full answers. It may happen that those answers score higher than those visible on the current page and as such, the stubs may appear between full answers; this is intentional and not a bug. Deleted answers, when visible, are always sorted last.

The weighing factors were not determined by any principle other than ‘they felt right’, so beware. In particular, the 5× multiplier on downvotes was chosen mainly as a counterbalance to the five-times-smaller penalty for receiving an answer downvote than the reward for receiving an answer upvote. Statistical analyses of vote distribution, number of eligible voters and the number of views on each site might allow us to come up with fairer weighing factors.


0 Of course, downvoting for spite is not unheard of, though I assume most of it occurs as serial downvoting, which is usually swiftly corrected.

Script code

// ==UserScript==
// @name     StackExchange: Sort Best First
// @match    https://*.stackexchange.com/*
// @match    https://*.superuser.com/*
// @match    https://*.stackoverflow.com/*
// @match    https://*.mathoverflow.net/*
// @match    https://*.serverfault.com/*
// @match    https://*.askubuntu.com/*
// @match    https://stackapps.com/*
// @exclude  https://chat.stackexchange.com/*
// @exclude  https://api.stackexchange.com/*
// @exclude  https://data.stackexchange.com/*
// @exclude  https://openid.stackexchange.com/*
// @exclude  https://stackexchange.com/*
// @exclude  https://contests.stackoverflow.com/*
// @exclude  /^https?:\/\/winterbash\d{4,}\.stackexchange\.com\//
// @grant    GM.getValue
// @grant    GM.setValue
// @grant    GM.deleteValue
// ==/UserScript==

"use strict";

const OPT_UNCOVER_VOTES = false;

const GM_KEY_ACTIVE = `active-on:${location.origin}`;

const sleep = t =>
    new Promise((accept, reject) => setTimeout(accept, t));

const isMeta = /(^|\.)(meta\.|stackapps\.com$)/.test(location.hostname);

const GM_KEY_BACKOFF = 'backoff-until';

const getAnswerScores = async qid => {
  let backoffUntil = await GM.getValue(GM_KEY_BACKOFF, -Infinity);
  
  const usp = new URLSearchParams({
    site: location.hostname,
    filter: 'RonSjxB7EEr63nptQ05Wk',
  });

  const fullResult = [];

  let page = 1;
  for (let page = 1;; page++) {
    usp.set('page', page);
    backoffUntil = await GM.getValue(GM_KEY_BACKOFF, -Infinity);
    while (Date.now() < backoffUntil) {
      console.warn(`[SB1st] backing off until ${new Date(backoffUntil)} as told`);
      await sleep(backoffUntil - Date.now());
      backoffUntil = await GM.getValue(GM_KEY_BACKOFF, -Infinity);
    }
    GM.deleteValue(GM_KEY_BACKOFF, backoffUntil);
    const response = await fetch(`https://api.stackexchange.com/2.3/questions/${+qid}/answers?${usp}`);
    const result = await response.json();
    
    if (result.backoff) {
      console.warn(`[SB1st] Told to back off for ${result.backoff} s`);
      backoffUntil = Date.now() + result.backoff * 1000 + 500;
      GM.setValue(GM_KEY_BACKOFF, backoffUntil);
    }

    if (result.error_id)
      throw new Error(`${result.error_message} [${result.error_name}, ${result.error_id}]`);

    console.info(`[SB1st] Remaining API quota ${result.quota_remaining} of ${result.quota_max}`);

    fullResult.push(...result.items);

    if (!result.has_more)
      break;
  }

  return fullResult;
};

const scoreWilson = (answer) => {
  const up = answer.up_vote_count + Math.max((answer.awarded_bounty_amount ?? 0) / 25 - 1, 0);
  const dn = isMeta
        ? answer.down_vote_count
    : Math.max(0, answer.down_vote_count - 2) * 5
            + (answer.down_vote_count > 1) * 2
            + (answer.down_vote_count > 0);
  const total = up + dn;
  const harshness = answer.up_vote_count + answer.down_vote_count;
  const z = 1.96;

  let upper = 1, lower = 0;

  if (total !== 0) {
    const upRatio = up / total;
    const B = upRatio + z * z / (2 * total);
    const C = Math.sqrt((upRatio * (1 - upRatio) + z * z / (4 * total)) / total);
    const D = (1 + z * z / total);

    lower = (B - z * C) / D;
    upper = (B + z * C) / D;
  }

  return { lower, upper, total, harshness };
};

const qid = document.getElementById('question').dataset.questionid;
const answersList = document.getElementById('answers');

const keyFunc = key => (apple, orange) => {
  const appleKey = key(apple);
  const orangeKey = key(orange);
  return (appleKey > orangeKey) - (appleKey < orangeKey);
};

class RawHtmlString extends String {}

const html = (lits, ...values) =>
  new RawHtmlString(
    String.raw({ raw: lits.map(s => s.replace(/^\n[ \t]*|\n[ \t]*$/gu, '')) },
               ...values.map(value => value instanceof RawHtmlString
                                              ? value
                                                : String(value).replace(/[&<"']/gu, s => `&#${s.charCodeAt(0)};`))));

const performSort = async () => {
  const answerNodes = answersList.querySelectorAll('.answer');
  if (answerNodes.length < 2)
    return;

  const answerNodeMap = new Map(Array.from(answerNodes).map(node => [+node.dataset.answerid, node]));
  const unscoredAnswers = new Set(answerNodeMap.keys());
  const afterLastAnswer = answerNodes[answerNodes.length - 1].nextSibling;
  const answers = await getAnswerScores(qid);

  const scoredAnswers = answers.map(answer => {
    unscoredAnswers.delete(answer.answer_id);

    const { upper, lower, total, harshness } = scoreWilson(answer);
    let node = answerNodeMap.get(answer.answer_id);
    let anchorNode;

    if (node) {
      anchorNode = node.previousElementSibling;
      
      const voteCounter = node.querySelector('.js-vote-count');
      if (OPT_UNCOVER_VOTES) {
        const signChar = (n, sign) => 0 < n && n < 100 ? sign : "";
        voteCounter.innerHTML = html`
          <div class="fc-green-600">${signChar(answer.up_vote_count, "+")}${answer.up_vote_count}</div>
          <hr class="vote-count-separator">
          <div class="fc-red-600">${signChar(answer.down_vote_count, "−")}${answer.down_vote_count}</div>
        `;
      }
    } else {
      const by = (user) => user == null ? '' : html`
                    by ${
                user.user_id !== void 0
                ? html`
              <a href="/u/${user.user_id}">
                                <img src="${user.profile_image}" width="24" height="24" class="va-middle"
                     alt="">&nbsp;${user.display_name}
              </a>
                        `
                : html`${user.display_name}`
            }
                `;
      const at = (ts) => html`
                    <time title="${new Date(ts * 1000).toISOString()}">${
                new Intl.DateTimeFormat("en-US", {
              dateStyle: 'medium',
              timeStyle: 'short',
              hourCycle: 'h23',
            }).format(ts * 1000)
            }</time>
                `;
      
      anchorNode = document.createElement('a');
      anchorNode.name = answer.answer_id;

      node = document.createElement('div');
      node.dataset.answerid = answer.answer_id;
      node.dataset.ownerid = answer.owner.user_id;
      node.dataset.json = JSON.stringify(answer);
      node.className = 'answer';
      node.innerHTML = html`
        <div class="post-layout">
          <div class="votecell">
            <div class="grid fd-column ai-center ta-center">
              +${answer.up_vote_count}
              <hr class="vote-count-separator">
              −${answer.down_vote_count}
              ${
                answer.awarded_bounty_amount
                ? html`
                    <div>
                        <div class="s-badge s-badge__bounty s-badge__sm" style="margin-top: 0.75em;">+${answer.awarded_bounty_amount}</div>
                    </div>
                  `
                : ''
              }
            </div>
          </div>
          <div class="answercell">
            <div>
              <p style="margin-bottom: 0.3em; line-height: 28px;"><a href="/a/${answer.answer_id}">answer</a>
                            added ${at(answer.creation_date)} ${by(answer.owner)}
                            ${answer.community_owned_date !== void 0 ? " (community wiki)" : ""}
                            ${answer.last_edit_date
                            ? html`<p style="margin-bottom: 0.3em">last edited ${at(answer.last_edit_date)} ${by(answer.last_editor)}`
                            : ''}
                            ${answer.comment_count ? html`<p style="margin-bottom: 0.3em">${answer.comment_count} comment(s)` : ''}
            </div>
          </div>
        </div>
      `;
    }

    return {
      data: answer,
      node,
      anchorNode,
      wilsonInterval: [lower, upper],
      wilsonTotal: total,
      harshness,
    };
  });

  const maxHarshness = Math.max(
    8,
    ...scoredAnswers
    .map(answer => answer.harshness)
    .filter(total => total === total) /* filter out NaN */
  );

  scoredAnswers.forEach(answer => {
    const { harshness, wilsonInterval: [lower, upper], data: { creation_date } } = answer;

    const harshFactor = (harshness / maxHarshness) * 0.5 + 0.5;

    answer.harshFactor = harshFactor;
    answer.wilsonScore = (lower * harshFactor + upper * (1 - harshFactor));
  });

  scoredAnswers.sort(keyFunc(answer => -answer.wilsonScore));
  
  for (const id of unscoredAnswers)
    scoredAnswers.push({
      node: answerNodeMap.get(id),
      anchorNode: answerNodeMap.get(id).previousElementSibling,
    });

  /* maintain the scroll position over what the user is (presumably) currently reading */
  
  let targetNode = document.querySelector('a:target + .answer, :target');

  /* hack for comment links */
  if (!targetNode) {
    const m = /^#(?:comment(\d+)_(\d+))$/.exec(location.hash);
    /* picking .comment-text below because for some reason getBoundingClientRect() on the comment node itself returns [0, 0, 0, 0] */
    if (m) {
        targetNode = document.getElementById(`comment-${m[1]}`)?.querySelector('.comment-text');
      targetNode = targetNode ?? document.getElementById(`answer-${m[2]}`);
        }
  }

  if (targetNode) {
    const html = document.documentElement;
    const rect = targetNode.getBoundingClientRect();
    const { left, right, top, bottom } = rect;
    if (!rect || !(rect.bottom >= 0 && rect.right >= 0 && rect.left <= html.clientWidth && rect.top <= html.clientHeight))
      targetNode = null;
  }

  targetNode = targetNode ?? scoredAnswers.find(
    answer => {
      const { offsetTop: ny0, offsetHeight: nh } = answer.node;
      const { scrollY: sy, innerHeight: wh } = window;
      const whh = wh / 2;
      const ny1 = ny0 + nh;
      return (ny0 - sy <= whh) && (ny1 - sy >= whh);
    })?.node;

  const targetNodeDelta = targetNode ? window.scrollY - targetNode.offsetTop : null;
  
  for (const answer of scoredAnswers) {
    const { node, anchorNode } = answer;

    afterLastAnswer.parentNode.insertBefore(node, afterLastAnswer);
    afterLastAnswer.parentNode.insertBefore(anchorNode, node);
  }
  
  if (targetNode) {
    window.scroll({ top: targetNode.offsetTop + targetNodeDelta });
  }
};

const sortSwitch = document.querySelector('select#answer-sort-dropdown-select-menu');
const sortReset = document.querySelector('label[for="answer-sort-dropdown-select-menu"] + .js-sort-preference-change');
let option = null;

const switchSort = async () => {
  const waitNode = answersList;

  waitNode.style.cursor = 'wait';
  waitNode.style.opacity = '0.5';
  try {
    await performSort();
  } catch (e) {
    alert(`Sort Best First failed: ${e}`);
    throw e;
  } finally {
    waitNode.style.cursor = '';
    waitNode.style.opacity = '';
  }

  if (option) {
    option.selected = true;
  }
  sortReset?.classList.remove('d-none');

  // hack to make links keep working
  try {
    const n = new URL(location.href);
    if (!n.searchParams.get('answertab'))
      return;
    n.searchParams.delete('answertab');
    const s = n?.toString().replace(/#.*$/u, "");
    history.replaceState(history.state, '', s);
  } catch (e) {
    console.warn(`[SB1st] error while resetting ?answertab=`, e);
  }
};

if (sortSwitch) {
  option = document.createElement('option');
  option.textContent = isMeta
    ? 'Wilson score (best first)'
    : 'Modified Wilson score (best first)';
  sortSwitch.addEventListener('change', async ev => {
    if (option.selected) {
      ev.stopPropagation();
      await switchSort();
    }
    GM.setValue(GM_KEY_ACTIVE, option.selected);
  }, false);
  sortSwitch.appendChild(option);
}

sortReset?.addEventListener('click', async ev => {
  if (ev.defaultPrevented)
    return;
  GM.setValue(GM_KEY_ACTIVE, false);
}, false);

if (answersList) {
    if (!new URL(location.href).searchParams.has('answertab')) {
    GM.getValue(GM_KEY_ACTIVE, false).then(value => {
      if (value)
        switchSort();
    });
  }
}
1
  • 2
    Thank you for contributing to Stack Apps; I can't help but thinking about one of my own scripts which adds the Wilson confidence rating to posts.
    – Glorfindel
    Apr 11, 2021 at 18:10

1 Answer 1

3

Test cases:

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .