5

Inspired by the Fastest Gun question on the central Meta, this script implements answer sorting based on a modified Wilson score. Installing the script adds a ‘Modified Wilson score’ sort option on question pages that attempts to sort the highest-quality answers first. One should hope this ordering method to boost hidden gems that would otherwise be buried under hastily-written popular-but-wrong answers that would otherwise hold the top spot under a naïve score tally sort.

Though initially directly inspired by the FGITW problem, ultimately the script aims to mitigate the related, but broader and more nebulous problem of popular-but-wrong (PBW) answers. As such, a number of weighing tweaks have been applied to take the specific voting structure of Stack Exchange sites into account:

  • Downvotes are counted as five times heavier than upvotes, as they are assumed a better indicator of answer quality for a number of reasons:

    • The reputation threshold to cast upvotes is very low (15), much lower than for downvotes (125 on most sites). As such, upvotes should be assumed used more often by users who are less informed than the downvoters (with respect to the topic at hand, or the Stack Exchange customs).
    • Downvotes cost reputation to cast, therefore they are used more sparingly and with more care, and presumably, usually for reasons of merit.0 Upvotes are assumed more likely to be cast for more superficial reasons (appreciation of effort, formatting, writing style, humour).

    This weighing is disabled on Stack Apps and Meta sites, including the central Meta; on those, the script reverts to using the plain, unweighed Wilson confidence interval. The above reasons don’t apply as much to Meta sites: on regular meta sites, neither upvotes nor downvotes cost reputation to cast, it is assumed most Meta participants can downvote and furthermore it is assumed that Meta answers represent the users’ opinions, where votes cannot be meaningfully ‘incorrect’ in judging an answer’s merit (there is no such things as a popular-but-wrong meta post); the best the script can do is estimate how well a given answer would have been received, had everyone read it.

  • Bounties are counted with upvotes; to avoid high bounties biasing the scores too much, bounty awards are scaled and square-rooted. A bounty of 25 is worth 1 extra upvote; a bounty of 500 is worth 10 upvotes.

  • Answers with more (unweighed) votes are judged more harshly: that is, the more votes an answer has relative to other answers, the closer the final score is to the lower end of its Wilson confidence interval. Answers with few votes have scores closer to the middle of the confidence interval; the answer with the most votes has the final score at the bottom of the confidence interval, unless all answers have fewer than 8 votes total (to avoid disturbing monotonicity where all answers have few votes overall).

  • Answer acceptance is not taken into account. In my experience, it tends to be a poor proxy for quality relative to other factors. Since askers usually upvote simultaneously with accepting the question anyway, I just did not bother accounting for acceptance at all.

If the question has answers that weren’t displayed on the current page, the script will generate stubs linking to the full answers. It may happen that those answers score higher than those visible on the current page and as such, the stubs may appear between full answers; this is intentional and not a bug. Deleted answers, when visible, are always sorted last.

The weighing factors were not determined by any principle other than ‘they felt right’, so beware. In particular, the 5× multiplier on downvotes was chosen mainly as a counterbalance to the five-times-smaller penalty for receiving an answer downvote than the reward for receiving an answer upvote. Statistical analyses of vote distribution, number of eligible voters and the number of views on each site might allow us to come up with fairer weighing factors.


0 Of course, downvoting for spite is not unheard of, though I assume most of it occurs as serial downvoting, which is usually swiftly corrected.

Script code

// ==UserScript==
// @name     StackExchange: Sort Best First
// @grant    none
// @match    https://*.stackexchange.com/*
// @match    https://*.superuser.com/*
// @match    https://*.stackoverflow.com/*
// @match    https://*.mathoverflow.net/*
// @match    https://*.serverfault.com/*
// @match    https://*.askubuntu.com/*
// @match    https://stackapps.com/*
// @exclude  https://chat.stackexchange.com/*
// @exclude  https://api.stackexchange.com/*
// @exclude  https://data.stackexchange.com/*
// @exclude  https://openid.stackexchange.com/*
// @exclude  https://stackexchange.com/*
// @exclude  https://contests.stackoverflow.com/*
// @exclude  /^https?:\/\/winterbash\d{4,}\.stackexchange\.com\//
// ==/UserScript==

const sleep = t =>
    new Promise((accept, reject) => setTimeout(accept, t));

const isMeta = /(^|\.)(meta\.|stackapps\.com$)/.test(location.hostname);

const getAnswerScores = async qid => {
  const usp = new URLSearchParams;
  usp.set('site', location.hostname);
  usp.set('filter', '!2oRRX0IVTN-3*1DAJ2N2SJiLCR.2VfC)CoC(Q7r-p.');

  const fullResult = [];

  let page = 1;
  for (let page = 1;; page++) {
    usp.set('page', page);
    const response = await fetch(`https://api.stackexchange.com/2.2/questions/${qid}/answers?${usp}`);
    const result = await response.json();

    if (result.error_id)
      throw new Error(`${result.error_message} [${result.error_name}, ${result.error_id}]`);

    fullResult.push(...result.items);

    if (!result.has_more)
      break;
    if (result.backoff)
      await sleep(result.backoff * 1000);
  }

  return fullResult;
};

const scoreWilson = (answer) => {
  const SCALE = Math.log(10) / Math.log(20);
  const up = answer.up_vote_count + (answer.awarded_bounty_amount ? Math.pow(answer.awarded_bounty_amount / 25, SCALE) : 0);
  const dn = isMeta ? answer.down_vote_count : 5 * answer.down_vote_count;
  const total = up + dn;
  const harshness = answer.up_vote_count + answer.down_vote_count;
  const z = 1.96;

  let upper = 1, lower = 0;

  if (total !== 0) {
    const upRatio = up / total;
    const B = upRatio + z * z / (2 * total);
    const C = Math.sqrt((upRatio * (1 - upRatio) + z * z / (4 * total)) / total);
    const D = (1 + z * z / total);

    lower = (B - z * C) / D;
    upper = (B + z * C) / D;
  }

  return { lower, upper, total, harshness };
};

const qid = document.getElementById('question').dataset.questionid;
const answersList = document.getElementById('answers');

const keyFunc = key => (apple, orange) => {
  const appleKey = key(apple);
  const orangeKey = key(orange);
  return (appleKey > orangeKey) - (appleKey < orangeKey);
};

const rawHTML = s => {
  const result = new String(s);
  result.rawHTML = true;
  return result;
};

const html = (lits, ...values) => {
  const ESCAPE = { '<': '&lt;', '&': '&amp;', '"': '&quot;', "'": '&#39;', };
  return rawHTML(String.raw({ raw: lits }, ...values.map(value => {
    if (value instanceof Node)
      return (new XMLSerializer).serializeToString(value);
    else if (value instanceof String && value.rawHTML)
      return value;
    else
      return String(value).replace(/[&<"']/gu, s => ESCAPE[s]);
  })));
};

const performSort = async () => {
  const answerNodes = answersList.querySelectorAll('.answer');
  if (answerNodes.length < 2)
    return;

  const answerNodeMap = new Map(Array.from(answerNodes).map(node => [+node.dataset.answerid, node]));
  const unscoredAnswers = new Set(answerNodeMap.keys());
  const afterLastAnswer = answerNodes[answerNodes.length - 1].nextSibling;
  const answers = await getAnswerScores(qid);

  const scoredAnswers = answers.map(answer => {
    unscoredAnswers.delete(answer.answer_id);

    const { upper, lower, total, harshness } = scoreWilson(answer);
    let node = answerNodeMap.get(answer.answer_id);
    let anchorNode;

    if (node)
      anchorNode = node.previousElementSibling;
    else {
      anchorNode = document.createElement('a');
      anchorNode.name = answer.answer_id;

      node = document.createElement('div');
      node.dataset.answerid = answer.answer_id;
      node.dataset.ownerid = answer.owner.user_id;
      node.className = 'answer';
      node.innerHTML = html`
        <div class="post-layout">
          <div class="votecell">
            <div class="grid fd-column ai-center ta-center">
              +${answer.up_vote_count}
              <hr class="vote-count-separator">
              −${answer.down_vote_count}
            </div>
          </div>
          <div class="answercell">
            <div>
              <a href="/a/${answer.answer_id}">answer</a>
              by ${
                answer.owner.user_id !== void 0
                  ? html`
                    <a href="/u/${answer.owner.user_id}">
                      <img src="${answer.owner.profile_image}" width="32" height="32" alt="" class="va-middle">
                      ${answer.owner.display_name}
                    </a>
                  `
                  : html`${answer.owner.display_name}`
              }
            </div>
          </div>
        </div>
      `;
    }

    return {
      data: answer,
      node,
      anchorNode,
      wilsonInterval: [lower, upper],
      wilsonTotal: total,
      harshness,
    };
  });

  const maxHarshness = Math.max(
    8,
    ...scoredAnswers
    .map(answer => answer.harshness)
    .filter(total => total === total) /* filter out NaN */
  );

  scoredAnswers.forEach(answer => {
    const { harshness, wilsonInterval: [lower, upper], data: { creation_date } } = answer;

    const harshFactor = (harshness / maxHarshness) * 0.5 + 0.5;

    answer.harshFactor = harshFactor;
    answer.wilsonScore = (lower * harshFactor + upper * (1 - harshFactor));
    
    answer.node.dataset.sbfData = JSON.stringify({
      lower, upper, harshness
    });
  });

  scoredAnswers.sort(keyFunc(answer => -answer.wilsonScore));
  
  for (const id of unscoredAnswers)
    scoredAnswers.push({
      node: answerNodeMap.get(id),
      anchorNode: answerNodeMap.get(id).previousElementSibling,
    });

  /* maintain the scroll position over what the user is (presumably) currently reading */
  
  let targetNode = document.querySelector('a:target + .answer, :target');

  /* hack for comment links */
  if (!targetNode) {
    const m = /^#(comment(\d+)_(\d+))$/.exec(location.hash);
    /* picking .comment-text below because for some reason getBoundingClientRect() on the comment node itself returns [0, 0, 0, 0] */
    if (m) {
        targetNode = document.getElementById(`comment-${m[2]}`)?.querySelector('.comment-text');
      targetNode = targetNode ?? document.getElementById(`answer-${m[3]}`);
        }
  }

  if (targetNode) {
    const html = document.documentElement;
    const rect = targetNode.getBoundingClientRect();
    const { left, right, top, bottom } = rect;
    if (!rect || !(rect.bottom >= 0 && rect.right >= 0 && rect.left <= html.clientWidth && rect.top <= html.clientHeight))
      targetNode = null;
  }

  targetNode = targetNode ?? scoredAnswers.find(
    answer => {
      const { offsetTop: ny0, offsetHeight: nh } = answer.node;
      const { scrollY: sy, innerHeight: wh } = window;
      const whh = wh / 2;
      const ny1 = ny0 + nh;
      return (ny0 - sy <= whh) && (ny1 - sy >= whh);
    })?.node;

  const targetNodeDelta = targetNode ? window.scrollY - targetNode.offsetTop : null;
  
  for (const answer of scoredAnswers) {
    const { node, anchorNode } = answer;

    afterLastAnswer.parentNode.insertBefore(node, afterLastAnswer);
    afterLastAnswer.parentNode.insertBefore(anchorNode, node);
  }
  
  if (targetNode) {
    window.scroll({ top: targetNode.offsetTop + targetNodeDelta });
  }
};

const switchSort = async () => {
  const waitNode = answersList;
  waitNode.style.cursor = 'wait';
  waitNode.style.opacity = '0.5';

  try {
    await performSort();
  } catch (e) {
    alert(`Sort Best First failed: ${e}`);
    throw e;
  } finally {
    waitNode.style.cursor = '';
    waitNode.style.opacity = '';
  }

  if (option) {
    option.selected = true;
  }

  // hack to make links keep working
  try {
    const n = new URL(window.location.href);
    if (!n.searchParams.get('answertab'))
      return;
    n.searchParams.delete("answertab");
    const s = n?.toString().replace(/#.*$/u, "");
    history.replaceState(history.state, '', s);
  } catch (e) {
    console.warn(e);
  }
};

const LSKEY = 'sbf-active';

const sortSwitch = document.querySelector('select#answer-sort-dropdown-select-menu');
let option = null;

if (sortSwitch) {
  option = document.createElement('option');
  option.textContent = isMeta
    ? 'Wilson score (best first)'
    : 'Modified Wilson score (best first)';
  sortSwitch.addEventListener('change', async ev => {
    if (option.selected) {
      ev.stopPropagation();
      await switchSort();
    }
    localStorage.setItem(LSKEY, option.selected ? '1' : '0');
  }, true);
  sortSwitch.appendChild(option);
}

if (answersList) {
  let doIt = false;

  try {
    doIt = (localStorage.getItem(LSKEY) === '1');
  } catch (e) {
    console.warn(e);
  }

  if (doIt)
    switchSort();
}
1
  • Thank you for contributing to Stack Apps; I can't help but thinking about one of my own scripts which adds the Wilson confidence rating to posts.
    – Glorfindel
    Apr 11, 2021 at 18:10

1 Answer 1

1

Test cases:

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .