3

(I've included the because https://api.stackexchange.com/docs starts with Stack Exchange API v2.3.)

From this answer I see there's no way to get all questions with a specific tag.

However, what about all questions/answers asked/answered by a user with a specific tag?

To avoid this question to be an example of the XY problem, I want to clarify that my final target is to be able to show on a personal site the reputation I earned on this or that tag.

I see the API provides

On the one hand, questions come with a tags field, whereas answers do not. On the other hand, answers come with a question_id.

Therefore, I think I could do the following:

  1. For questions

    1.1. make the /users/{ids}/questions request;

    1.2 filter the obtained questions based on a specific tag;

    1.3. extract the score field from each question;

    1.4. sum up all entries.

  2. For answers

    2.1 make the /users/{ids}/answers request;

    2.2 for each of them, get the question_id;

    2.3 for each question_id, make a new request using the /questions/{ids} method;

    2.4 on the array of questions, apply steps from 1.2 to 1.4 above.

Then the two amounts would be added up and multiplied by 10.


However, before venturing, I'd like some feedback on whether I'm taking the correct route or not.

6
  • "From this answer I see there's no way to get all questions with a specific tag." That answer does not say "there's no way to get all questions with a specific tag". That answer says it's not possible in a single request. The answer is saying that because the answer assumes there are > 100 questions with the desired tag and the SE API will return at most 100 questions per request. The answer goes on to explain how you can get all questions in a tag by making additional requests.
    – Makyen
    Nov 14 at 16:52
  • The amount of reputation you've earned from a specific tag may be more or less than the upvotes and downvotes you've received in a particular tag. It is very likely rep is not accurately reflected as score for a particular tag, or even the sum of the score for questions and answers in a particular tag. You gain reputation for upvotes, bounties, and accepts (both questions and answers, but not on accepting your own answers on your own questions). You loose rep for downvotes, but a different amount, so can't just use score, and bounties. Reputation gain is also limited to a daily max.
    – Makyen
    Nov 14 at 17:16
  • When "score" is stated, it's almost always (upvotes - downvotes). In your example, 2 upvotes and 1 downvote is a net increase of 18 reputation. A user's reputation starts at 1, and can never go below 1. If the above 2 upvotes and 1 downvote are the only reputation changes for the user, or the only changes for the user after returning to a reputation of 1, then the user's total reputation is 19.
    – Makyen
    Nov 14 at 17:41
  • Why almost always? And why 2up+1down=+18rep instead of 19? I see a downvote costs me -1, not -2.
    – Enlico
    Nov 14 at 17:44
  • 1
    I said "almost always" primarily to hedge my bets. I'm not thinking of a specific place where "score" is used, but doesn't mean (upvotes - downvotes). Unfortunately, there are quite a lot of times/places where terminology isn't used precisely, so there's likely to be places where "score" is used where "upvotes" is really meant. There are definitely places on the site where "votes" is used where it means "score".
    – Makyen
    Nov 14 at 17:52
  • 1
    Receiving a downvote on one of your posts costs you 2 reputation. You casting a downvote on an answer costs you 1 reputation (so -1 for you and -2 for the author of the answer; if it's a community wiki post, then the author doesn't get -2 reputation, but also doesn't get +10 for upvotes). The cost of casting a downvote on a question is 0.
    – Makyen
    Nov 14 at 17:53
3

It is easier to use the /users/{id}/top-tags (JSON) endpoint for a single user.

That will give you for a given user all their tags with score and count for question and answers. Don't let the "top" misguide you. The user I used as an example has 440 tags in the API result and that is roughly 50 tags off compared to their site profile (and SEDE). Close enough I would say.

{
  "items": [
    {
      "user_id": 811,
      "answer_count": 176,
      "answer_score": 10331,
      "question_count": 0,
      "question_score": 0,
      "tag_name": "javascript"
    },
    {
      "user_id": 811,
      "answer_count": 37,
      "answer_score": 4211,
      "question_count": 0,
      "question_score": 0,
      "tag_name": "html"
    },
    {
      "user_id": 811,
      "answer_count": 95,
      "answer_score": 4031,
      "question_count": 0,
      "question_score": 0,
      "tag_name": "jquery"
    },

Do notice the default filter doesn't include Total in the wrapper, so you have to add that field if you want to know upfront how many pages you have to fetch.

5
  • Yes!!! I knew there was another easier way, thanks! And yes, I got misguided by the top: when I show those entries have the top- part, I didn't even look at them; even the description is misleading, I think, e.g. Get the top tags (by score) a single user has posted in. However, I've not really understood the part "The user I used [...] Close enough I would say". What inaccuracy are you referring to? What is 440 and what 40?
    – Enlico
    Nov 14 at 14:16
  • Also, as regards the last paragraph, I see that for a request of mine it shows "total":440, however page 15 is the last one to have a non-empty "items" field. I have verified that indeed each page has 30 items and the last just 20, so 14*30+20=440. For that total to be useful to know upfront how many pages I have to fetch, I need to know how many entries are in each page.
    – Enlico
    Nov 14 at 14:30
  • (cont.d) In my case it looks like it's 30, but where does this number come from?
    – Enlico
    Nov 14 at 14:30
  • Ok, page_size seems to contain that info, so Math.ceil(total/page_size) tells me the pages I need to fetch.
    – Enlico
    Nov 14 at 14:31
  • 1
    @Enlico the number 30 as well as how the paging works is explained in the API docs. P.S. I would go for recursively fetching the pages with a breaking condition of has_more being set to false - there is no need to pre-count the number of pages. Nov 16 at 3:58
2

As a more involved alternative, there is a dedicated endpoint for retrieving reputation changes - /reputation. It returns a list of Reputation objects, each having a post_id and reputation_change field. This is the only accurate metric for counting reputation changes - score field does not represent this information accurately for various reasons (including retroactive changes to vote weight by SE - yes, it happened before).

After retrieving the list of reputation changes, you need to extract the post ids, uniquify them (to avoid extra API calls as the query is heavy as is), fetch the /questions batch endpoint to get the list of questions, filter by tag, and map the result to a list of ids.

Then the only thing you need to do is to filter the reputation changes depending on whether the post_id is present in the abovementioned list or not. This will yield you reputation changes for bounties, suggested edits, and questions. The only thing left is getting changes for answers, but I leave that as an exercise for the reader.

The following snippet demonstrates how one could convert this idea into working code:

Important note: depending on your activity levels, the number of API calls might be a little heavy, please account for that if you are an avid userscript user and thus have to be weary about the daily quota usage

const API_VER = 2.3;
const API_BASE = "https://api.stackexchange.com";

const sleep = (sec = 1) => new Promise((r) => setTimeout(r, sec * 1e3));
const uniquify = (arr) => [...new Set(arr)];

const getQuestions = async(postIds, {
  site = "stackoverflow",
  page = 1,
  ...rest
}) => {
  const url = new URL(`${API_BASE}/${API_VER}/questions/${postIds.join(";")}`);
  const params = new URLSearchParams({
    site,
    page: page.toFixed(0),
    ...rest,
  });
  url.search = params.toString();
  const res = await fetch(url.toString());
  if (!res.ok)
    return [];
  const {
    items = [], has_more = false, backoff,
  } = (await res.json());
  if (backoff) {
    await sleep(backoff);
    return getQuestions(postIds, {
      site,
      page,
      ...rest
    });
  }
  if (!has_more)
    return items;
  const more = await getQuestions(postIds, {
    page: page + 1,
    site,
    ...rest,
  });
  return [...items, ...more];
};

const getReputation = async(userIds, {
  page = 1,
  site = "stackoverflow",
  ...rest
}) => {
  const url = new URL(`${API_BASE}/${API_VER}/users/${userIds.join(";")}/reputation`);
  const params = new URLSearchParams({
    site,
    page: page.toFixed(0),
    ...rest,
  });
  url.search = params.toString();
  const res = await fetch(url.toString());
  if (!res.ok)
    return [];
  const {
    items = [], has_more = false, backoff,
  } = (await res.json());
  if (backoff) {
    await sleep(backoff);
    return getReputation(userIds, {
      site,
      page,
      ...rest
    });
  }
  if (!has_more)
    return items;
  const more = await getReputation(userIds, {
    page: page + 1,
    site,
    ...rest,
  });
  return [...items, ...more];
};

const partition = (arr, perPart = 1) => {
  const parts = Math.ceil(arr.length / perPart);
  const partitions = [];
  for (let i = 0; i < parts; i++) {
    const start = i * perPart;
    partitions.push(arr.slice(start, start + perPart));
  }
  return partitions;
};

(async() => {
  const useridInput = document.getElementById("user_id");
  const siteInput = document.getElementById("site");
  const tagInput = document.getElementById("tag");
  const progress = document.getElementById("load");

  const submit = document.getElementById("search");
  submit.addEventListener("click", async() => {
    progress.value = 0;

    const tag = tagInput.value;
    const uids = [+useridInput.value];
    const site = siteInput.value;
    const key = "zhhhBNmsqZLZ967tc2dn8w((";
    
    const common = { key, site, pagesize: 100 };

    const changes = await getReputation(uids, common);
    
    progress.value = 35;

    const postIds = uniquify(changes.map(({
      post_id
    }) => post_id));

    const partitionedIds = partition(postIds, 100);
    const results = await Promise.all(partitionedIds.map((i) => getQuestions(i, common)));
    const questions = results.flat();
    
    progress.value = 70;
    
    const tagged = questions.filter(({
      tags = []
    }) => tags.includes(tag));
    const taggedIds = tagged.map(({
      question_id
    }) => question_id);
    const relevantChanges = changes.filter(({
      post_id
    }) => taggedIds.includes(post_id));
    const totals = {
      up: 0,
      down: 0,
      tag
    };
    relevantChanges.forEach(({
      reputation_change
    }) => {
      totals[reputation_change > 0 ? "up" : "down"] += Math.abs(reputation_change);
    });
    
    progress.value = 100;
    
    const up = document.getElementById("up");
    const down = document.getElementById("down");
    
    up.textContent = totals.up;
    down.textContent = totals.down;
  });
})();
label {
  display: block;
  font-weight: 600;
}

caption,
label {
  font-weight: 600;
  margin: 1vh 0;
}

input {
  display: block;
}

button, progress {
  margin-top: 2vh;
  margin-bottom: 2vh;
}
<form>
  <label for="site">Site API slug</label>
  <input id="site" type="text" title="API slug" placeholder="stackoverflow" value="stackoverflow" />

  <label for="user_id">User Id</label>
  <input id="user_id" type="text" title="User Id" placeholder="12345" value="" />

  <label for="tag">Tag Name</label>
  <input id="tag" type="text" title="Tag Name" placeholder="typescript" value="" />

  <button id="search" type="button">Get Changes</button>
</form>

<progress id="load" max="100" value="0"></progress>

<p>Up: <span id="up"></span></p>
<p>Down: <span id="down"></span></p>

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .