How do I get the total number of records for a given search?

Question

I want to use the Stack Exchange API and get the total number of results returned from the search route.

I wrote this recursive script to find the last page with the has_more=False, is there a better way to do so?

If not, can this be a feature?

My Code:

import requests
api_url = "https://api.stackexchange.com/2.2/"

def recursive_total(site, q, pagesize=100, pagemin=0, pagemax=1000000):
    pageindex = (pagemin + pagemax) / 2
    page = requests.get(api_url + '/search',
                        {'site': site, 'intitle': q, 'page': pageindex + 1, 'pagesize': pagesize}).json()
    if page['has_more']:
        return recursive_total(site, q, pagesize, (pagemin + pagemax) / 2, pagemax)
    else:
        if len(page['items']) == 0:
            return recursive_total(site, q, pagesize, pagemin, (pagemax + pagemin) / 2)
        else:
            return pagesize * pageindex + len(page['items'])

Answer 1

In order to get the total, you need to use a filter that includes it.

If you just want the total, as your sample code implies, then use the built-in total filter. For example:

          /2.2/search?intitle=X&site=stackoverflow&filter=total

If you want the total AND the results, then you must use a custom filter with the .wrapper -> total property selected. For example:

          /2.2/search?intitle=X&site=stackoverflow&filter=!w-1xWerkkwLn6avAMY

See this answer for help creating custom filters.

---

Note: you should use care when requesting totals. IIRC:

• Fetching just the total is relatively inexpensive.
• Fetching mixed results and the total is relatively costly (resources/SQL plan).
• More expensive requests take longer and attract more backoff warnings and throttling.

So if you want both the total and the individual results, I recommend that you either: (a) total them up yourself, or (b) only ask for the total once, at the beginning, and then page through the results using a normal filter.