Take the 2-minute tour ×
Stack Apps is a question and answer site for apps, scripts, and development with the Stack Exchange API. It's 100% free, no registration required.

UPDATE: 2.x support is now mainline! Please read the wiki page for important information about the update.

A warm welcome to you, traveller. You have arrived at the home of Py-StackExchange, the library definitively proven to be the best library for using the SE API from Python. If you are still interested (and by golly, you should be) after glancing at the masterpiece below, please check the wiki on Github.

† Ahem.

About

So, what is Py-StackExchange? Well, I'm glad you asked.

It is a Python library for querying the StackExchange API from your Python applications. Integration, ahoy!

So why should you use it? After all, the SE API is sooo simple that you might think it'd be quicker to just write your own, and that it'd be faster and you wouldn't have to look at all that documentation and do all that thinking... well:

Let's start with the API coverage - what can the API do? And, more importantly, what can the library do?

  • Access any StackExchange site, with just its URL! Even those that aren't online yet!
  • If you just can't decide which one to use, you can use StackAuth to look up the full list of sites.
  • Once you're online, you can view everything about users, questions, answers, badges, comments and tags.
  • You can even go back in time by playing with post revisions.
  • Stalk Generate a detailed profile of a user's life Help users by looking up every StackExchange account they have. Every single one.
  • And, on any of those sites, peruse a detailed history of everything they've ever done - every edit, every comment, every time they were awarded a badge... Watch StackOverflow become the new Facebook overnight with the timeline feature.
  • See how well an SE site is doing; obsessively check its site statistics.
  • Search the questions of StackExchange sites.

So, why not write your own classes to consume said pure, concentrated brilliance?

  • Let someone else deal with all that laborious HTTP request business... you know you want to...
  • All the little idiosyncratic potholes on your road to API happiness have been filled in for you. We have little elves which jump into your code and parse your JSON and your dates and your lists until every response is itself a little baby python.
  • URLs change 99.9% more often than the interface of this module. Fact.
  • It's faster than Michael Palin on a broken bicycle. It also knows about request throttling, so when it gets too fast for its own good, it applies the brakes just enough to restore order.
  • It loads lazily information that would take another request to fetch, meaning you never use more of your limit than you need to.
  • It caches requests automatically, so you need to care slightly less about writing efficient code! (new in 1.1)

Now, onto the religious advantages:

  • Documentation? Bah, we have naming conventions. (This feature was inspired by Rails.)
    Pssst - don't tell anyone, but there is documentation too, if that's your style. (README/Wiki)
  • Naming conventions? Who needs them? We have an interactive program that writes your code for you while you look around the StackExchange site of your choice. (This feature was inspired by Jon Skeet.)
  • Almost-sentient, artificially intelligent programming programs? Ugh, how 20th century. There are metric heaps of example code available in the source repo, a small excerpt of which is presented below for your viewing pleasure.

Please note: This is not an official product of Stack Overflow Internet Services, Inc.

Code Snippet

The wiki has details of all the example code in the code repository. In fact, here's a small taster from the Narcissism demo.

#!/usr/bin/env python

# a hack so you can run it 'python demo/stats.py'
import sys
from stackauth import StackAuth
from stackexchange import Site, StackOverflow

user_id = 41981 if len(sys.argv) < 2 else int(sys.argv[1])
print 'StackOverflow user %d\'s accounts:' % user_id

stack_auth = StackAuth()
so = Site(StackOverflow)
accounts = stack_auth.associated(so, user_id)
reputation = {}

for account in accounts:
    print '  %s: %s / %d reputation' % (account.display_name, account.on_site.name, account.reputation)

    reputation[account.reputation] = account.on_site.name

print 'Most reputation on: %s' % reputation[max(reputation)]

Or how about a scrolling list of questions?

import stackexchange
so = stackexchange.StackOverflow()

for q in so.questions(pagesize=50):
    print q.title

About

This is a Python library/wrapper around the StackExchange and StackAuth APIs. It provides a clean, object-oriented API for accessing the various sites.

License

The script is licensed under the Simplified BSD license. You can find the full text of the license here, but the gist of it is that:

  • You need to give attribution when you distribute (compiled or in source form) the library - not your application unless you include the library files `in the box'.
  • The standard "NO WARRANTY" (in caps!) is provided.

Other than that you can more or less do what you like!

Download

In the bad old days (i.e. about 2 hours before I wrote this), you had to manually install Py-StackExchange after cloning the Git repository.

You can still do this: http://github.com/lucjon/Py-StackExchange. You can also download a ZIP or TGZ file from there.

However, there is a new and improved way to get Py-StackExchange: you can install it straight from the PyPI! Just type:

~$ easy_install py-stackexchange

Also, distutils gives me fantastical benefits on the side, such as a completely original Windows installer with an all-new design. You can also find a stable source distribution on the downloads page @ Github.

Platform

The library is written in standard Python 2.6, with, as far as I am aware, no specific platform dependency. As long as your Python install has the full standard library available, it should work fine.

Python 2.6 is required for the json module. (EDIT: @ADB in the comments has noted that the SimpleJson library can be used instead. This means it works on Python 2.5 and also on the Google App Engine.)

Python 3.x is also supported.

Contact

The library is being written by Lucas Jones (lucasjones.co.uk / SO). If you want to contact me, send me some mail at lucas @ lucasjones.co.uk.

share|improve this question
1  
Awesome. I'm looking forward to the final version! –  Nathan Osman May 23 '10 at 2:55
1  
I started playing with it and it works. If anyone wants to use it with Google App Engine, replace the json import with SimpleJson-2.0.9 and it will work. –  ADB May 25 '10 at 11:05
    
@ADB: I'm glad it works on someone else's machine! :) –  Lucas Jones May 25 '10 at 15:22
    
Sorry to all to whom I haven't responded recently - I've been quite busy. –  Lucas Jones Jun 12 '10 at 21:40
    
Just for your information: you need to update to the latest api version. –  Edan Maor Jun 24 '10 at 17:51
    
@Edan: Just noticed that! Thanks. :) –  Lucas Jones Jun 24 '10 at 18:12
1  
@Edan: Done now. 0.8 stuff in separate branch now. –  Lucas Jones Jun 24 '10 at 19:10
    
Looks good! Please consider posting this library on StackList. I'd love to see it there and vote it 5-stars! –  Nathan Osman Jul 3 '10 at 22:02
    
@George Edison: Why not? :) –  Lucas Jones Jul 6 '10 at 20:39
    
@Lucas: ...and done! 5-stars for your library. –  Nathan Osman Jul 6 '10 at 21:18
    
@George: Thanks. Neat site, by the way - more structured than StackApps for that kind of thing. –  Lucas Jones Jul 6 '10 at 21:42
    
@Lucas: Thank you! Any suggestions for new features or enhancements are much appreciated. –  Nathan Osman Jul 6 '10 at 22:17
    
@George: Perhaps integration with source repos? :) Like displaying latest commit(s), etc. That would be useful - more of a centralised page. (Or, if you wanted to take it in that direction, some sort of basic Wiki-like feature; although another project forge is probably reinventing the wheel.) –  Lucas Jones Jul 7 '10 at 13:11
    
Well.... I don't think that I want to take on hosting code. But I could probably display commits, etc. for popular VCSs. –  Nathan Osman Jul 7 '10 at 16:18
1  
@Lucas great work, any plan to support also the V2.0 Api? –  systempuntoout Apr 12 '12 at 10:27

24 Answers 24

import stackexchange so =
stackexchange.Site(stackexchange.StackOverflow)
comment = so.comment(valid_comment_id)

the above code returns the following error in init.py:

    --->166         self.post_type = PostType.from_string(json.post_type)
        167 
        168         def _get_post(self):

AttributeError: DictObject instance has no attribute 'post_type'

I get the valid comment id returned from Firehose JSON streaming event service

share|improve this answer
1  
Thanks for bringing this to my attention, I've fixed the problem in the latest revision on the Github repository (github.com/lucjon/Py-StackExchange). Your code should now work if you use the version of the library from there. –  Lucas Jones Aug 28 at 16:58

Lucas,

Something must have changed in JSON data format sent from the Stack Exchange API because yesterday for no reason I started getting:

    ValueError("Unexpected UTF-8 BOM (decode using utf-8-sig)")

(I searched through many forums, reinstalled python a couple of times etc etc)

As the error says, there is a problem with the encoding. Basically all I had to do is change the encoding type

in the line 157 of the web.py file

from:

    parsed_result = json.loads(req.data.decode('utf8'))

to:

    parsed_result = json.loads(req.data.decode('utf-8-sig'))

and then reinstall the library.

Hope that helps anyone with the same issue...

share|improve this answer
    
Thanks for letting me know about this; however, I've not been able to reproduce it myself. Is there anything unusual about your setup (in particular, are there any proxies between you and Stack Exchange?) which might get in the way? Either way, if changing the 'encoding' is as harmless as it seems to be from my initial look at the documentation, I may well just change it. –  Lucas Jones Jul 4 at 0:56
    
I'm not sure what caused the problem. It seems to be something quite unusual that appears when you combine things like windows 8.1, cygwin and python3 altogether. –  Tomazz Jul 10 at 14:02

I'm trying to access different attributes of the Answer class but I get this error:

AttributeError: 'Answer' object has no attribute 'accepted'

I looked into your init.py file and I can see that you have a tuple there that lists the attributes:

transfer = ('accepted', 'locked_date', 'question_id', 'up_vote_count', 'down_vote_count', 'view_count', 'score','community_owned', 'title', 'body')

My intuition is that those attributes should be accessible in some way but it looks that I am doing something wrong. Could you please enlighten me?

share|improve this answer
    
Thanks for letting me know about this, it's a 2.x change that slipped through. The field is now called is_accepted; I've pushed a fix to the Github repository. –  Lucas Jones Jul 2 at 14:14

When I try running the code with python 3.4 I always get this error:

AttributeError: 'HTTPMessage' object has no attribute 'getheader'

Here is the full traceback. I'd be very happy if any of you could tell me what the problem is. Is it the fact that I'm using python3? I converted the example files using 2to3 and it seems to me that it should work fine...

Traceback (most recent call last):
  File ".\question.py", line 11, in <module>
    question = site.question(id)
  File "C:\Python34\lib\site-packages\py_stackexchange-1.1_4-py3.4.egg\stackexchange\__init__.py", line 677, in question
    q, = self.questions((nid,), **kw)
  File "C:\Python34\lib\site-packages\py_stackexchange-1.1_4-py3.4.egg\stackexchange\__init__.py", line 473, in __call__
    return self.site._get(Question, ids, 'questions', kw)
  File "C:\Python34\lib\site-packages\py_stackexchange-1.1_4-py3.4.egg\stackexchange\__init__.py", line 623, in _get
    return self.build(root, typ, coll, kw)
  File "C:\Python34\lib\site-packages\py_stackexchange-1.1_4-py3.4.egg\stackexchange\__init__.py", line 600, in build
    json = self._request(url, kw)
  File "C:\Python34\lib\site-packages\py_stackexchange-1.1_4-py3.4.egg\stackexchange\__init__.py", line 572, in _request
    json, info = request_mgr.json_request(url, new_params)
  File "C:\Python34\lib\site-packages\py_stackexchange-1.1_4-py3.4.egg\stackexchange\web.py", line 119, in json_request
    req = self.request(to, params)
  File "C:\Python34\lib\site-packages\py_stackexchange-1.1_4-py3.4.egg\stackexchange\web.py", line 98, in request
    if conn.info().getheader('Content-Encoding') == 'gzip':
AttributeError: 'HTTPMessage' object has no attribute 'getheader'
share|improve this answer
    
This is indeed a Python 3 issue; I have fixed this and a few other compatibility problems just now in the latest version of the library available on Github. The master branch targets v2.x of the StackExchange API, as will the latest PyPI version in the near future. If you want to continue using v1.x (e.g. you have an API key registered), the compatibility fixes have been backported to the v1.1 branch on Github. –  Lucas Jones Jun 26 at 22:13
    
Many thanks! I had no problems after your fix. –  Tomazz Jun 30 at 13:47

The API was working but now I receive an error when I try to run the demo examples.
Sample error dump:

/usr/bin/python2.7 /home/donbeo/PycharmProjects/stack_overflow/experience.py
StackOverflow user 41981's experience:
Traceback (most recent call last):
  File "/home/donbeo/PycharmProjects/stack_overflow/experience.py", line 13, in <module>
    user = so.user(user_id)
  File "/home/donbeo/Py-StackExchange-master/stackexchange/__init__.py", line 699, in user
    u, = self.users((nid,), **kw)
  File "/home/donbeo/Py-StackExchange-master/stackexchange/__init__.py", line 704, in users
    return self._get(User, ids, 'users', kw)
  File "/home/donbeo/Py-StackExchange-master/stackexchange/__init__.py", line 694, in _get
    return self.build(root, typ, coll, kw)
  File "/home/donbeo/Py-StackExchange-master/stackexchange/__init__.py", line 671, in build
    json = self._request(url, kw)
  File "/home/donbeo/Py-StackExchange-master/stackexchange/__init__.py", line 642, in _request
    json, info = request_mgr.json_request(url, new_params)
  File "/home/donbeo/Py-StackExchange-master/stackexchange/web.py", line 139, in json_request
    req = self.request(to, params)
  File "/home/donbeo/Py-StackExchange-master/stackexchange/web.py", line 111, in request
    conn = req_open.open(request)
  File "/usr/lib/python2.7/urllib2.py", line 410, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 523, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 448, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 531, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 400: Bad Request

Process finished with exit code 1

How can I solve this?

share|improve this answer
    
Thanks for getting in touch. The demo works for me locally; could I ask you to do the following: (1) ensure that you are using the latest version of the library from the Github repository; (2) if the problem persists, enable debug mode by adding the line stackexchange.web.WebRequestManager.debug = True to the top of your script. –  Lucas Jones May 27 at 22:40

The library currently does not yet support the use of access_token obtained through OAuth 2.0, does it?

share|improve this answer
    
This is, unfortunately, correct. –  Lucas Jones Dec 20 '12 at 18:35

I am running py-stackexchange v1.1-4 and I'm hitting a strange issue after not touching this library for perhaps eight months or so... My queries used to be relatively fast... I could get results in 30 seconds.

Now my queries literally take hours with the same code... example query:

def filtered_question_dict(sesite, mytags, pagesize=100,
    max_timedelta_hours=24):
    ## Get questions updated in the last 24 hours and not created more than 2 weeks ago

    all = dict()
    now = datetime.now()
    MAX_LAST_DELTA = timedelta(hours=max_timedelta_hours)
    MAX_CREATE_DELTA = timedelta(weeks=2)
    for ii in [sesite.questions.no_answers, sesite.recent_questions,
        sesite.questions.unanswered]:
        ## Iterate over questions, no answers sent, no body sent...
        for qq in ii(pagesize=pagesize, answers=False, body=False):
            qcreate = datetime.fromtimestamp(qq.json['creation_date'])
            qlast = datetime.fromtimestamp(qq.json['last_activity_date'])
            qtags = set(qq.json['tags'])
            if ((now-qlast)>MAX_LAST_DELTA) or \
                ((now-qcreate)>MAX_CREATE_DELTA) or \
                (len(mytags.intersection(qtags))==0):
                continue
            elif not accepted(qq.json):
                all[qtitle] = (qq, mytags, qcreate)
    return all

Just iterating over sesite.questions.no_answers ran for six hours last night before I hit control-C to stop it. I was also sniffing the session... I had pulled over 4000 'pages' in this query...

23467.909060 172.16.1.52 -> 64.34.119.12 HTTP GET /1.1/questions/no-answers?body=false&pagesize=100&answers=false&comments=false&key=REDACTED&page=4688 HTTP/1.1

Maybe Stack Exchange has changed their servers to rate-limit things, but if there is no server-side filtering it seems like the library would be unusable at this rate.

Is there something I should be doing differently?

share|improve this answer
    
I have a feeling I may have fixed this bug in the latest version of the code, but that I haven't pushed it to PyPI. I'll try and push the latest code up; in the meantime you can get it straight from the repository at github.com/lucjon/Py-StackExchange. –  Lucas Jones May 7 '12 at 18:06
    
OK. The behaviour of the iterators changed recently. Site.questions.no_answers, Site.questions.recent_questions and Site.questions.unanswered will go on forever. The pagesize= argument actually tells the API how many items to return on each iteration - it will just keep iterating until it's returned all the pages. I agree this is not the most useful behaviour for this case, and I'm now planning to add a cleaner interface. For now, though, change your loop to for qq in ii(...).items, which will only iterate through the questions on the first page, of size pagesize. –  Lucas Jones May 11 '12 at 14:31
    
@LucasJones, I will try that. Thank you for following up –  Mike Pennington May 11 '12 at 15:18

Based on the demo *recent_questions.py* example, I am using the following code to retrieve the 10 most recent questions at the command line:

so = stackexchange.Site(stackexchange.StackOverflow)
questions = so.recent_questions(pagesize=10)
for q in questions:
    print "question retrieved:", q.title.encode('ascii', 'ignore'), "\t", q.id
    ...

As you can see I am displaying the title and id attributes of each question object.

My question has 2 parts:

  1. How can I get a list of attributes to each method (such as recent_questions)? I simply guessed correctly that id would work, but it does not seem to be listed when I refer to >>> help(stackexchange)
  2. I would like to extend my script to mark as 'starred' ('favourite') some questions that grab my interest. Is this a reasonable prospect? Is there a method/attribute for this, and in which class? Most importantly, is authentication even possible via the API? This would presumably be necessary in order to 'star' questions.
share|improve this answer
    
Currently, keyword arguments are effectively passed verbatim to the API (albeit with a bit of processing to make them work in a URL). The best documentation I can offer in that regard are the official API docs. Regarding your second question: unfortunately, the current version of the API (v1.1) and the next planned version (v2.0) are read-only; you won't be able to star questions, although I think v2.0 will provide authentication functionality. –  Lucas Jones Jan 7 '12 at 0:34

I was willing yo use your lib to do some statistics on the questions from SO (just for fun, nothing really serious, though it can become serious if I like the results :P).

For this I'd have to get the data from I whole bunch of questions. As an example, I tried to get all questions from roughly 1 month (2e6 seconds). I tried this:

import time
from numpy import *
from stackexchange import *
from stackauth import *

currentDate = floor(time.time())
aWhileAgo = currentDate - 2000000

print currentDate, aWhileAgo


stack_auth = StackAuth()
so = Site(StackOverflow)

questions = list(so.questions(sort=Sort.Creation, order=DESC,\
                    fromdate=aWhileAgo, todate=currentDate) )
for q in questions:
    print q

But this gives me just around 20 questions... is this a limitation of the API? Is there a way around this?

share|improve this answer
    
Unfortunately, about all you can do is pass pagesize=100 to Site.questions. You can then use the methods on the returned resultset to advance through the pages. You won't be able to get anything bigger than 100 at a time, though. There are a couple of other options: you could try and export results from data.stackexchange.com, but those will probably be size-limited too. 'Worst case', you could download the data dump, but it's rather large (multi-gigabyte range) and hard to deal with. –  Lucas Jones Jun 9 '11 at 23:38

Hi Lucas,

I'm running into an issue finding a user's accept rate. I hope to find it across all the stack exchange sites, but when I pull ii.get_user().unaccepted_questions, it is always an empty list.

    accounts = STACK_AUTH.associated(site, owner_id)
    for ii in accounts:
       questions, unaccepted_questions, site = \
            ii.get_user().questions.count,  \
            ii.get_user().unaccepted_questions.count,  \
            ii.on_site.name

Am I doing something wrong?

share|improve this answer
    
I'll double-check this, but unaccepted_questions is probably one of the fields that is not fetched automatically; instead use ii.get_user().unaccepted_questions.fetch().count. I know this is a little clunky, but otherwise it would need to pull in tonnes of data each request. –  Lucas Jones May 25 '11 at 20:37

I'm running a program using py-stackexchange that gets an error,

HTTPError: HTTP Error 500: Internal Server Error

Do you have any debugging suggestions? If I could turn on a debugging flag, that would cause py-SE to print the exact URL that was requested, that would help... Then I could visit that URL myself in the browser and see if a more detailed error message was shown, such as "your parameter X was invalid" or "you've exceeded your API key limit" or something.

I could modify __init__.py myself to print such debugging messages, but I'm not up on how to recompile python functions within an egg and redeploy them.

Thanks...

Update:

P.S. It wasn't my intention to ask you to find the actual problem, but heck, I'd be just as happy to have that answer as to have the debugging tool described above, and it might be easier to supply. So here's my code:

currentDate = floor(time.time())
aWhileAgo = currentDate - 10

# E.g. fromdate=1293840000&todate=1294444800

questions = so.questions(sort=Sort.Creation, order=DESC, fromdate=aWhileAgo, \
    todate=currentDate)

The last line is where I get the 500 error.

But calling this URL directly (which is what I think py-SE should be using under the hood):

http://api.stackoverflow.com/1.1/questions?fromdate=1305580368&order=desc&sort=creation&todate=1305580478

is successful.

P.P.S.

If I take away the fromdate and todate keywords, it runs successfully:

questions = so.questions(sort=Sort.Creation, order=DESC)

So I guess the problem has to do with those keywords. Probably I'm using Py-SE incorrectly (as opposed to trying to get the SE API to do something it's not designed to do).

SOLVED.

I figured it out: I'm supposed to pass from_date and to_date, not fromdate and todate, despite the SE API parameter names.

I figured that out by guessing. Next, I would like to know how I was supposed to find that out? Naming conventions? If so, where are they documented? Just by example?

share|improve this answer
1  
Hello; sorry for the delay, glad you solved your problem. To answer your first question, you can turn on debug printing of URLs by setting stackexchange.web.WebRequestManager.debug = True. I agree that the documentation in that area is lacking; a wiki page is coming up. I'll look into that inconsistency; it seems quite pointless... I apologise on behalf of my younger self. –  Lucas Jones May 18 '11 at 15:05
    
While I'm here, I'll try putting in some more descriptive exceptions, too. –  Lucas Jones May 18 '11 at 15:13
    
Doing some testing... strange: with from_date=aWhileAgo and to_date=currentDate it works, as it does with fromdate=int(aWhileAgo) and todate. But using a float as fromdate (as aWhileAgo was), which is formatted with a trailing .0 fails with error 500. So both work with integers; this might be an API inconsistency. –  Lucas Jones May 18 '11 at 15:21
    
@Lucas, thanks for your response. That helps. –  LarsH May 18 '11 at 15:40
    
Ahem. Revisited this, doing browser testing. The reason from_date and to_date work is that they are ignored. Compare the total field on api.stackoverflow.com/1.1/… to api.stackoverflow.com/1.1/…. Either way, you can pass in a float if you want now; in the latest revision it'll be appropriately converted behind-the-scenes. –  Lucas Jones Jun 10 '11 at 0:46
    
@Lucas: thanks for the response. –  LarsH Jun 10 '11 at 1:00

I'm building a Django app, and I'l like to be able to search for users' SO account based on their name. I've got the following, which works from a python shell, but this doesn't work when called from within a Django view. Any ideas?

def so_user_search(full_name):
    so = Site(StackOverflow, MY_API_KEY)
    results = so.users([], **{'filter':full_name})
    results = results.fetch()

    result_list = []
    for item in results:
        d = { 
            'id':item.id, 
            'display_name':item.display_name, 
            'url':item.url, 
            'website':item.website_url, 
            'email_hash':item.email_hash
        }   
        result_list.append(d)
    return result_list
share|improve this answer
    
Sorry for the delay - OpenID/hosting problems... Anyway, doing some testing, I noticed that Django view parameters are Unicode strings. I hadn't thought to have handled Unicode explicitly in the library, and it turned out that the urllib module encodes it as UTF-16 by default, which the API (understandably) doesn't like. I've changed it to UTF-8, which now works under Django for me. Summary: update to the latest version in the Git repo. (Or, I can push a new release of the egg out if necessary.) –  Lucas Jones May 4 '11 at 21:00

I was wondering I am using this api in web application. Will I hit the limit? or can I have each user authenticate and be limit less. Since the app if successful will be making alot of requests.

share|improve this answer
    
You could well hit the limit if you get lots of hits. See stackapps.com/questions/67/how-api-keys-work for the official line. If your quota is increased, you won't need to change any code except to make sure you're using your API key. –  Lucas Jones Apr 30 '11 at 10:58
    
@Lucas so there is no way for each request to be done per user, as supposed to the API Key for the app itself. so I dont have to worry about the limit since I am just displaying data that my app request on the users behalf. –  garbagecollector May 3 '11 at 15:46
    
Not as far as I know, unless you use client-side JavaScript or something like that to fetch the result. You'll probably be fine with a single key, with an increased quota if necessary. The SO team are very approachable ;). –  Lucas Jones May 4 '11 at 20:57

Possible bug in the latest build? Found this (excellent) library via a Bryce Boe post and dropped in Py-Stackoverflow via easy_install.

Running the script triggered the following, however:

sog@bishop:~/Dropbox/Code/StackOverflow$ ./minestack.py 
Traceback (most recent call last):
  File "./minestack.py", line 3, in <module>
    from stackexchange import Site, StackOverflow
  File "/usr/local/lib/python2.6/dist-packages/py_stackexchange-1.1-py2.6.egg/stackexchange.py", line 6, in <module>
    from stacksites import *

As it's a problem with the library, I thought that pulling it down from GitHub and recompiling might solve the problem. Running the build.sh script in the repo, however, eventually returned this:

[...]
  File "/usr/lib/python2.6/httplib.py", line 904, in endheaders
    self._send_output()
  File "/usr/lib/python2.6/httplib.py", line 776, in _send_output
    self.send(msg)
  File "/usr/lib/python2.6/httplib.py", line 735, in send
    self.connect()
  File "/usr/lib/python2.6/httplib.py", line 716, in connect
    self.timeout)
RuntimeError: maximum recursion depth exceeded

Any ideas as to where I went wrong?

share|improve this answer
    
Aha. I should probably rename build.sh to release.sh - it's the script I use for publishing new releases to PyPI. I'll write a README or something too... anyway, it should just work, from Github, out-of-the-box without running the script. If it doesn't, let me know! :) –  Lucas Jones Feb 23 '11 at 22:07
    
Updated version on Github, and the egg should work now. –  Lucas Jones Feb 23 '11 at 23:50

There's a bug with deleted users. Attempting to call question.comments.fetch() on a question that has comments by a deleted users, such as this, results in:

  File "stackcloud.py", line 60, in <module>
    for comment in question.comments.fetch():
  File "stackcore.py", line 151, in fetch
    res = self.site.build(self.url, self.m_type, self.collection, kw)
  File "stackexchange.py", line 341, in build
    return JSONMangler.json_to_resultset(self, json, typ, collection, (self, url, typ, collection, kw))
  File "stackcore.py", line 188, in json_to_resultset
    return cls.paginated_to_resultset(site, json, typ, collection, params)
  File "stackcore.py", line 174, in paginated_to_resultset
    items.append(typ(json_item, site))
  File "stackcore.py", line 19, in __init__
    self._extend(self.json_ob, site)
  File "stackexchange.py", line 105, in _extend
    self.owner_id = json.owner['owner_id'] if 'owner_id' in json.owner else json.owner['user_id']
AttributeError: DictObject instance has no attribute 'owner'

Here's a fix proposal.

share|improve this answer
    
Thanks! I really should have tested that... thanks again, though. I'll incorporate that just now. :) –  Lucas Jones Jan 20 '11 at 18:16
1  
Committed and pushed! –  Lucas Jones Jan 20 '11 at 19:16

Quick bug:

Doing this:

anss = so.answers(user_id=175645, pagesize=100, body=True)
ans1 = anss[0]
ans1.owner

Produces this stacktrace:

Traceback (most recent call last):
  File "<pyshell#46>", line 1, in <module>
    a.owner
  File "E:\Projects\Web\Stack2Blog\repos\stack2blog-src-main\stack2blog\stack2blogapp\stackexchange.py", line 218, in <lambda>
    owner = property(lambda self: self._owner if self._owner is not None else self._get_user(self.owner_id), _set_user)
  File "E:\Projects\Web\Stack2Blog\repos\stack2blog-src-main\stack2blog\stack2blogapp\stackexchange.py", line 207, in _get_user
    s._owner = self.site.user(id)
NameError: global name 'self' is not defined
share|improve this answer
    
D'oh. Fixing... –  Lucas Jones Jun 16 '10 at 15:04
1  
Done in latest revision. –  Lucas Jones Jun 16 '10 at 15:05

Another quick question:

Is there any way to get a user's answers without going through the user object? I want to save a fetch of a user's answers, and since I already have his id, I could go to: http://api.stackoverflow.com/0.8/users/id/answers?body=true&pagesize=100.

I'm just not sure how to do such a thing using the wrapper.

If it doesn't exist, it would probably make sense to be able to do:

# so is my stackexchange object
so.answers(userid=id, other_keywords...)

Also, for bonus points, when getting a user's answers, you get back info on the user as well (e.g. you get the user's display name). So such a call could automatically create the user object that's linked to each answer.

share|improve this answer
    
OK, that seems reasonable. And those bonus points should be attainable quite easily! :) –  Lucas Jones Jun 15 '10 at 15:01
1  
Finished in latest revision. –  Lucas Jones Jun 15 '10 at 17:17
    
Looking good so far. One quick question: right now, doing site.answers(user_id=some_id_that_does_not_exist) just gives back an empty list. I don't know about this, but would it be better to raise a "User does not exist" exception? You could also raise this in other places that have the same "problem". Not sure if this would actually be better, just an idea. –  Edan Maor Jun 16 '10 at 6:50
    
I was thinking about that when I accidentally requested page 2 of what I expected to be a multi-page list (it was only 14 items!). Some distinction could be useful... –  Lucas Jones Jun 16 '10 at 15:07
    
Note to future readers: a nicer API is supported in v1.1: site.questions.by_user(uid). –  Lucas Jones May 18 '11 at 15:27

Feature Request

A little feature I think could come in handy.

Right now, there is no way to get the url of a Question/Answer from the Answer object (I'm talking about the actual URL on the site, e.g. stackoverflow.com/answers/id). This is also not returned by the api itself (see this answer).

Ideally, I think the wrapper should include a method that builds the url for you, i.e. the Answer object will have a getUrl method which will build up the url based on which site you're querying.

If not, another good idea would be to provide the Site object with a method that gets the url of the site. When you build a Site object you send in a constant like "api.stackoverflow.com", so the object should have a method which strips out the api part.

share|improve this answer
    
That seems like a good idea on both counts! Will have a look at this. –  Lucas Jones Jun 14 '10 at 18:12
    
Done in latest revision. site.root_domain and {user|question|answer}.url. Beware that this is hard-coded and relatively hacky! :) –  Lucas Jones Jun 14 '10 at 18:50
1  
Hacky is my middle name. :) –  Edan Maor Jun 14 '10 at 21:31

OK, I'm probably missing the obvious, but how do I get the body of an answer?

For example, in your code snippet:

>>> so = stackexchange.Site('api.stackoverflow.com')
>>> me = so.user(41981)
>>> me.answers.fetch()
>>> me.answers[0].body
Traceback (most recent call last):
  File "<pyshell#45>", line 1, in <module>
    me.answers[0].body
AttributeError: 'Answer' object has no attribute 'body'
share|improve this answer
1  
Ah, sorry. Due to a change in the API (I think - or perhaps me just being stupid and not reading the docs close enough!) question and answer bodies need to be explicitly requested. You can do this through so.be_inclusive(), which you should call before your first request. –  Lucas Jones Jun 13 '10 at 15:36
    
Yeah just figured out the be_inclusive() bit. Is there any way to specify it per-request? (I assume most people won't want to get all the bodies and all the comments from every request, just from specific requests). I can see that there are places where you check for a "body" keyword param, but I'm not sure where I can send one in the snippet above... –  Edan Maor Jun 13 '10 at 15:38
    
I've also updated the FAQ with this, as it's not documented very clearly. –  Lucas Jones Jun 13 '10 at 15:57
    
@Edan: Not in your specific case right now (see the FAQ - link in question - for those that are covered), as I'm not sure how best to implement it. Do you think calling me.answers.fetch(body='true') would be best? –  Lucas Jones Jun 13 '10 at 15:59
    
I think so. It's the only use case I've actually run into. –  Edan Maor Jun 13 '10 at 16:07
    
Right. I have an idea how I'd do this! :) –  Lucas Jones Jun 14 '10 at 18:13
    
Is this done in the latest version? If so, do I use it like your wrote in your comment? –  Edan Maor Jun 15 '10 at 13:44
    
@Edan: Not quite, but it should be done soon. –  Lucas Jones Jun 15 '10 at 16:48
1  
@Edan: Done now in the latest revision, with the same syntax. fetch_page (but not other overloads right now) can do the same, too. –  Lucas Jones Jun 15 '10 at 17:18
    
@Edan: Darn comment edit time limits. It's actually user_id not userid in the library code. –  Lucas Jones Jun 15 '10 at 17:24

Two of the demo scripts currently do not work because there is no 'answers' field in the json :(

It might be an ugly workaround, but for now this diff does the trick:

badp@delta:~/Py-StackExchange$ git diff 0a5c60f71310757a4c1de6547cf113c7d6834e85
diff --git a/stackexchange.py b/stackexchange.py
index 959cf3e..4a17b10 100644
--- a/stackexchange.py
+++ b/stackexchange.py
@@ -492,10 +492,12 @@ unlike on the actual site, you will receive an error rathe

        def questions(self, ids, **kw):
                """Retrieves a set of the comments with the IDs specified in the
+               kw["answers"] = "true"
                return self._get(Question, ids, 'questions', kw)

        def recent_questions(self, **kw):
                """Returns the set of the most recent questions on the site, by 
+               kw["answers"] = "true"
                return self.build('questions', Question, 'questions', kw)

        def users_with_badge(self, bid, **kw):

However, this fails to take account of the good points behind the change.

share|improve this answer
    
Thanks - I'll add the diff for now, to keep compatibility. I may factor it into the be_inclusive() method later, though. –  Lucas Jones Jun 12 '10 at 21:43
    
I've added the patch manually (it's only small :D), but I can't figure out how to do it with Git - git apply didn't work. Just for future reference, do you know how I'd do that? –  Lucas Jones Jun 12 '10 at 21:47
    
Done in the latest revision. –  Lucas Jones Jun 12 '10 at 21:57

Edit:

My problem below has been "solved", and it's a problem with the fact that I'm getting back gzip-compressed data from stackapps. See the SO Answer. I'm still unsure why this happens only on my computer (possible reason: routers in my network adding content-headers), but I'm guessing this should be fixed in the wrapper itself.

Come to think of it, the wrapper should probably be requesting gzip-compress data in the first place, to save download time.

Original question

I'm having problems using the library. I'm doing the following:

site = stackexchange.Site(stackexchange.StackOverflow)
user = site.user(userid)

And I'm getting the following exception:

ValueError at /answers
No JSON object could be decoded

After some debugging, I see that the line which throws the error is: dump = json.load(conn) (around line 410). When I try printing the urllib object I get back (conn), I get the following:

\x1f\x8b\x08\x00\x00\x00\x00\x00\x04\x00\xed\xbd\x07`\x1cI\x96%&/m\xca{\x7fJ\xf5J\xd7\xe0t\xa1\x08\x80`\x ...

I don't have any experience with urllib, so I'm not sure if that's a good output, although it doesn't look right to me (by contrast, opening http://www.google.com gives back actual meaningful text).

Any ideas as to what the problem could be?

Edit: Just running the code snippet in your question in IDLE is causing a problem.

share|improve this answer
    
About gzip compression, I'll have a look. I assume it's not built-in to urllib2, but I'll take a look. –  Lucas Jones Jun 12 '10 at 21:41
    
In my to-do list now! wiki.github.com/lucjon/Py-StackExchange :) –  Lucas Jones Jun 12 '10 at 22:00
    
If it helps, I used this site to understand how to work with the gzip compressed responses: diveintopython.org/http_web_services/gzip_compression.html. –  Edan Maor Jun 13 '10 at 3:45
    
Thanks for that! I'll have a look at implementing that today. –  Lucas Jones Jun 13 '10 at 14:35
1  
Done in latest revision. –  Lucas Jones Jun 13 '10 at 15:21

Using the most recent version of the library as of yesterday. I needed to get some information on users answers and the question itself. However this didn't work. Every time I tried to access another page of answers I got ()

answers = me.answers.fetch()

answerDetails = []

while True:
    for answer in answers:
        answerDetails.append({
            'question_id'       :answer.question_id,
            'up_vote_count'     :answer.up_vote_count,
            'down_vote_count'   :answer.down_vote_count,
            'accepted'          :answer.accepted
            'tags'              :so.question(answer['question_id']).tags
        })

    answers = answers.fetch_next() #returns ()
    if answers == None or len(answers) == 0:
        break

I had to move the requests to get the tags outside of the loop that got the answers:

#[code cut]

    for answer in answers:
        answerDetails.append({
            'question_id'       :answer.question_id,
            'up_vote_count'     :answer.up_vote_count,
            'down_vote_count'   :answer.down_vote_count,
            'accepted'          :answer.accepted
            #'tags'              :so.question(answer['question_id']).tags
        })

#[code cut]

for answer in answerDetails:
    answer['tags'] = so.question(answer['question_id']).tags

While the latter is for various reasons a better solution, it was quite a surprise that it didn't do what I expected.

A better code sample to reproduce is below:

Doesn't work as expected:

so = stackexchange.Site(stackexchange.StackOverflow)
me = so.user(118145)
answers = me.answers.fetch()
so.question(answers[0].question_id).tags
print answers.fetch_next() #prints ()

Works as expected:

so = stackexchange.Site(stackexchange.StackOverflow)
me = so.user(118145)
answers = me.answers.fetch()
print answers.fetch_next() #prints the correct array
share|improve this answer
    
That is very weird. I have added it to my quick-and-dirty to-do list wiki.github.com/lucjon/Py-StackExchange. –  Lucas Jones Jun 12 '10 at 21:58

I am using Py-StackExchange in our Stack2Blog application.

How do I retrieve a list of answers of a specific user using this wrapper?

After retrieving a user object, the .answers member variable returns an empty list while I know the user does have posts.

share|improve this answer
    
Managed to figure it out. The list is populated only after the list's .fetch() method is called. –  Yuval Cohen Jun 11 '10 at 15:52
    
Yes, sorry - this is by-design to limit the number of API calls the app makes. That looks like an awesome app, though! –  Lucas Jones Jun 12 '10 at 21:41

You can get around the json/simplejson easily in your wrapper. Just change your import line from

import urllib2, json, httplib, datetime, operator

to

import urllib2, httplib, datetime, operator
try:
    import json
except ImportError:
    import simplejson as json

That way you can just use json in the rest of your code, but client pythons will load whichever library is available on that system.

Keep up the good work, I'll probably be using your library assuming I can dream up something cool to do with it.

share|improve this answer
    
Thanks! I'll fix that as soon as I can. –  Lucas Jones May 27 '10 at 15:47
    
This is done in the latest revision. –  Lucas Jones May 28 '10 at 16:06
    
Fyi, this doesn't work in Python 2.5. Since I'm using Django, I modified the above to try and import from django as well (from django.utils import simplejson as json). I don't know if you should add this to the wrapper or not, but if anyone is using Django, just use that line instead. –  Edan Maor Jun 16 '10 at 8:08
    
@Edan: I think it might be too specialised a case, but I'll definitely include it in the FAQ. –  Lucas Jones Jun 16 '10 at 15:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .