4

I am working on an desktop inbox notifier for StackOverflow, using the API with Python.

The script I am working on first logs the user in on StackExchange, and then requests authorisation for the application.

Assuming the application has been authorised through web-browser interaction of the user. The application should be able to make requests to the API with authentication, hence it needs the access token specific to the user. This is done with the URL: https://stackexchange.com/oauth/dialog?client_id=54&scope=read_inbox&redirect_uri=https://stackexchange.com/oauth/login_success

When requesting authorisation via the web-browser the redirect is taking place and an access code is returned after a #. However, when requesting this same URL with Python (urllib2), no hash or key is returned in the response.

Why is this and how do I get it to work?

Here is my script (it's experimental) and remember: it assumes the user has already authorised the application.

#!/usr/bin/env python

import urllib
import urllib2
import cookielib
from BeautifulSoup import BeautifulSoup
from getpass import getpass    

# Define URLs
parameters = [ 'client_id=54',
               'scope=read_inbox',
               'redirect_uri=https://stackexchange.com/oauth/login_success'
             ]

dialog_url = 'https://stackexchange.com/oauth/dialog?' + '&'.join(parameters)
login_url = 'https://openid.stackexchange.com/account/login'
submit_url = 'https://openid.stackexchange.com/account/login/submit'
authentication_url = 'http://stackexchange.com/users/authenticate?openid_identifier='

# Set counter for requests:
counter = 0

# Build opener
jar = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(jar))


def authenticate(username='', password=''):

    '''
        Authenticates to StackExchange using user-provided username and password
    '''

    # Build up headers
    user_agent = 'Mozilla/5.0 (Ubuntu; X11; Linux i686; rv:8.0) Gecko/20100101 Firefox/8.0'
    headers = {'User-Agent' : user_agent}

    # Set Data to None
    data = None

    # 1. Build up URL request with headers and data    
    request = urllib2.Request(login_url, data, headers)
    response = opener.open(request)

    # Build up POST data for authentication
    html = response.read()
    fkey = BeautifulSoup(html).findAll(attrs={'name' : 'fkey'})[0].get('value').encode()

    values = {'email' : username,
              'password' : password,
              'fkey' : fkey}

    data = urllib.urlencode(values)

    # 2. Build up URL for authentication
    request = urllib2.Request(submit_url, data, headers)
    response = opener.open(request)


    # Check if logged in
    if response.url == 'https://openid.stackexchange.com/user':
        print ' Logged in! :) '
    else:
        print ' Login failed! :( '

    # Find user ID URL    
    html = response.read()
    id_url = BeautifulSoup(html).findAll('code')[0].text.split('"')[-2].encode()


    # 3. Build up URL for OpenID authentication
    data = None
    url = authentication_url + urllib.quote_plus(id_url)
    request = urllib2.Request(url, data, headers)
    response = opener.open(request)

    # 4. Build up URL request with headers and data
    request = urllib2.Request(dialog_url, data, headers)
    response = opener.open(request)

    if '#' in response.url:
        print 'Access code provided in URL.'
    else:
        print 'No access code provided in URL.'


if __name__ == '__main__':
    username = raw_input('Enter your username: ')
    password = getpass('Enter your password: ')
    authenticate(username, password)
8
  • You shouldn't be getting the users user name and password yourself. Not all users are registered with StackExchange, some use Google, Facebook etc (I dont know python so I'm not 100% sure exactly what your doing). just show the dialog to the user and they will log themselves. It's also a security issue as the user doesn't know that you're not keeping the password and doing bad stuff with it. Oauth means you never have to store the username and password, and you don't even need to store the access token persistently.
    – Jonathan.
    Jan 6, 2012 at 13:31
  • @Jonathan: you got me really confused here. In order to authorise an application, one needs to be logged in. What the script does is log in the user first--doing through the application's interface what one would do through the browser. Once one is logged in, the OAuth requests is made with the URL quoted above. I am also aware the user can register using different openID providers, the above is a draft using SE login only, but the final script would allow other IDs too. Jan 8, 2012 at 14:32
  • What I mean is don't log the user in through the application's interface. Just show the user the browser, and let them log themselves in. The login dialog is subject to change, there may be more login services added, or possible removed and the structure of it may change.
    – Jonathan.
    Jan 8, 2012 at 15:21
  • @Jonathan: OK. It doesn't solve the initial problem though. Besides, if i'd proceed through the web-browser, I'd not be able to retrieve the access token at all, would I? Jan 8, 2012 at 15:34
  • With python isn't there a way of showing a web page inside one of your application's window? You wouldn't show the user the dialog in the users default web browser. You can then monitor that browser control for when it loads a page then get the URL that was loaded and extract the hash. My point was meant to be that the dialog is not meant to be interacted with by the application, only the user, so it maybe for that reason that the hash is being lost.
    – Jonathan.
    Jan 8, 2012 at 15:52
  • Use something like qtWebkit to show the login dialog: rkblog.rk.edu.pl/w/p/webkit-pyqt-rendering-web-pages
    – Jonathan.
    Jan 8, 2012 at 16:04
  • @Jonathan: I understand your suggestion and it makes sense. On the other hand, Firefox is also an application with an interface like any other. What matters here is the handling the HTTP requests, not so much the rendering of HTML (take for instance the w3m text-browser). So although I consider your advice, my point is to understand these processes and reproduce them programmatically, so embedding a ready-made browser wouldn't make much sense to me in this context. Jan 8, 2012 at 16:08
  • With a browser control inside your application you would be able to retrieve the access token, with Firefox you can't. What your doing is unsupported so it not passing the access_token is probably because of that. If you want to get the access_token you have to let the user interact with the dialog directly.
    – Jonathan.
    Jan 8, 2012 at 16:33

1 Answer 1

2

The token does not appear because of the way urllib2 handles the redirect. I am not familiar with the details so I won't elaborate here.

The solution is to catch the 302 before urllib2 handles the redirect. This can be done by sub-classing the urllib2.HTTPRedirectHandler to get the redirect with its hash and token.
Here is a short example of subclassing the handler:

class MyHTTPRedirectHandler(urllib2.HTTPRedirectHandler):
    def http_error_302(self, req, fp, code, msg, headers):
        print "Going through 302:\n"
        print headers
        return urllib2.HTTPRedirectHandler.http_error_302(self, req, fp, code, msg, headers)

In the headers the location attribute will provide the redirect URL in full length, i.e. including the hash and token:

Output extract:

...
Going through 302:

Server: nginx/0.7.65
Date: Mon, 09 Jan 2012 20:20:11 GMT
Content-Type: text/html; charset=utf-8
Connection: close
Cache-Control: private
Location: https://stackexchange.com/oauth/login_success#access_token=K4zKd*HkKw5Opx(a8t12FA))&expires=86400
Content-Length: 218
...

More on catching redirects with urllib2 on Stack Overflow (of course).

1
  • Hi, I am trying the above but I am unable to see the access token returned: I see the following:Cache-Control: private Content-Type: text/html; charset=utf-8 Location: /oauth/dialog?client_id=2858&redirect_uri=https%3a%2f%2fstackexchange.com%2foauth%2flogin_success&scope=read_inbox&response_type=token&state=&returnurl=%2foauth%2fdialog%3fclient_id%3d2858%26redirect_uri%3dhttps%253a%252f%252fstackexchange.com%252foauth%252flogin_success%26scope%3dread_inbox%26response_type%3dtoken%26state%3d X-Frame-Options: SAMEORIGIN Date: Sat, 12 Apr 2014 19:10:56 GMT Connection: close Content-Length: 464
    – vkmv
    Apr 12, 2014 at 19:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .