Python的urllib2、cookielib模块

• 没有验证码的情况

比如登录人人网，在前几次的时候不需要输入验证码，可以先将用户名、密码进行urllib.urlencode编码，用cookielib.CookieJar()生成cookie，然后生成opener=urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))，之后登陆页面req = opener.open(login_page, post_data)就可以了；实际示例如下：

import urllib, urllib2, cookielib, re
def login_func():
    login_page = "http://www.renren.com/ajaxLogin/login"
    data = {'email': 'your_email', 'password': 'your_password'}
    post_data = urllib.urlencode(data)
    cj = cookielib.CookieJar()
    opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
    urllib2.install_opener(opener)
    print u"登录人人网"
    req = opener.open(login_page, post_data)
    req = urllib2.urlopen("http://www.renren.com/home")
    html = req.read()
    uid = re.search("'ruid':'(d+)'", html).group(1)
    print  u"登陆成功"
    return uid

import urllib2
import urllib
import cookielib

data = {"email":"your_mail", "password":"your_passwd"}
post_data = urllib.urlencode(data)
cj = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
req = urllib2.Request("http://www.renren.com/PLogin.do", post_data)
content = opener.open(req)

 

• 如何绕过验证码

可以在手动登录了之后用控制台或Fiddler抓取cookie信息，然后添加至header（有2种方式）：

Use the headers argument to the Request constructor, or:

import urllib2
req = urllib2.Request('http://www.example.com/')
req.add_header('Referer', 'http://www.python.org/')
r = urllib2.urlopen(req)

OpenerDirector automatically adds a User-Agent header to every Request. To change this:

import urllib2
opener = urllib2.build_opener()
opener.addheaders = [('User-agent', 'Mozilla/5.0')]
opener.open('http://www.example.com/')

Also, remember that a few standard headers (Content-Length, Content-Type and Host) are added when the Request is passed to urlopen() (or OpenerDirector.open()).

后面的操作就和平时差不多了。

参考：

• 20.6. urllib2 — extensible library for opening URLs — Python 2.7.8 documentation
• python: urllib2 how to send cookie with urlopen request – Stack Overflow

 

其它

• http://cn.bing.com/search?q=python+%E5%B7%B2%E6%9C%89cookie+%E7%99%BB%E5%BD%95
• http://search.aol.com/aol/search?q=Python+existing+cookie+login+web
• http://search.aol.com/aol/search?q=Python+cookieJar
• https://docs.python.org/2/library/urllib2.html
• https://docs.python.org/2/library/cookielib.html
• http://stackoverflow.com/questions/6878418/putting-a-cookie-in-a-cookiejar
• http://stackoverflow.com/questions/2169281/how-to-add-cookie-to-existing-cookielib-cookiejar-instance-in-python

 

总结下来就是：

因为cookie就是HTTP Header的一部分，所以直接添加到Header中就行，和添加UserAgent一样。