首页 > 代码库 > Python爬虫
Python爬虫
#Author:xuesong li import gevent from gevent import monkey monkey.patch_all() from urllib.request import urlopen import time def pa_web_page(url): print("GET url",url) req = urlopen(url) data =req.read() print(data) f = open("test",‘w‘) f.read(data) f.close() print(‘%d bytes received from %s.‘ % (len(data), url)) t_start = time.time() pa_web_page("http://www.autohome.com.cn/beijing/") pa_web_page("http://www.xiaohuar.com/") print("time cost:",time.time()-t_start) t2_start = time.time() gevent.joinall([ #gevent.spawn(pa_web_page, ‘https://www.python.org/‘), gevent.spawn(pa_web_page, ‘http://www.autohome.com.cn/beijing/‘), gevent.spawn(pa_web_page, ‘http://www.xiaohuar.com/‘), #gevent.spawn(pa_web_page, ‘https://github.com/‘), ]) print("time cost t2:",time.time()-t2_start)
Python爬虫
声明:以上内容来自用户投稿及互联网公开渠道收集整理发布,本网站不拥有所有权,未作人工编辑处理,也不承担相关法律责任,若内容有误或涉及侵权可进行投诉: 投诉/举报 工作人员会在5个工作日内联系你,一经查实,本站将立刻删除涉嫌侵权内容。