首页 > 代码库 > python100题目的爬取
python100题目的爬取
import requests
from bs4 import BeautifulSoup
def getHTMLText(url):
try:
r = requests.get(url)
r.raise_for_status()
r.encoding = ‘utf-8‘
return r.text
except:
return ‘‘
def fillUnivList(ulist, html):
soup = BeautifulSoup(html, ‘html.parser‘)
meta = soup.find_all(‘meta‘, attrs={‘name‘: ‘description‘})
ulist.append(meta[0].attrs[‘content‘])
def main():
start_url = ‘http://www.runoob.com/python/python-exercise-example‘
uinfo = []
for i in range(101):
url = start_url + str(i) +‘.html‘
try:
html = getHTMLText(url)
fillUnivList(uinfo, html)
except:
continue
for i in range(101):
try:
with open(‘100.txt‘, ‘a‘) as f:
f.write(uinfo[i] + ‘\n‘)
except:
continue
print(uinfo)
main()
python100题目的爬取
声明:以上内容来自用户投稿及互联网公开渠道收集整理发布,本网站不拥有所有权,未作人工编辑处理,也不承担相关法律责任,若内容有误或涉及侵权可进行投诉: 投诉/举报 工作人员会在5个工作日内联系你,一经查实,本站将立刻删除涉嫌侵权内容。