首页 > 代码库 > scrapy snippet
scrapy snippet
1. spider文件
from scrapy.contrib.spiders import CrawlSpider, Rulefrom scrapy.contrib.linkextractors.sgml import SgmlLinkExtractorfrom scrapy.selector import HtmlXPathSelectoritem = DomzItem()image_urls = hxs.select(‘//img/@src‘).extract()item[‘image_urls‘] = ["http:" + x for x in image_urls]return item
from scrapy.selector import HtmlXPathSelectorhxs = HtmlXPathSelector(response)
name = "wikipedia"allowed_domains = ["wikipedia.org"]start_urls = [ "http://en.wikipedia.org/wiki/Pune"]
2. setting文件
ITEM_PIPELINES = [‘scrapy.contrib.pipeline.images.ImagesPipeline‘]IMAGES_STORE= ‘...‘
3. item 文件
image_urls = Field() images = Field()
scrapy snippet
声明：以上内容来自用户投稿及互联网公开渠道收集整理发布，本网站不拥有所有权，未作人工编辑处理，也不承担相关法律责任，若内容有误或涉及侵权可进行投诉： 投诉/举报 工作人员会在5个工作日内联系你，一经查实，本站将立刻删除涉嫌侵权内容。