首页 > 代码库 > python 解析html基础 HTMLParser库,方法,及代码实例

python 解析html基础 HTMLParser库,方法,及代码实例

2024-07-06 13:26:54   217人阅读

             HTMLParser, a simple lib as html/xhtml parser

             官方解释:

             This module defines a class HTMLParser which serves as the basis for parsing text files formatted in HTML (HyperText Mark-up Language) and XHTML.Unlike the parser inhtmllib, this parser is not based on the SGML parser insgmllib.

class HTMLParser.HTMLParser

An HTMLParser instance is fed HTML data and calls handler methodswhen start tags, end tags, text, comments, and other markup elements areencountered. The user should subclass HTMLParser and override its methods to implement the desired behavior.

The HTMLParser class is instantiated without arguments.

Unlike the parser in htmllib, this parser does not check that end tags match start tags or call the end-tag handler for elements which are closedimplicitly by closing an outer element.

An exception is defined as well:

exceptionHTMLParser.HTMLParseError

HTMLParser is able to handle broken markup, but in some cases it might raise this exception when it encounters an error while parsing.This exception provides three attributes: msg is a briefmessage explaining the error, lineno is the number of the line onwhich the broken construct was detected, andoffset is the number ofcharacters into the line at which the construct starts.

            下面仅就重点进行解释，与html解析的pythonlib除了HTMLParser，还有htmllib，htmllib不同于HTMLParser的一点是htmllib基于SGMLParser
#!/usr/bin/python
#a simple htmlparser demo

from HTMLParser import HTMLParser

class SimpleParser(HTMLParser):
    def handle_starttag(self,tag,attr): print tag
    def handle_endtag(self,tag): print tag
    def handle_data(self,data): print data
simpleparser = SimpleParser()
simpleparser.feed("<title></title>")

这是一个很简单的例子，但是HTMLParser的基本原理也就是这样，当然，为了更好的完成工作我们还要处理一些非标准的html，因为现在浏览器的
发展是在不断迎合识别各种非标准的html，所以造成了今天我们对html的解析变得异常复杂，我们会看到<title>...然后没有终止标记这样的非标准格式，
当然，像xhtml这种语言是比较严格的，不会允许这种情况的出现，下面给出一个在《python基础网络编程》的例子，这个例子中除了处理这种不标准

的html标记外，还同时用

HTMLParser.handle_entityref(name)

This method is called to process a named character reference of the form&name; (e.g.&gt;), where name is a general entity reference(e.g.‘gt‘).

处理特殊符号，因为在过滤中会自动把这一类的符号给过滤掉，不能转义也不能作为字面值出现:

下面是代码:

#!/usr/bin/env python
from HTMLParser import HTMLParser
#handle entity like &
from htmlentitydefs import entitydefs
import sys

class TitleParser(HTMLParser):
	def __init__(self):
		self.taglevels = []                #处理title ul li 只有开始无结束标记的情况                self.handledtags = ['title', 'ul', 'li']
		self.processing = None
		HTMLParser.__init__(self)
	
	def handle_starttag(self, tag, attrs):
		if len(self.taglevels) and self.taglevels[-1] == tag:
			self.handle_endtag(tag)
		
		self.taglevels.append(tag)
		if tag in self.handledtags:
			self.data = http://www.mamicode.com/''>


联系
我们