html5lib is a pure-python library for parsing HTML. It is designed to conform to the WHATWG HTML specification, as is implemented by all major web browsers.
Simple usage follows this pattern:
import html5lib
with open("mydocument.html", "rb") as f:
document = html5lib.parse(f)or:
import html5lib
document = html5lib.parse("<p>Hello World!")By default, the document will be an xml.etree element instance.
Whenever possible, html5lib chooses the accelerated ElementTree
implementation (i.e. xml.etree.cElementTree on Python 2.x).
Two other tree types are supported: xml.dom.minidom and
lxml.etree. To use an alternative format, specify the name of
a treebuilder:
import html5lib
with open("mydocument.html", "rb") as f:
lxml_etree_document = html5lib.parse(f, treebuilder="lxml")When using with urllib2 (Python 2), the charset from HTTP should be
pass into html5lib as follows:
from contextlib import closing
from urllib2 import urlopen
import html5lib
with closing(urlopen("http://example.com/")) as f:
document = html5lib.parse(f, transport_encoding=f.info().getparam(