A cleanup tool for HTML.
Removes unwanted tags and content. See the Cleaner class for details.
||Turn any URLs into links.|
||Breaks any long words found in the body of the text (not attributes).|
Instances cleans the document of each of the possible offending elements. The cleaning is controlled by attributes; you can override attributes in a subclass, or set them in the constructor.
- Removes any
- Removes any comments.
- Removes any style tags.
- Removes any style attributes. Defaults to the value of the
- Removes any
- Removes any
- Structural parts of a page:
- Removes any processing instructions.
- Removes any embedded objects (flash, iframes)
- Removes any frame-related tags
- Removes any form tags
- Tags that aren’t wrong, but are annoying.
- A list of tags to remove. Only the tags will be removed, their content will get pulled up into the parent tag.
- A list of tags to kill. Killing also removes the tag’s content, i.e. the whole subtree, not just the tag itself.
- A list of tags to include (default include all).
- Remove any tags that aren’t standard parts of HTML.
- If true, only include ‘safe’ attributes (specifically the list from the feedparser HTML sanitisation web site).
- A set of attribute names to override the default list of attributes considered ‘safe’ (when safe_attrs_only=True).
- If true, then any <a> tags will have
rel="nofollow"added to them.
A list or set of hosts that you can use for embedded content (for content like
<link rel="stylesheet">, etc). You can also implement/override the method
allow_element(el)to implement more complex rules for what can be embedded. Anything that passes this test will be shown, regardless of the value of (for instance)
Note that this parameter might not work as intended if you do not make the links absolute before doing the cleaning.
Note that you may also need to set
- A set of tags that can be included with
host_whitelist. The default is
embed; you may wish to include other tags like
script, or you may want to implement
allow_embedded_urlfor more control. Set to None to include all tags.
This modifies the document in place.
Cleans the document.
Override to suppress rel=”nofollow” on some anchors.
IE conditional comments basically embed HTML that the parser doesn’t normally see. We can’t allow anything like that, so we’ll kill any comments that could be conditional.
_kill_elements(doc, condition, iterate=None)¶
Depending on the browser, stuff like
e x p r e s s i o n(...)can get interpreted, or
expre/* stuff */ssion(...). This checks for attempt to do stuff like this.
Turn any URLs into links.
It will search for links identified by the given regular expressions (by default mailto and http(s) links).
It won’t link text in an element in avoid_elements, or an element with a class in avoid_classes. It won’t link to anything with a host that matches one of the regular expressions in avoid_hosts (default localhost and 127.0.0.1).
If you pass in an element, the element’s tail will not be substituted, only the contents of the element.
word_break(el, max_width=40, avoid_elements=_avoid_word_break_elements, avoid_classes=_avoid_word_break_classes, break_character=unichr)¶
Breaks any long words found in the body of the text (not attributes).
Doesn’t effect any of the tags in avoid_elements, by default
Breaks words by inserting ​, which is a unicode character for Zero Width Space character. This generally takes up no space in rendering, but does copy as a space, and in monospace contexts usually takes up space.
See http://www.cs.tut.fi/~jkorpela/html/nobr.html for a discussion
word_break_html(html, *args, **kw)¶
_break_text(text, max_width, break_character)¶
_insert_break(word, width, break_character)¶