HTML::Manipulator - Perl extension for manipulating HTML files
use HTML::Manipulator; my $html = <<HTML; <h1 id=title>Old news</h1> <a href='http://www.google.com' id=link>Google</a>.... HTML # replace a tag content my $new = HTML::Manipulator::replace($html, title => 'New news'); # replace a tag attribute and content my $new = HTML::Manipulator::replace($html, link => { _content => 'Slashdot', href=>'http://www.slashdot.org/' } ); # extract a tag content my $content = HTML::Manipulator::extract_content($html, 'link'); # extract a tag content and attributes my $tag = HTML::Manipulator::extract($html, 'link'); # returns a hash ref like # { href => 'http://www.google.com', id => 'link', _content => 'Google' }
This module manipulates of the contents of HTML files. It can query and replace the content or attributes of any HTML tag.
The advertised usage pattern is to update static HTML files.
HTML::Manipulator is NOT yet another templating module. There are, for example, no template files. It works on normal HTML files without any special markup (you only have to give element IDs to tags you are interested in, or wrap them in comments).
While you could probably use this module for producing your web application's output, DON'T. It does not offer a lot of features for this area (no loops, no conditionals, no includes) and is not optimized for performance. Have a look at HTML::Template instead.
HTML::Manipulator is meant to work on real-life HTML files (in all their non-standards-compliant ugliness). It uses the HTML::Parser module to find elements (tags) inside those files, which you can then replace or modify. All you have to do is give those elements a DOM ID, for example
<h3 id=headline77>Headline</h3>
No other markup is necessary.
As an alternative to element ID, HTML::Manipulator can also identify sections enclosed in HTML comments.
<h3><!-- headline77 -->Headline<!-- --></h3>
HTML::Manipulator tries to cope with malformed input data. All you have to ensure is that you properly close the element you are working on (any other tags can be unbalanced) and that the IDs are unique. It will also preserve the content outside the element you asked it to operate on. It does not rewrite your HTML any more than it has to.
HTML is case insensitive in its tag and attribute names. This means that
and
<H3 iD=headline77>Headline</h3>
are treated as identical.
However, HTML::Manipulator respects case when comparing the IDs of elements (not sure about the HTML standard here), so that you could NOT address above h3 element as HeadLine77.
When HTML::Manipulator has to rewrite tags (this happens when you ask it to change element attributes) it will output the tag and attribute names as lower-case. It will also rearrange their order. When changing only the content of an element, it preserves the original opening and closing tags.
When matching HTML comments (see below), case and whitespace are ignored.
You can change the content or attributes of any HTML element with an attached ID.
my $new = HTML::Manipulator::replace($html, title => 'New news');
The function takes as input the HTML data and returns the modified data (as a long scalar).
my $new = HTML::Manipulator::replace($html, title => 'New news', headline77=>'All clear?');
You can just pass many IDs and new contents to the function as well. The caveat here is that if those elements are nested, only the outermost will be applied: The complete content of the outermost element will be replaced with the new content, eliminating any nested tags. Even if the new content contains nested elements, these will not be evaluated. No recursion today.
If you want to replace attribute values (such as a link href), you use the same function described above, but pass a hashref instead of the string with the new content:
my $new = HTML::Manipulator::replace($html, link => { href=>'http://www.slashdot.org/' } );
The hashref can contain as many key/value pairs as you want. Any attributes that you specify here will appear in the output HTML. Any attributes that you do not specify will retain their old value.
You can also change content and attributes at the same time, by adding the special "attribute" _content to the attribute hashref.
my $new = HTML::Manipulator::replace($html, link => { _content => 'Slashdot', href=>'http://www.slashdot.org/' } );
There is a family of functions to insert some text into the source HTML document before or after a given DOM element.
my $new = HTML::Manipulator::insert_before_begin($html, headline77 => '<b>', ); my $new = HTML::Manipulator::insert_after_begin($html, headline77 => '[**] ', ); my $new = HTML::Manipulator::insert_before_end($html, headline77 => '[**]', ); my $new = HTML::Manipulator::insert_after_end($html, headline77 => '</b>', );
You can set the document title (the stuff between the <title> tags) like
my $new = HTML::Manipulator::replace_title($html, 'new title');
In addition to replacing parts of the HTML document, you can also query it for the current content.
my $content = HTML::Manipulator::extract_content($html, 'link');
gives you a scalar containing the content of the tag with the ID 'link'.
my $content = HTML::Manipulator::extract_all_content($html);
gives you a hashref with all element IDs as keys and their contents as values.
my $content = HTML::Manipulator::extract($html, 'link');
gives you a hashref with information about the tag with the ID 'link'. There is a key for every attribute in the tag, and the special key '_content' which contains the content. The structure of the hashref is identical to what you would use when calling the replace function.
There is also a function to get information about all elements (that have an ID):
my $content = HTML::Manipulator::extract_all($html);
This returns a hashref of hashrefs, so that you could get the href of the "link" element like $content->{link}{href}.
You can selectively use the extract_all* functions by passing in the IDs you are interested in. This is optional. The default returns data for all elements with IDs.
my $content = HTML::Manipulator::extract_all_content ($html, 'one', 'two', 'three');
You can also mix in regular expressions. Any elements with IDs that match will be returned. This way you can also achieve case-insensitivity with IDs.
my $content = HTML::Manipulator::extract_all_content ($html, qr/^...$/i, 'two', qr/^some.*/);
You can query for a list of all element IDs and their tag type.
$data = HTML::Manipulator::extract_all_ids($html);
This returns a hashref where the element IDs of the document are they keys. The associated value is the type of the element (the tag type, such as div, span, a), which is returned as lowercase.
You can filter this in the same way as with the extract_all_content function above:
$data = HTML::Manipulator::extract_all_ids($html, qr/^...$/i, 'two', qr/^some.*/);
my $title = HTML::Manipulator::extract_title($html);
Instead of (or in addition to) using a DOM element ID to identify the section of the document you want to work on, you can also enclose that section in HTML comments. This approach is used by many software packages, for example Dreamweaver.
<!-- start description --> blah blah blah <!-- end -->
If you have HTML like above, you can get the section enclosed by the two comments using the same extraction functions as with element ID. Instead of an ID, you just use the opening comment tag:
my $content = HTML::Manipulator::extract_content($html, '<!-- start description -->');
There are some caveats:
The section starts with the comment tag you used in lieu of an element ID. It ends with the first subsequent comment tag, no matter how that tag looks like. This means that the enclosed section cannot contain HTML comments itself.
When matching the opening comment tag, all whitespace and case is ignored. '<!-- start description -->' and '<!--STARTDESCRIPTION-->' are the same thing
When using extract_all_content or extract_all without any parameters, the functions return the content of all elements with an ID. They do not return any sections marked up by comments (because it is impossible to figure out if a given HTML comment is supposed to mark a section in that manner without more information). If you want to get comment-enclosed sections, you have to explicitly name them. You also cannot use regular expressions directly in the extract_* functions like you could to match an ID.
You can query the document for all comments that have a certain form by using another function (extract_all_comments) and use the results of this function to specify what you need in extract_all_content.
my @array = HTML::Manipulator::extract_all_comments($html, qr/START/i);
This will give you something like: ( '<!-- start description -->', '<!-- start footer -->').
You can use the replace() function with the opening comment tag instead of an element ID:
my $new = HTML::Manipulator::replace($html, '<!-- title -->' => 'New news');
You can also call all of the above functions with a file handle instead of the string holding the HTML. HTML::Manipulator (or HTML::Parser deeper down the line) will read from the file.
use FileHandle; my $new = HTML::Manipulator::replace(new FileHandle('myfile.html'), title => 'New news'); open IN, 'myfile.html'; my $new = HTML::Manipulator::replace(*IN, title => 'New news'); close IN;
HTML::Manipulator will only read from the file handles you give it. It does not change them. Nor does it open them, you have to have done that yourself. Or you can use HTML::Manipulator::Document, which does open files.
The module exports none of its functions. You have to prefix the full module name to use them.
If you want an object-oriented interface instead, consider HTML::Manipulator::Document.
HTML::Manipulator uses HTML::Parser for parsing the input file. If you find yourself in the unfortunate situation of having to process HTML, have a look at that module.
For specific purposes there are also some other modules to work with, for example
HTML::HeadParser extracts info from the <head> section
HTML::LinkExtor and HTML::LinkExtractor extract links
HTML::FillInForm populates HTML forms with data
HTML::Manipulator is not a templating engine. You should not use it to produce output for your CGI script. If you want to do that, take a look at
HTML::Template (personal favorite)
Template Toolkit (also very popular)
Petal (innovative, uses attributes rather than tags for markup)
A great tool to produce and manage a large number of related static HTML pages is HTML::WebMake.
This is a young module. It works for me, but it has not been extensively tested in the wild. Handle with care. Report bugs to get them fixed.
Thilo Planz, <thilo@cpan.org>
Copyright 2004/05 by Thilo Planz
This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
To install HTML::Manipulator, copy and paste the appropriate command in to your terminal.
cpanm
cpanm HTML::Manipulator
CPAN shell
perl -MCPAN -e shell install HTML::Manipulator
For more information on module installation, please visit the detailed CPAN module installation guide.