NAME

HTML::Parser - SGML parser class

SYNOPSIS

require HTML::Parser;
$p = HTML::Parser->new;  # should really a be subclass
$p->parse($chunk1);
$p->parse($chunk2);
#...
$p->eof;                 # signal end of document

# Parse directly from file
$p->parse_file("foo.html");
# or
open(F, "foo.html") || die;
$p->parse_file(\*F);

DESCRIPTION

The HTML::Parser will tokenize an HTML document when the parse() method is called by invoking various callback methods. The document to be parsed can be supplied in arbitrary chunks.

The external interface the an HTML::Parser is:

$p = HTML::Parser->new

The object constructor takes no arguments.

$p->parse( $string );

Parse the $string as an HTML document. Can be called multiple times. The return value is a reference to the parser object.

$p->eof

Signals end of document. Call eof() to flush any remaining buffered text. The return value is a reference to the parser object.

$p->parse_file( $file );

This method can be called to parse text from a file. The argument can be a filename or an already opened file handle. The return value from parse_file() is a reference to the parser object.

$p->strict_comment( [$bool] )

By default we parse comments similar to how the popular browsers (like Netscape and MSIE) do it. This means that comments will always be terminated by the first occurence of "-->". This is not correct according to the "official" HTML standards. The official behaviour can be enabled by calling the strict_comment() method with a TRUE argument.

The return value from strict_comment() is the old attribute value.

In order to make the parser do anything interesting, you must make a subclass where you override one or more of the following methods as appropriate:

$self->declaration($decl)

This method is called when a markup declaration has been recognized. For typical HTML documents, the only declaration you are likely to find is <!DOCTYPE ...>. The initial "<!" and ending ">" is not part of the string passed as argument. Comments are removed and entities will not be expanded.

$self->start($tag, $attr, $attrseq, $origtext)

This method is called when a complete start tag has been recognized. The first argument is the tag name (in lower case) and the second argument is a reference to a hash that contain all attributes found within the start tag. The attribute keys are converted to lower case. Entities found in the attribute values are already expanded. The third argument is a reference to an array with the lower case attribute keys in the original order. The fourth argument is the original HTML text.

$self->end($tag, $origtext)

This method is called when an end tag has been recognized. The first argument is the lower case tag name, the second the original HTML text of the tag.

$self->text($text)

This method is called when plain text in the document is recognized. The text is passed on unmodified and might contain multiple lines. Note that for efficiency reasons entities in the text are not expanded. You should call HTML::Entities::decode($text) before you process the text any further.

A sequence of text in the HTML document can be broken between several invokations of $self->text. The parser will make sure that it does not break a word or a sequence of spaces between two invokations of $self->text().

$self->comment($comment)

This method is called as comments are recognized. The leading and trailing "--" sequences have been stripped off the comment text.

The default implementation of these methods do nothing, i.e., the tokens are just ignored.

There is really nothing in the basic parser that is HTML specific, so it is likely that the parser can parse other kinds of SGML documents. SGML has many obscure features (not implemented by this module) that prevent us from renaming this module as SGML::Parser.

EFFICIENCY

The parser is fairly inefficient if the chunks passed to $p->parse() are too big. The reason is probably that perl ends up with a lot of character copying when tokens are removed from the beginning of the strings. A chunck size of about 256-512 bytes was optimal in a test I made with some real world HTML documents. (The parser was about 3 times slower with a chunck size of 20K).

SEE ALSO

HTML::Entities, HTML::TokeParser, HTML::Filter, HTML::HeadParser, HTML::LinkExtor

HTML::TreeBuilder (part of the HTML-Tree distribution)

COPYRIGHT

Copyright 1996-1998 Gisle Aas. All rights reserved.

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.