GunghoX::FollowLinks - Automatically Follow Links Within Responses
follow_links: parsers: - module: HTML config: rules: - module: HTML::SelectedTags config: tags: - a - module: MIME config: types: - text/html - module: Text config: rules: - module: URI config: match: - host: ^example\.com action: FOLLOW_ALLOW package MyHandler; sub handle_response { my ($self, $c, $req, $res) = @_; $c->follow_links($res); }
The most common action that a crawler takes is to follow links on a page. This module helps you with that task.
Parses the given HTTP::Response/Gungho::Response object and dispatches the appropriate parser from its content-type.
For each URL found, Automatically dispatches the rules given to the parser, and if the rules match, the URL is sent to Gungho->send_request.
Returns the number of matches found.
Copyright (c) 2007 Daisuke Maki <daisuke@endeworks.jp>
This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
See http://www.perl.com/perl/misc/Artistic.html
To install GunghoX::FollowLinks, copy and paste the appropriate command in to your terminal.
cpanm
cpanm GunghoX::FollowLinks
CPAN shell
perl -MCPAN -e shell install GunghoX::FollowLinks
For more information on module installation, please visit the detailed CPAN module installation guide.