Gungho::Component::RobotsMETA - Automatically Parse Robots META
components: - RobotsMETA
This module automatically parses any text/html document for robots exclusion directies embedded in the document.
Initializes the component.
Overrides Gungho::Component::Core::handle_response()
HTML::RobotsMETA
To install Gungho, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Gungho
CPAN shell
perl -MCPAN -e shell install Gungho
For more information on module installation, please visit the detailed CPAN module installation guide.