This is the user agent class. It is responsible to handle the page visit,
and page content/parsing calls.
The default ua is HTTP::Tiny. However, it is possible to create a new class
just like this one and make it work with other user agents.
Will visit the url you appended/prepended to the queue ex. $self->robot->queue->append( search => 'http://www.url.com', { passed_key_values => { send => 'var across requests', some => 'vars i collected here...... and ....', i_will => 'pass them to the next page because ...', i_need => 'stuff from this page and the other ', }, request => [ <---- OPTIONAL... force custom request 'GET', 'http://www.lopes.com.br/imoveis/busca/-/'.$estado.'/-/-/-/aluguel-de-0-a-10000/de-0-ate-1000-m2/-/60', { headers => { 'Content-Type' => 'application/x-www-form-urlencoded', }, content => '', } ] } );
Here the useragent will loop over defined content types and
will call the proper subroutine to treat page->content based
on content type.
To install HTML::Robot::Scrapper, copy and paste the appropriate command in to your terminal.
cpanm
cpanm HTML::Robot::Scrapper
CPAN shell
perl -MCPAN -e shell install HTML::Robot::Scrapper
For more information on module installation, please visit the detailed CPAN module installation guide.