DESCRIPTION

This is the user agent class. It is responsible to handle the page visit,

 and page content/parsing calls.

ua

The default ua is HTTP::Tiny. However, it is possible to create a new class

just like this one and make it work with other user agents.

visit

    Will visit the url you appended/prepended to the queue
    
    ex.

    $self->robot->queue->append( search => 'http://www.url.com', {
        passed_key_values => {
            send   => 'var across requests',
            some   => 'vars i collected here...... and ....',
            i_will => 'pass them to the next page because ...',
            i_need => 'stuff from this page and the other ',
        },
        request => [ <---- OPTIONAL... force custom request
            'GET',
            'http://www.lopes.com.br/imoveis/busca/-/'.$estado.'/-/-/-/aluguel-de-0-a-10000/de-0-ate-1000-m2/-/60',
            {
                headers => {
                    'Content-Type' => 'application/x-www-form-urlencoded',
                },
                content => '',
            }
        ]
    } );
    

parse_content

Here the useragent will loop over defined content types and

will call the proper subroutine to treat page->content based

on content type.