LWP::RobotUA - a class for well-behaved Web robots River stage four • 2206 direct dependents • 7667 total dependents

This class implements a user agent that is suitable for robot applications. Robots should be nice to the servers they visit. They should consult the /robots.txt file to ensure that they are welcomed and they should not make requests too frequently. B...

OALDERS/libwww-perl-6.41 - 28 Oct 2019 14:45:05 GMT

LWP::Parallel::RobotUA - A class for Parallel Web Robots River stage one • 4 direct dependents • 5 total dependents

This class implements a user agent that is suitable for robot applications. Robots should be nice to the servers they visit. They should consult the /robots.txt file to ensure that they are welcomed and they should not make requests too frequently. B...

MSOUTH/ParallelUserAgent-2.62 - 29 May 2016 18:55:38 GMT

WWW::Scraper - framework for scraping results from search engines. River stage one • 1 direct dependent • 1 total dependent

NOTE: You can find a full description of the Scraper framework in WWW::Scraper::ScraperPOD.pm. "Scraper" is a framework for issuing queries to a search engine, and scraping the data from the resultant multi-page responses, and the associated detail p...

GLENNWOOD/Scraper-3.05 - 02 Aug 2003 07:47:12 GMT

LWP::RobotPUA - Parallel LWP::RobotUA River stage one • 4 direct dependents • 5 total dependents

RobotPUA is a simple frontend to the LWP::Parallel::RobotUA module. It is here in order to maintain the compatibility with previous releases. However, in order to prevent the previous need for changing the original LWP sources, all extension files ha...

MSOUTH/ParallelUserAgent-2.62 - 29 May 2016 18:55:38 GMT

LWP - The World-Wide Web library for Perl River stage four • 2206 direct dependents • 7667 total dependents

The libwww-perl collection is a set of Perl modules which provides a simple and consistent application programming interface (API) to the World-Wide Web. The main focus of the library is to provide classes and functions that allow you to write WWW cl...

OALDERS/libwww-perl-6.41 - 28 Oct 2019 14:45:05 GMT

LWP::Parallel - Extension for LWP to allow parallel HTTP and FTP access River stage one • 4 direct dependents • 5 total dependents

Introduction ParallelUserAgent is an extension to the existing libwww module. It allows you to take a list of URLs (it currently supports HTTP, FTP, and FILE URLs. HTTPS might work, too) and connect to all of them _in parallel_, then wait for the res...

MSOUTH/ParallelUserAgent-2.62 - 29 May 2016 18:55:38 GMT

LWP::FastRobot - a robot which lets us check how long it will wait River stage zero No dependents

Just like a RobotUA, but you can see how long it will wait for a given link and then try another....

MIKEDLR/Link_Controller-0.037 - 09 Feb 2002 18:12:34 GMT

LWP::NoStopRobot - a robot doesn't stop, but remembers where it has been River stage zero No dependents

This module implements a user agent which remembers where it has been and when so that the user can avoid too fast visits, but doesn't actual implement that wait....

MIKEDLR/Link_Controller-0.037 - 09 Feb 2002 18:12:34 GMT

8 results (0.035 seconds)