/ 
LWP-UserAgent-Throttled-0.11
River stage one • 1 direct dependent • 1 total dependent
/ LWP::UserAgent::Throttled

NAME

LWP::UserAgent::Throttled - Throttle requests to a site

VERSION

Version 0.11

SYNOPSIS

Some sites with REST APIs, such as openstreetmap.org, will blacklist you if you do too many requests. LWP::UserAgent::Throttled is a sub-class of LWP::UserAgent.

use LWP::UserAgent::Throttled;
my $ua = LWP::UserAgent::Throttled->new();
$ua->throttle({ 'www.example.com' => 5 });
print $ua->get('http://www.example.com/page1.html');
sleep (2);
print $ua->get('http://www.example.com/page2.html');	# Will wait at least 3 seconds before the GET is sent

SUBROUTINES/METHODS

send_request

See LWP::UserAgent.

throttle

Get/set the number of seconds between each request for sites.

my $ua = LWP::UserAgent::Throttled->new();
$ua->throttle({ 'search.cpan.org' => 0.1, 'www.example.com' => 1 });
print $ua->throttle('search.cpan.org'), "\n";    # prints 0.1
print $ua->throttle('perl.org'), "\n";    # prints 0

When setting a throttle it returns itself, so you can daisy chain messages.

ua

Get/set the user agent if you wish to use that rather than itself

use LWP::UserAgent::Cached;

$ua->ua(LWP::UserAgent::Cached->new(cache_dir => '/home/home/.cache/lwp-cache'));
my $resp = $ua->get('https://www.nigelhorne.com');	# Throttles, then checks cache, then gets

AUTHOR

Nigel Horne, <njh at bandsman.co.uk>

BUGS

Please report any bugs or feature requests to bug-lwp-useragent-throttled at rt.cpan.org, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=LWP-UserAgent-Throttled. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.

Redirects to other domains can confuse it, so you need to program those manually.

SEE ALSO

LWP::UserAgent

SUPPORT

You can find documentation for this module with the perldoc command.

perldoc LWP::UserAgent::Throttled

You can also look for information at:

LICENSE AND COPYRIGHT

Copyright 2017-2024 Nigel Horne.

This program is released under the following licence: GPL2