The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.
 / 
LWP-UserAgent-Throttled-0.10
River stage one • 1 direct dependent • 1 total dependent
1 ++
/ LWP::UserAgent::Throttled

NAME

LWP::UserAgent::Throttled - Throttle requests to a site

VERSION

Version 0.10

SYNOPSIS

Some sites with REST APIs, such as openstreetmap.org, will blacklist you if you do too many requests. LWP::UserAgent::Throttled is a sub-class of LWP::UserAgent.

    use LWP::UserAgent::Throttled;
    my $ua = LWP::UserAgent::Throttled->new();
    $ua->throttle({ 'www.example.com' => 5 });
    print $ua->get('http://www.example.com/page1.html');
    sleep (2);
    print $ua->get('http://www.example.com/page2.html');        # Will wait at least 3 seconds before the GET is sent

SUBROUTINES/METHODS

send_request

See LWP::UserAgent.

throttle

Get/set the number of seconds between each request for sites.

    my $ua = LWP::UserAgent::Throttled->new();
    $ua->throttle({ 'search.cpan.org' => 0.1, 'www.example.com' => 1 });
    print $ua->throttle('search.cpan.org'), "\n";    # prints 0.1
    print $ua->throttle('perl.org'), "\n";    # prints 0

When setting a throttle it returns itself, so you can daisy chain messages.

ua

Get/set the user agent if you wish to use that rather than itself

    use LWP::UserAgent::Cached;

    $ua->ua(LWP::UserAgent::Cached->new(cache_dir => '/home/home/.cache/lwp-cache'));
    my $resp = $ua->get('https://www.nigelhorne.com');  # Throttles, then checks cache, then gets

AUTHOR

Nigel Horne, <njh at bandsman.co.uk>

BUGS

Please report any bugs or feature requests to bug-lwp-useragent-throttled at rt.cpan.org, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=LWP-UserAgent-Throttled. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.

Redirects to other domains can confuse it, so you need to program those manually.

SEE ALSO

LWP::UserAgent

SUPPORT

You can find documentation for this module with the perldoc command.

    perldoc LWP::UserAgent::Throttled

You can also look for information at:

LICENSE AND COPYRIGHT

Copyright 2017-2024 Nigel Horne.

This program is released under the following licence: GPL2