-
-
28 Jan 2022 16:16:59 UTC
- Distribution: LWP-UserAgent-Throttled
- Module version: 0.09
- Source (raw)
- Browse (raw)
- Changes
- How to Contribute
- Repository (git clone)
- Issues
- Testers (357 / 7 / 0)
- Kwalitee
Bus factor: 1- 95.89% Coverage
- License: open_source
- Perl: v5.6.2
- Activity
24 month- Tools
- Download (15.57KB)
- MetaCPAN Explorer
- Permissions
- Subscribe to distribution
- Permalinks
- This version
- Latest version
- Dependencies
- LWP
- LWP::UserAgent
- Time::HiRes
- and possibly others
- Reverse dependencies
- CPAN Testers List
- Dependency graph
NAME
LWP::UserAgent::Throttled - Throttle requests to a site
VERSION
Version 0.09
SYNOPSIS
Some sites with REST APIs, such as openstreetmap.org, will blacklist you if you do too many requests. LWP::UserAgent::Throttled is a sub-class of LWP::UserAgent.
use LWP::UserAgent::Throttled; my $ua = LWP::UserAgent::Throttled->new(); $ua->throttle({ 'www.example.com' => 5 }); print $ua->get('http://www.example.com/page1.html'); sleep (2); print $ua->get('http://www.example.com/page2.html'); # Will wait at least 3 seconds before the GET is sent
SUBROUTINES/METHODS
send_request
See LWP::UserAgent.
throttle
Get/set the number of seconds between each request for sites.
my $ua = LWP::UserAgent::Throttled->new(); $ua->throttle({ 'search.cpan.org' => 0.1, 'www.example.com' => 1 }); print $ua->throttle('search.cpan.org'), "\n"; # prints 0.1 print $ua->throttle('perl.org'), "\n"; # prints 0
When setting a throttle it returns itself, so you can daisy chain messages.
ua
Get/set the user agent if you wish to use that rather than itself
use LWP::UserAgent::Cached; $ua->ua(LWP::UserAgent::Cached->new(cache_dir => '/home/home/.cache/lwp-cache')); my $resp = $ua->get('https://www.nigelhorne.com'); # Throttles, then checks cache, then gets
AUTHOR
Nigel Horne,
<njh at bandsman.co.uk>
BUGS
Please report any bugs or feature requests to
bug-lwp-useragent-throttled at rt.cpan.org
, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=LWP-UserAgent-Throttled. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.Redirects to other domains can confuse it, so you need to program those manually.
SEE ALSO
SUPPORT
You can find documentation for this module with the perldoc command.
perldoc LWP::UserAgent::Throttled
You can also look for information at:
MetaCPAN
RT: CPAN's request tracker
https://rt.cpan.org/NoAuth/Bugs.html?Dist=LWP-UserAgent-Throttled
CPANTS
CPAN Testers' Matrix
CPAN Ratings
CPAN Testers Dependencies
http://deps.cpantesters.org/?module=LWP::UserAgent::Throttled
LICENSE AND COPYRIGHT
Copyright 2017-2022 Nigel Horne.
This program is released under the following licence: GPL2
Module Install Instructions
To install LWP::UserAgent::Throttled, copy and paste the appropriate command in to your terminal.
cpanm LWP::UserAgent::Throttled
perl -MCPAN -e shell install LWP::UserAgent::Throttled
For more information on module installation, please visit the detailed CPAN module installation guide.