LWP::UserAgent::ExponentialBackoff - LWP::UserAgent extension that retries errors with exponential backoff


  my $before_request = sub {
    print "# /Trying ", $_[1][0]->uri, " in ", $_[3], " seconds\n";

  my @failCodes    = qw(500 503);
  my %failCodesMap = map { $_ => $_ } @failCodes;

  %options = (
          before_request => $before_request,
          after_request  => $after_request,
          tolerance      => .20,
          retryCount     => 5,
          minBackoff     => 3,
          maxBackoff     => 120,
          deltaBackoff   => 3,
          failCodes      => \%failCodesMap

  my $ua = LWP::UserAgent::ExponentialBackoff->new(%options);
  my $request   = HTTP::Request->new( 'GET', $uri );
  my $response  = $ua->request($request);


LWP::UserAgent::ExponentialBackoff is a LWP::UserAgent extention. It retries requests on error using an exponential backoff algorthim.


The following constructor methods are available:

$ua = LWP::UserAgent::ExponentialBackoff->new( %options )

This method constructs a new LWP::UserAgent::ExponentialBackoff object and returns it. Key/value pair arguments may be provided to set up the initial state.

   KEY                     DEFAULT
   -----------             --------------------
   sum                     undef
   retryCount              3
   minBackoff              3
   maxBackoff              90
   tolerance               .20
   deltaBackoff            3
   failCodes               { map { $_ => $_ } qw(408 500 502 503 504) } 

See LWP::UserAgent for additional key/value pair arguments that may be provided.


This module inherits all of LWP::UserAgent's methods, and adds the following.

$browser->before_request( \&some_routine );
$browser->after_request( \&some_routine );

These read (first two) or set (second two) callbacks that are called before the actual HTTP/FTP/etc request is made. By default, these are set to undef, meaning nothing special is called. If you want to alter try requests, or inspect responses before any retrying is considered, you can set up these callbacks.

The arguments passed to these routines are:

0: the current $browser object
1: an arrayref of the arguments we pass to LWP::UserAgent::simple_request (the first of which is the request object)
2: the number of seconds we sleep on this retry.
(3): And, only for after_request, the response we just got.


This class works by overriding LWP::UserAgent's simple_request method with an exponential backoff algortihm.


LWP, LWP::UserAgent, LWP::UserAgent::Determined


Copyright 2011, Michael Marrotte, all rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

This program is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose.


Michael Marrotte

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

2 POD Errors

The following errors were encountered while parsing the POD:

Around line 204:

You forgot a '=back' before '=head1'

Around line 241:

You forgot a '=back' before '=head1'