FastRobot - a robot which lets us check how long it will wait
use FastRobot my $ua=FastRobt::new(....) $ua->check_wait
Just like a RobotUA, but you can see how long it will wait for a given link and then try another.
robot_check - checks whether a request will be allowed by the robot rules but doesn't actually send a request. This has the advantage that it means that the host_wait function is then accurate.
Returns the number of seconds (from now) you must wait before you can make a new request to this host.
Sets a regular expression for links for which the robot agent should not wait. Typically these would be local pages or servers in the same organisation as the link checking is being carried out by.
1 POD Error
The following errors were encountered while parsing the POD:
- Around line 30:
'=item' outside of any '=over'
=over without closing =back