NAME
Scrappy::Queue - Scrappy Request Scheduler and Queue System
VERSION
version 0.9111110
SYNOPSIS
#!/usr/bin/perl
use Scrappy;
my $surl = ... # starting url
my $scraper = Scrappy->new;
while (my $url = $scraper->queue($surl)->next) {
if ($scraper->get($url)) {
...
}
}DESCRIPTION
Scrappy::Queue provides Scrappy with methods for navigating a list of stored URLs.
METHODS
list
The list method is used to return the ordered list queued URLs.
my $queue = Scrappy::Queue->new;
my @urls = $queue->list;add
The add method is used to add URLs to the queue.
my $queue = Scrappy::Queue->new;
$queue->add('http://search.cpan.org', 'http://google.com');clear
The clear method empties the URLs queue and resets the queue cursor.
my $queue = Scrappy::Queue->new;
$queue->clear;reset
The reset method resets the queue cursor only.
my $queue = Scrappy::Queue->new;
$queue->add(...);
my $url = $queue->next;
$queue->reset;current
The current method returns the value of the current position in the queue.
my $queue = Scrappy::Queue->new;
print $queue->current;next
The next method returns the next value from the current position of the queue.
my $queue = Scrappy::Queue->new;
print $queue->next;previous
The previous method returns the previous value from the current position in the queue.
my $queue = Scrappy::Queue->new;
print $queue->previous;first
The first method returns the first value in the queue.
my $queue = Scrappy::Queue->new;
print $queue->first;last
The last method returns the last value in the queue.
my $queue = Scrappy::Queue->new;
print $queue->last;index
The index method returns the value of the specified position in the queue.
my $queue = Scrappy::Queue->new;
my $index = 0; # first position (same as array)
print $queue->index($index);cursor
The cursor method returns the value (index position) of the cursor.
my $queue = Scrappy::Queue->new;
print $queue->cursor;AUTHOR
Al Newkirk <awncorp@cpan.org>
COPYRIGHT AND LICENSE
This software is copyright (c) 2010 by awncorp.
This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.