Dezi::Bot::Queue::DBI - web crawler queue with DBI storage
use Dezi::Bot::Queue::DBI; my $queue = Dezi::Bot::Queue->new( type => 'DBI', dsn => "DBI:mysql:database=$database;host=$hostname;port=$port", username => 'myuser', password => 'mysecret', ); my $uri = 'http://dezi.org/bot.html'; $queue->put($uri); $queue->size(); # returns number of items in queue $queue->peek; # returns $uri (next value for get()) $queue->get; # returns $uri and removes it from queue
The Dezi::Bot::Queue::DBI class is a subclass of Dezi::Bot::Queue that uses DBI for storage.
Sets up the internal database handle (accessible via conn() attribute).
Returns the internal DBIx::Connector object.
Add item to the queue.
Returns the next item from the queue, marking it as unavailable. Default is to return 1 item, but set limit to return multiple.
update_cols is an optional hashref of column/value pairs to update when each item is locked.
Remove item from the queue completely.
Remove all locked items from the queue.
Returns the next item value, but leaves it on the stack as available.
Returns the number of items currently in the queue.
Callable as a function or class method. Returns string suitable for initializing a dezi_queue SQL table.
Example:
perl -e 'use Dezi::Bot::Queue::DBI; print Dezi::Bot::Queue::DBI::schema' |\ sqlite3 dezi.index/bot.db
Peter Karman, <karman at cpan.org>
<karman at cpan.org>
Please report any bugs or feature requests to bug-dezi-bot at rt.cpan.org, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Dezi-Bot. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.
bug-dezi-bot at rt.cpan.org
You can find documentation for this module with the perldoc command.
perldoc Dezi::Bot
You can also look for information at:
RT: CPAN's request tracker
http://rt.cpan.org/NoAuth/Bugs.html?Dist=Dezi-Bot
AnnoCPAN: Annotated CPAN documentation
http://annocpan.org/dist/Dezi-Bot
CPAN Ratings
http://cpanratings.perl.org/d/Dezi-Bot
Search CPAN
http://search.cpan.org/dist/Dezi-Bot/
Copyright 2013 Peter Karman.
This program is free software; you can redistribute it and/or modify it under the terms of either: the GNU General Public License as published by the Free Software Foundation; or the Artistic License.
See http://dev.perl.org/licenses/ for more information.
To install Dezi::Bot, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Dezi::Bot
CPAN shell
perl -MCPAN -e shell install Dezi::Bot
For more information on module installation, please visit the detailed CPAN module installation guide.