++ed by:
PAVELSR ROZCOVO DUNCS

3 PAUSE users
1 non-PAUSE user.

Keita Sugama

NAME

WWW::Crawler::Mojo::Queue::Memory - Crawler queue with memory

SYNOPSIS

DESCRIPTION

Crawler queue with memory.

ATTRIBUTES

This class inherits all methods from WWW::Crawler::Mojo::Queue and implements following new ones.

cap

Capacity of queue, indecating how many jobs can be kept in queue at a time. If you enqueue over capacity, the oldest job will be automatically disposed.

jobs

jobs.

redundancy_storage

A hash ref in which the class keeps DONE flags for each jobs in order to avoid to perform resembling jobs multiple times.

    # Mark a job as DONE
    $queue->redundancy_storage->{$job->digest} = 1;
    
    # Delete the mark
    delete($queue->redundancy_storage->{$job->digest});

METHODS

This class inherits all methods from WWW::Crawler::Mojo::Queue class and implements following new ones.

dequeue

Implementation for WWW::Crawler::Mojo::Queue interface.

enqueue

Implementation for WWW::Crawler::Mojo::Queue interface.

length

Implementation for WWW::Crawler::Mojo::Queue interface.

next

Implementation for WWW::Crawler::Mojo::Queue interface.

requeue

Implementation for WWW::Crawler::Mojo::Queue interface.

shuffle

Implementation for WWW::Crawler::Mojo::Queue interface.

AUTHOR

Keita Sugama, <sugama@jamadam.com>

COPYRIGHT AND LICENSE

Copyright (C) Keita Sugama.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.