Queue::Q::ClaimFIFO - FIFO queue keeping track of claimed items
use Queue::Q::ClaimFIFO::Redis; # or ::Perl or ... my $q = ... create object of chosen ClaimFIFO implementation... # producer: $q->enqueue_item([qw(my data structure)]); # rinse repeat... # consumer: my $item = $q->claim_item; my $data = $item->data; # work with data... $q->mark_item_as_done($item); # Implementation dependent: somewhere in a recovery cron job # - Fetch claimed items older than $n minutes/seconds # - Requeue or log&drop those timed-out items
Abstract interface class for a FIFO queue that keeps track of all in-flight ("claimed") items. Implementations are required to provide strict ordering and adhere to the run-time complexities listed below (or better).
The general workflow with Queue::Q::ClaimFIFO based queue is:
Queue::Q::ClaimFIFO
Producer enqueues one or multiple items.
Consumer claims one or multiple items and works on them.
Consumer marks its claimed items as done.
To recover from failed consumers, one may apply in any one of many application specific recovery strategies such as periodically re-enqueuing all claimed items that are older than a threshold or possible simply clearing them out and logging the fact.
Since the actual recovery strategy is application-dependent and the support by the queue implementation may vary, there's no API for this in this abstract base class. That may change in a future release.
Given a data structure, that data structure is added to the queue. Items enqueued with enqueue_item in order must be returned by claim_item in the same order.
enqueue_item
claim_item
The data structure passed to enqueue_item will be automatically wrapped in a Queue::Q::ClaimFIFO::Item object by enqueue_item. You cannot pass an object of that class (or its subclasses) as the (top-level) data structure to prevent using the same Item objects in multiple queues accidentally.
Returns the Queue::Q::ClaimFIFO::Item.
Queue::Q::ClaimFIFO::Item
Complexity: O(1)
Given a number data structures, enqueues them in order. This is conceptually the same as calling enqueue_item multiple times, but may help save on network round-trips.
As with enqueue_item this will refused to accept prefabricated Queue::Q::ClaimFIFO::Items.
Returns the Queue::Q::ClaimFIFO::Items.
Complexity: O(n) where n is the number of items to enqueue.
Returns the oldest item in the queue. Returns undef if there is none left.
undef
This does not return the originally enqueued data structure directly but the Queue::Q::ClaimFIFO::Item object that wraps it. It is this returned object that you need to pass to mark_item_as_done to remove it from the tracked in-flight items.
mark_item_as_done
Implementations may (but should not) deviate from the strict O(1) complexity for the number of claimed items at any given time. That is acceptable as high as O(log(n)). This is likely acceptable because the number of items being worked on is likely not to be extremely large. Implementations that make use of this relaxed requirement must document that clearly. The number of queued items must not affect the complexity, however.
As enqueue_items is to enqueue_item, claim_items is to claim_item. Takes one optional parameter (defaults to 1): The number of items to fetch and return:
enqueue_items
claim_items
my @items = $q->claim_items(20); # returns a batch of 20 items
If there are less than the desired number of items to be claimed, returns a correspondingly shorter list.
Complexity: O(n) where n is the number of items claimed.
See the documentation of claim_item for a potential relaxation on the complexity bounds with respect to the number of in-flight, claimed items.
Given a Queue::Q::ClaimFIFO::Item object that was previously claimed from this queue, it is removed from the claimed-items tracking and thus removed from the queue altogether.
Complexity: Generally O(1), but O(log(n)) under relaxed requirements (see above).
Given any number of Queue::Q::ClaimFIFO::Item objects that were previously claimed from this queue, they are removed from the claimed-items tracking and thus removed from the queue altogether.
Complexity: Generally O(n), but O(n*log(n)) under relaxed requirements (see above) with n understood to be the number of items to mark as done.
Returns the number of items available in the queue.
Removes all content in the queue.
Complexity: O(n) where n is the number of items in the queue.
Steffen Mueller, <smueller@cpan.org>
Copyright (C) 2012 by Steffen Mueller
This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.8.1 or, at your option, any later version of Perl 5 you may have available.
To install Queue::Q, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Queue::Q
CPAN shell
perl -MCPAN -e shell install Queue::Q
For more information on module installation, please visit the detailed CPAN module installation guide.