Net::Async::Beanstalk - Non-blocking beanstalk client
use IO::Async::Loop; use Net::Async::Beanstalk; my $loop = IO::Async::Loop->new(); my $client = Net::Async::Beanstalk->new(); $loop->add($client); $client->connect(host => 'localhost', service => '11300')->get(); $client->put("anything")->get(); $loop->run();
Receiving on_disconnect after sending quit might not work.
In fact disconnecting hasn't been tested at all, even ad-hoc.
This document is even longer.
There are no tests
See if it's appropriate to steal the tests out of Beanstalk::Client.
Implements the client-side of the beanstalk 1.10 protocol described in https://raw.githubusercontent.com/kr/beanstalkd/v1.10/doc/protocol.txt using IO::Async to provide an asynchronous non-blocking API.
Net::Async::Beanstalk is based on Moo and IO::Async::Stream. Refer to those modules' documentation for basic usage. In particular "connect" in IO::Async::Loop.
Includes the command stack from Net::Async::Beanstalk::Send.
Default values to associate with a job when it is "put" on the beanstalk server. The defaults here are arbitrary; they have been chosen to match the default values from AnyEvent::Beanstalk.
A coderef which will be used to deserialise or serialise jobs as they are retreived from or sent to a beanstalk server.
This is not related to how the result of list or stats commands are deserialised. This is always done using "Load" in YAML.
list
stats
The name of the tube which was recently used.
A hashref who's keys are the tubes which are being watched. The values are ignored.
watch
Use the accessor watching to get the list of watched tubes instead of using the attribute directly.
watching
A Net::Async::Beanstalk object represents a single connection to a beanstalkd server. Once a connection has been established (see "CONNECTING") commands may be submitted by calling the objects methods (see "COMMAND METHODS").
The command methods all return a Future which will either be completed (marked done) with the result if the command was a success or failed with the error.
done
The command methods are named after the beanstalk API command that they implement, with the hyphens changed to an underscore (s/-/_/g). All command methods but "put" take the same options in the same order as the respective API command. "reserve" also has an option added to it so that it can be used to make reservations with or without a timeout.
s/-/_/g
Some events may happen without a command having been sent or in spite of a command being actively executed. These invoke the events documented below and do not complete or fail the associated Future. See the documentation of each error (there aren't many) for the details in "ERRORS".
Although this class implements a non-blocking beanstalk client, the protocol itself is not asynchronous. Each command sent will receive a response before the next command will be processed although in practice network buffering makes it appear that commands can be sent while waiting for a previous command's response.
Ordinarily this is irrelevant as all the commands except reserve and reserve-with-timeout respond quickly enough that any delay will be negligible and this class' internal command stack smooths over the times where that's not the case.
reserve
reserve-with-timeout
When any command which blocks the server has been sent, other commands will be stacked up waiting to be sent to the server but will not be handled (that is, not even put on the wire) until the running command command has completed (perhaps with a timeout).
If this is a concern, the beanstalkd server is explicitly written to support multiple connections with low overhead, so there is no need to perform all operations using the same client object. Just be aware that the list of active tubes is not copied (by default) from one client to another. Each client starts off using and watching the default tube.
default
When a job has been reserved by a client (which remains connected) that job is invisible to any other clients. It cannot, for example, be deleted except over the same connected in which it was reserved and if that connection is closed the job will return to the ready queue and may be reserved by another client.
Voodoo.
There are not many error conditions described by the beanstalk protocol. Net::Async::Beanstalk::Receive also defines errors in the event of bugs revealing mistakes in this code or communication failures. Each will cause either the current Future to fail, raise an event, or both.
See each error's description but by and large the errors that happen because a command failed (which is not the same thing thing as "while a command was active") fail the Future while the error conditions that arise spontaneously invoke an on_error event (defined in IO::Async::Notifier). If the Net::Async::Beanstalk object (or its parent) is created without an on_error handler then the default on_error handler will be used which calls die.
on_error
Except where noted each error or failure includes the arguments which were sent with the command to the server (not including the command itself). If you call a command such as this
my $give_it_a_rest = $beanstalk->bury(0x6642, 9_001);
then $give_it_a_rest will hold a Future which will eventually fail and call its handler with:
$h->("...buried: 26180 not found", "beanstalk-peek", 0x6632, 9_001);
The exceptional errors are:
Protocol error: Bad format
Category: beanstalk-internal
beanstalk-internal
Arguments: The buffer which was written into the communication stream.
This error invokes an on_error event and fails the active Future.
The server received a command line that was not well-formed. This should never happen and when it does it indicates an error in this module. Please report it so that it can be repaired.
Protocol error: Internal error
Category: beanstalk-server
beanstalk-server
Arguments: Everything.
This error invokes on_error only.
The server suffered from an internal error. This should never happen and when it does it indicates an error in the server. Please report it to the beanstalk maintainer so that it can be repaired.
This error does not attempt to fail the current or any pending Futures, however the server's probably about to crash so your code should deal with that and the pending Futures gracefully.
Protocol error: Out of memory
Arguments: The command name and then the arguments as usual.
This error only fails the active Future.
The server ran out of memory. This happens sometimes but generally it should not. Please report it to your system administrator so that he can be repaired.
Protocol error: Unknown command
This module sent a command the server did not understand. This should never happen and when it does it indicates an error in the server or a protocol mismatch between the server and client.
Protocol error: Unknown response
Arguments: The buffer which was received from the communication stream as an arrayref of each received chunk.
The server sent a message this client did not understand. This should never happen and when it does it indicates an error in the server or a protocol mismatch between the server and client.
This error does not attempt to fail the current or any pending Futures, however the server's speaking gibberish so nobody knows what's going to happen next. Your code should deal with that and the pending Futures gracefully.
In order to make the command stack work, each Future is created with an on_ready handler which sends the next pending command. In the event of an error the pending commands may become invalid. This class makes no attempt to deal with that.
on_ready
One other protocol error (Expected cr+lf) can be received only in response to a "put" command (it does not invoke an on_error event).
Expected cr+lf
Methods which initiate a command on the server are implemented in Net::Async::Beanstalk::Send. The server response is processed by the event handlers in Net::Async::Beanstalk::Receive. Every command method returns a Future which will complete with the server's response to that command, whether success or failure.
With few exceptions, documented below, each method expects exactly the arguments that the respective command requires. The commands which expect to receive a YAML structure as the response (primarily the list-* commands) deserialise the response before returning it as a (non-reference) list or hash.
list-*
The methods are named with a _ where the API command has a -.
_
-
See the protocol documentation for further details on each command. They are:
Put a job onto the currently used tube. The job data can be passed as the method's first argument or in %options.
use
%options
The job's priority, delay or ttr can be set by including those values in %options. If they are not then the object's default value is used (see above).
priority
delay
ttr
The job may be passed as the method's first or only argument or as data in %options. It will be serialised using "encoder" if it's a reference and does not overload the stringify ("") operator.
data
""
The job may instead be passed as raw_data in %options if it has already been serialised.
raw_data
Regardless of whether /encoder is used to serialise the job it is changed to a string of bytes using utf8::encode.
/encoder
It is an error to pass the job data in more than one form or to included unknown options and put will croak.
put
Possible failures:
Protocol error: Expected cr+lf
Category: beanstalk-put
beanstalk-put
Arguments: As with bad format, this should but does not include the buffer which was sent.
bad format
The client sent badly-formed job data which was not terminated by a CR+LF pair. This should never happen and when it does it indicates an error in this module. Please report it so that it can be repaired.
Invalid job: too big
The client sent job data which was rejected by the server for being too large. The job has not beed stored in any queue.
Job was inserted but buried (out of memory): ID $id
The job was successfully received by the server but it was unable to allocate memory to put it into the ready queue and so the job has been buried.
Job was not inserted: Server is draining
The server is currently being drained and is not accepting new jobs. The job has not been stored in any queue.
Reserve the next available job. timeout may be passed in %options in which case the reserve-with-timout command is sent instead. timeout may be 0.
timeout
reserve-with-timout
0
The data returned by the server is transformed into a string of characters with "decode" in utf8 then deserialised using /decoder.
/decoder
If the asis option is set to a true value then the data returned by the server is transformed into characters but is not deserialised.
asis
If the raw option is set to a true value then the data is left completely untouched.
raw
No job was reserved: Deadline soon
Category: beanstalk-reserve
beanstalk-reserve
A job which was previously reserved by this client and has not been handled is nearing the time when its reservation will expire and the server will restore it to the ready queue.
Implemented by calling "reserve" with a timeout option. $time may be 0 which will cause the Future to fail immediately with a Timed out error if there are no jobs available.
$time
Timed out
No job was reserved: Timed out
The number of seconds specified in the timeout value to reserve-with-timeout has expired without a job becoming ready to reserve.
In addition all the failures possible in response to the "reserve" command can be received in response to reserve_with_timeout.
reserve_with_timeout
The job could not be buried: $id not found
Category: beanstalk-job
beanstalk-job
The job with ID $id could not be buried because it does not exist or has not been previously reserved by this client.
$id
The job could not be deleted: $id not found
The job with ID $id could not be deleted because it does not exist, has not been previously reserved by this client or is not in a ready or buried state.
ready
buried
The last tube cannot be ignored: $tube
Category: beanstalk-tube
beanstalk-tube
The client attempted to ignore the only tube remaining in its watch list.
The job could not be kicked: $id not found
The job with ID $id could not be kicked because it "is not in a kickable state".
This command should not fail.
The tube could not be paused: $tube not found
The tube could not be paused because it doesn't exist.
The job could not be peeked at: $id not found
Category: beanstalk-peek
beanstalk-peek
The specified job could not be retrieved because it does not exist.
The next buried job could not be peeked at: None found
The next job in a buried state could not be retrieved because one does not exist.
The next delayed job could not be peeked at: None found
The next job in a delayed state could not be retrieved because one does not exist.
The next ready job could not be peeked at: None found
The next job in a ready state could not be retrieved because one does not exist.
In theory this will raise an on_disconnect in addition to completing the Future it returns. In practice I haven't written it yet.
on_disconnect
The job could not be released: $id not found
The job with ID $id could not be released because it does not exist or has not been previously reserved by this client.
The job could not be released (out of memory): ID $id
The job with ID $id could not be released because the server ran out of memory.
Statistics were not found for the job: $id not found
No statistics are available for the job with ID $id because it does not exist.
Statistics were not found for the tube: $tube not found
No statistics are available for the tube named $tube because it does not exist.
$tube
The job could not be touched: $id not found
The job with ID $id could not be touched because it does not exist or has not been previously reserved by this client.
Returns a all the entries in the command stack which refer to a reserve or reserve-with-timeout command.
An alias for quit.
Returns a Future which completes when all pending commands have been responded to.
Send a list-tubes-watched command and based on its result send a series of watch and then ignore commands so that the tubes being watched for this client exactly matches @tubes.
list-tubes-watched
ignore
@tubes
Raises an exception of the word received from the server is not something expected in response to the command which has most recently been sent.
Remove the current command from the command stack and fail its Future with this method's arguments.
The Future returned is the one which returned when initiating a command and can be safely ignored.
This is used by Net::Async::Beanstalk::Receive when the client received an expected response which nevertheless indicates an error of some kind, such as DEADLINE_SOON received in response to a reserve command.
DEADLINE_SOON
Remove the current command from the command stack and complete its Future with this method's arguments.
This is used by Net::Async::Beanstalk::Receive when the server sent a response to a command which indicates successful completion.
A good module and asynchronous but it uses AnyEvent which ... the less said the better. The core of the protocol is implemented but it does not handle all error conditions. I have attempted to make Net::Async::Beanstalk's API superficially similar to this one.
Also written by Graham Barr, this module seems to be slightly more functionally complete than its AnyEvent counterpart and has proven itself stable and fast but unfortunately does not operate asynchronously.
Unfortunately also based on AnyEvent which is a shame because it implements what appears to be an interesting FSA using beanstalk queues.
Ancient, presumably unsupported and based on an out-dated version of the beanstalk protocol.
IO::Async
Future
http://kr.github.com/beanstalkd/
https://raw.githubusercontent.com/kr/beanstalkd/v1.10/doc/protocol.txt
Matthew King <chohag@jtan.com>
To install Net::Async::Beanstalk, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Net::Async::Beanstalk
CPAN shell
perl -MCPAN -e shell install Net::Async::Beanstalk
For more information on module installation, please visit the detailed CPAN module installation guide.