/ 
Net-Async-Beanstalk-0.002
River stage zero No dependents
/ Net::Async::Beanstalk

NAME

Net::Async::Beanstalk - Non-blocking beanstalk client

SYNOPSIS

use IO::Async::Loop;
use Net::Async::Beanstalk;

my $loop = IO::Async::Loop->new();

my $client = Net::Async::Beanstalk->new();
$loop->add($client);

$client->connect(host => 'localhost', service => '11300')->get();
$client->put("anything")->get();

$loop->run();

BUGS

  • Receiving on_disconnect after sending quit might not work.

    In fact disconnecting hasn't been tested at all, even ad-hoc.

  • This document is even longer.

  • There are no tests

    See if it's appropriate to steal the tests out of Beanstalk::Client.

DESCRIPTION

Implements the client-side of the beanstalk 1.10 protocol described in https://raw.githubusercontent.com/kr/beanstalkd/v1.10/doc/protocol.txt using IO::Async to provide an asynchronous non-blocking API.

Net::Async::Beanstalk is based on Moo and IO::Async::Stream. Refer to those modules' documentation for basic usage. In particular "connect" in IO::Async::Loop.

ATTRIBUTES

Includes the command stack from Net::Async::Beanstalk::Send.

default_priority (10,000)
defauly_delay (0)
default_ttr (120)

Default values to associate with a job when it is "put" on the beanstalk server. The defaults here are arbitrary; they have been chosen to match the default values from AnyEvent::Beanstalk.

decoder (&YAML::Load)
encoder (&YAML::Dump)

A coderef which will be used to deserialise or serialise jobs as they are retreived from or sent to a beanstalk server.

This is not related to how the result of list or stats commands are deserialised. This is always done using "Load" in YAML.

using

The name of the tube which was recently used.

_watching

A hashref who's keys are the tubes which are being watched. The values are ignored.

Use the accessor watching to get the list of watched tubes instead of using the attribute directly.

CLASS

A Net::Async::Beanstalk object represents a single connection to a beanstalkd server. Once a connection has been established (see "CONNECTING") commands may be submitted by calling the objects methods (see "COMMAND METHODS").

The command methods all return a Future which will either be completed (marked done) with the result if the command was a success or failed with the error.

The command methods are named after the beanstalk API command that they implement, with the hyphens changed to an underscore (s/-/_/g). All command methods but "put" take the same options in the same order as the respective API command. "reserve" also has an option added to it so that it can be used to make reservations with or without a timeout.

Some events may happen without a command having been sent or in spite of a command being actively executed. These invoke the events documented below and do not complete or fail the associated Future. See the documentation of each error (there aren't many) for the details in "ERRORS".

Although this class implements a non-blocking beanstalk client, the protocol itself is not asynchronous. Each command sent will receive a response before the next command will be processed although in practice network buffering makes it appear that commands can be sent while waiting for a previous command's response.

Ordinarily this is irrelevant as all the commands except reserve and reserve-with-timeout respond quickly enough that any delay will be negligible and this class' internal command stack smooths over the times where that's not the case.

When any command which blocks the server has been sent, other commands will be stacked up waiting to be sent to the server but will not be handled (that is, not even put on the wire) until the running command command has completed (perhaps with a timeout).

If this is a concern, the beanstalkd server is explicitly written to support multiple connections with low overhead, so there is no need to perform all operations using the same client object. Just be aware that the list of active tubes is not copied (by default) from one client to another. Each client starts off using and watching the default tube.

When a job has been reserved by a client (which remains connected) that job is invisible to any other clients. It cannot, for example, be deleted except over the same connected in which it was reserved and if that connection is closed the job will return to the ready queue and may be reserved by another client.

CONNECTING

Voodoo.

ERRORS

There are not many error conditions described by the beanstalk protocol. Net::Async::Beanstalk::Receive also defines errors in the event of bugs revealing mistakes in this code or communication failures. Each will cause either the current Future to fail, raise an event, or both.

See each error's description but by and large the errors that happen because a command failed (which is not the same thing thing as "while a command was active") fail the Future while the error conditions that arise spontaneously invoke an on_error event (defined in IO::Async::Notifier). If the Net::Async::Beanstalk object (or its parent) is created without an on_error handler then the default on_error handler will be used which calls die.

Except where noted each error or failure includes the arguments which were sent with the command to the server (not including the command itself). If you call a command such as this

my $give_it_a_rest = $beanstalk->bury(0x6642, 9_001);

then $give_it_a_rest will hold a Future which will eventually fail and call its handler with:

$h->("...buried: 26180 not found", "beanstalk-peek", 0x6632, 9_001);

The exceptional errors are:

Protocol error: Bad format

Category: beanstalk-internal

Arguments: The buffer which was written into the communication stream.

This error invokes an on_error event and fails the active Future.

The server received a command line that was not well-formed. This should never happen and when it does it indicates an error in this module. Please report it so that it can be repaired.

Protocol error: Internal error

Category: beanstalk-server

Arguments: Everything.

This error invokes on_error only.

The server suffered from an internal error. This should never happen and when it does it indicates an error in the server. Please report it to the beanstalk maintainer so that it can be repaired.

This error does not attempt to fail the current or any pending Futures, however the server's probably about to crash so your code should deal with that and the pending Futures gracefully.

Protocol error: Out of memory

Category: beanstalk-server

Arguments: The command name and then the arguments as usual.

This error only fails the active Future.

The server ran out of memory. This happens sometimes but generally it should not. Please report it to your system administrator so that he can be repaired.

Protocol error: Unknown command

Category: beanstalk-internal

Arguments: The command name and then the arguments as usual.

This error invokes an on_error event and fails the active Future.

This module sent a command the server did not understand. This should never happen and when it does it indicates an error in the server or a protocol mismatch between the server and client.

Protocol error: Unknown response

Category: beanstalk-server

Arguments: The buffer which was received from the communication stream as an arrayref of each received chunk.

This error invokes on_error only.

The server sent a message this client did not understand. This should never happen and when it does it indicates an error in the server or a protocol mismatch between the server and client.

This error does not attempt to fail the current or any pending Futures, however the server's speaking gibberish so nobody knows what's going to happen next. Your code should deal with that and the pending Futures gracefully.

In order to make the command stack work, each Future is created with an on_ready handler which sends the next pending command. In the event of an error the pending commands may become invalid. This class makes no attempt to deal with that.

One other protocol error (Expected cr+lf) can be received only in response to a "put" command (it does not invoke an on_error event).

COMMAND METHODS

Methods which initiate a command on the server are implemented in Net::Async::Beanstalk::Send. The server response is processed by the event handlers in Net::Async::Beanstalk::Receive. Every command method returns a Future which will complete with the server's response to that command, whether success or failure.

With few exceptions, documented below, each method expects exactly the arguments that the respective command requires. The commands which expect to receive a YAML structure as the response (primarily the list-* commands) deserialise the response before returning it as a (non-reference) list or hash.

The methods are named with a _ where the API command has a -.

See the protocol documentation for further details on each command. They are:

put ($job, %options) or put (%options)

Put a job onto the currently used tube. The job data can be passed as the method's first argument or in %options.

The job's priority, delay or ttr can be set by including those values in %options. If they are not then the object's default value is used (see above).

The job may be passed as the method's first or only argument or as data in %options. It will be serialised using "encoder" if it's a reference and does not overload the stringify ("") operator.

The job may instead be passed as raw_data in %options if it has already been serialised.

Regardless of whether /encoder is used to serialise the job it is changed to a string of bytes using utf8::encode.

It is an error to pass the job data in more than one form or to included unknown options and put will croak.

Possible failures:

Protocol error: Expected cr+lf

Category: beanstalk-put

Arguments: As with bad format, this should but does not include the buffer which was sent.

This error only fails the active Future.

The client sent badly-formed job data which was not terminated by a CR+LF pair. This should never happen and when it does it indicates an error in this module. Please report it so that it can be repaired.

Invalid job: too big

Category: beanstalk-put

The client sent job data which was rejected by the server for being too large. The job has not beed stored in any queue.

Job was inserted but buried (out of memory): ID $id

Category: beanstalk-put

The job was successfully received by the server but it was unable to allocate memory to put it into the ready queue and so the job has been buried.

Job was not inserted: Server is draining

Category: beanstalk-put

The server is currently being drained and is not accepting new jobs. The job has not been stored in any queue.

reserve (%options)

Reserve the next available job. timeout may be passed in %options in which case the reserve-with-timout command is sent instead. timeout may be 0.

The data returned by the server is transformed into a string of characters with "decode" in utf8 then deserialised using /decoder.

If the asis option is set to a true value then the data returned by the server is transformed into characters but is not deserialised.

If the raw option is set to a true value then the data is left completely untouched.

Possible failures:

No job was reserved: Deadline soon

Category: beanstalk-reserve

A job which was previously reserved by this client and has not been handled is nearing the time when its reservation will expire and the server will restore it to the ready queue.

reserve_with_timeout ($time, %options)

Implemented by calling "reserve" with a timeout option. $time may be 0 which will cause the Future to fail immediately with a Timed out error if there are no jobs available.

Possible failures:

No job was reserved: Timed out

Category: beanstalk-reserve

The number of seconds specified in the timeout value to reserve-with-timeout has expired without a job becoming ready to reserve.

In addition all the failures possible in response to the "reserve" command can be received in response to reserve_with_timeout.

bury ($job_id)

Possible failures:

The job could not be buried: $id not found

Category: beanstalk-job

The job with ID $id could not be buried because it does not exist or has not been previously reserved by this client.

delete ($job_id)

Possible failures:

The job could not be deleted: $id not found

Category: beanstalk-job

The job with ID $id could not be deleted because it does not exist, has not been previously reserved by this client or is not in a ready or buried state.

ignore ($tube_name)

Possible failures:

The last tube cannot be ignored: $tube

Category: beanstalk-tube

The client attempted to ignore the only tube remaining in its watch list.

kick_job ($job_id)

Possible failures:

The job could not be kicked: $id not found

Category: beanstalk-job

The job with ID $id could not be kicked because it "is not in a kickable state".

kick ($max)

This command should not fail.

list_tubes ()

This command should not fail.

list_tubes_watched ()

This command should not fail.

list_tube_used ()

This command should not fail.

pause_tube ($tube_name, $delay)

Possible failures:

The tube could not be paused: $tube not found

Category: beanstalk-tube

The tube could not be paused because it doesn't exist.

peek ($job_id)

Possible failures:

The job could not be peeked at: $id not found

Category: beanstalk-peek

The specified job could not be retrieved because it does not exist.

peek_buried ()

Possible failures:

The next buried job could not be peeked at: None found

Category: beanstalk-peek

The next job in a buried state could not be retrieved because one does not exist.

peek_delayed ()

Possible failures:

The next delayed job could not be peeked at: None found

Category: beanstalk-peek

The next job in a delayed state could not be retrieved because one does not exist.

peek_ready ()

Possible failures:

The next ready job could not be peeked at: None found

Category: beanstalk-peek

The next job in a ready state could not be retrieved because one does not exist.

quit ()

In theory this will raise an on_disconnect in addition to completing the Future it returns. In practice I haven't written it yet.

release ($job_id, $priority, $delay)

Possible failures:

The job could not be released: $id not found

Category: beanstalk-job

The job with ID $id could not be released because it does not exist or has not been previously reserved by this client.

The job could not be released (out of memory): ID $id

Category: beanstalk-job

The job with ID $id could not be released because the server ran out of memory.

stats ()

This command should not fail.

stats_job ($job_id)

Possible failures:

Statistics were not found for the job: $id not found

Category: beanstalk-job

No statistics are available for the job with ID $id because it does not exist.

stats_tube ($tube_name)

Possible failures:

Statistics were not found for the tube: $tube not found

Category: beanstalk-tube

No statistics are available for the tube named $tube because it does not exist.

touch ($job_id)

Possible failures:

The job could not be touched: $id not found

Category: beanstalk-job

The job with ID $id could not be touched because it does not exist or has not been previously reserved by this client.

use ($tube_name)

This command should not fail.

watch ($tube_name)

This command should not fail.

OTHER METHODS

reserve_pending () => @commands

Returns a all the entries in the command stack which refer to a reserve or reserve-with-timeout command.

disconnect () => $future

An alias for quit.

sync () => $future

Returns a Future which completes when all pending commands have been responded to.

watch_only (@tubes) => $future

Send a list-tubes-watched command and based on its result send a series of watch and then ignore commands so that the tubes being watched for this client exactly matches @tubes.

INTERNAL METHODS

_assert_state($response_word) => VOID

Raises an exception of the word received from the server is not something expected in response to the command which has most recently been sent.

fail_command($message, $exception, @args) => $future

Remove the current command from the command stack and fail its Future with this method's arguments.

The Future returned is the one which returned when initiating a command and can be safely ignored.

This is used by Net::Async::Beanstalk::Receive when the client received an expected response which nevertheless indicates an error of some kind, such as DEADLINE_SOON received in response to a reserve command.

finish_command($event, @args) => $future

Remove the current command from the command stack and complete its Future with this method's arguments.

The Future returned is the one which returned when initiating a command and can be safely ignored.

This is used by Net::Async::Beanstalk::Receive when the server sent a response to a command which indicates successful completion.

ALTERNATIVE IMPLEMENTATIONS

AnyEvent::Beanstalk

A good module and asynchronous but it uses AnyEvent which ... the less said the better. The core of the protocol is implemented but it does not handle all error conditions. I have attempted to make Net::Async::Beanstalk's API superficially similar to this one.

Beanstalk::Client

Also written by Graham Barr, this module seems to be slightly more functionally complete than its AnyEvent counterpart and has proven itself stable and fast but unfortunately does not operate asynchronously.

AnyEvent::Beanstalk::Worker

Unfortunately also based on AnyEvent which is a shame because it implements what appears to be an interesting FSA using beanstalk queues.

Queue::Beanstalk

Ancient, presumably unsupported and based on an out-dated version of the beanstalk protocol.

SEE ALSO

IO::Async

Future

http://kr.github.com/beanstalkd/

https://raw.githubusercontent.com/kr/beanstalkd/v1.10/doc/protocol.txt

AUTHOR

Matthew King <chohag@jtan.com>