Disbatch::Web - Disbatch Command Interface (JSON REST API and web browser interface to Disbatch).
version 4.102
parse_params, send_json_options, template
Parameters: path to the Disbatch config file. Default is /etc/disbatch/config.json.
/etc/disbatch/config.json
Initializes the settings for the web server, including loading any custom routes via config.web_extensions (see "CUSTOM ROUTES" below).
config.web_extensions
Returns nothing.
Parameters: template (.tt) file name in the config.views_dir directory, HASH of parameters for the template.
.tt
config.views_dir
HASH
Creates a web page based on the passed data.
Sets Content-Type to text/html.
Content-Type
text/html
Returns the generated html document.
NOTE: this sub is automatically exported, so any package using Disbatch::Web can call it.
Parameters: none
Parses request parameters in the following order:
* from the request body if the Content-Type is application/x-www-form-urlencoded
application/x-www-form-urlencoded
* from the request body if the Content-Type is application/json
application/json
* from the request query otherwise
It then puts any fields starting with . into their own HASH $options.
.
$options
Returns the HASH of the parsed request parameters, and if wantarray also returns the HASH of options.
wantarray
Used to enable the following options when returning JSON: allow_blessed, canonical, and convert_blessed.
allow_blessed
canonical
convert_blessed
Returns a list of key/value pairs of options to pass to send_json.
list
send_json
Parses Accept header.
Accept
Returns a HASH where keys are types and values are q-factor weights.
Returns true if Accept header has application/json with a higher q-factor weight than text/html.
Note: if not specified, text/html has an assumed q-factor weight of 0 and application/json has an assumed q-factor weight of 1.
0
1
Returns an array of node objects defined, with timestamp stringified and id the stringified _id.
timestamp
id
_id
Returns a HASH of defined queues plugins and any defined config.plugins, where values match the keys.
config.plugins
Parameters: Queue ID as a string, or queue name.
Returns a MongoDB::OID object representing this queue's _id.
MongoDB::OID
Parameters: MongoDB::OID object of the queue _id, ARRAY of task params.
ARRAY
Creates one queued task document for the given queue _id per $tasks entry. Each $task entry becomes the value of the params field of the document.
$tasks
$task
params
Returns: the repsonse object from a MongoDB::Collection#insert_many request.
MongoDB::Collection#insert_many
Parameters: legacy params (optional, used by routes in Disbatch::Web::Tasks), also parses request parameters
Handles creating tasks to insert, and then creates them via create_tasks(). See POST /tasks below for usage.
create_tasks()
POST /tasks
Returns the resonse of create_tasks() as JSON with the key the ref type of the response and the value the response turned into a HASH, or on error sets HTTP status to 400 and returns JSON of {"error":message}.
400
{"error":message}
Parameters: ARRAY of task documents, HASH of param options
Options handled are .terse, .full, and .epoch, all booleans.
.terse
.full
.epoch
If .terse, stdout and stderr values of each document will be [terse mode] if defined and not a MongoDB::OID object. Else if .full, stdout and stderr values of each document will be actual content instead of MongoDB::OID objects. If .epoch, ctime and mtime will be turned into hires_epoch (ex: 1548272576.574) insteaad of stringified (ex: 2019-01-23T19:42:56) if they are DateTime objects.
stdout
stderr
[terse mode]
ctime
mtime
hires_epoch
1548272576.574
2019-01-23T19:42:56
DateTime
Returns nothing, modifies passed tasks.
Returns a HASH of the balance doc without the _id field, with the following added: field known_queues with value an ARRAY of all existing queue names, field settings with value the HASH of config.balance. If the balance doc does not exist, the field notice with value balance document not found is added.
known_queues
settings
config.balance
notice
balance document not found
Parameters: none (but parses request parameters, see POST /balance below)
POST /balance
Sets the balance document fields given in the request parameters to the given values.
balance
Returns { status => 'success: queuebalance modified' } on success, or { status => 'failed: invalid json passed ' . $_ } with HTTP status of 400 on error.
{ status => 'success: queuebalance modified' }
{ status => 'failed: invalid json passed ' . $_ }
Checks if Disbatch nodes exist and determines if any have been running within the last 60 seconds.
Returns { status => 'WARNING', message => 'No Disbatch nodes found' } if no nodes, { status => 'OK', message => 'Disbatch is running on one or more nodes', nodes => $status } if at least one node recently running, or { status => 'CRITICAL', message => 'No active Disbatch nodes found', nodes => $status } if not. On error, returns { status => 'CRITICAL', message => "Could not get current Disbatch nodes: $_" }.
{ status => 'WARNING', message => 'No Disbatch nodes found' }
{ status => 'OK', message => 'Disbatch is running on one or more nodes', nodes => $status }
{ status => 'CRITICAL', message => 'No active Disbatch nodes found', nodes => $status }
{ status => 'CRITICAL', message => "Could not get current Disbatch nodes: $_" }
Checks if QueueBalance has been running within the last 60 seconds.
Returns { status => 'OK', message => 'queuebalance disabled' } if config.balance.enabled is false. If the balance doc has status of CRITICAL and no timestamp, returns { status => 'CRITICAL', message => $message }. If the timestamp value is older than 60 seconds, returns { status => 'CRITICAL' , message => "queuebalanced not running for ${seconds}s" }. If the status value is not OK, WARNING, or CRITICAL, returns { status => 'CRITICAL', message => 'queuebalanced unknown status', result => $doc }. Otherwise returns the doc: { status => $status, message => $message, timestamp => $timestamp }.
{ status => 'OK', message => 'queuebalance disabled' }
config.balance.enabled
status
CRITICAL
{ status => 'CRITICAL', message => $message }
{ status => 'CRITICAL' , message => "queuebalanced not running for ${seconds}s" }
OK
WARNING
{ status => 'CRITICAL', message => 'queuebalanced unknown status', result => $doc }
{ status => $status, message => $message, timestamp => $timestamp }
Checks the status of Disbatch and QueueBalance.
If config.monitoring, calls check_disbatch() and check_queuebalance().
config.monitoring
check_disbatch()
check_queuebalance()
Returns { disbatch => check_disbatch() , queuebalance => check_queuebalance() } if config.monitoring is true, otherwise { disbatch => { status => 'OK', message => 'monitoring disabled' }, queuebalance => $checks->{queuebalance} = { status => 'OK', message => 'monitoring disabled' } }.
{ disbatch => check_disbatch() , queuebalance => check_queuebalance() }
{ disbatch => { status => 'OK', message => 'monitoring disabled' }, queuebalance => $checks->{queuebalance} = { status => 'OK', message => 'monitoring disabled' } }
Parameters: MongoDB::Collection.
MongoDB::Collection
Returns an ARRAY of ARRAYs of current indexes for the given collection.
Note: _id is turned into id because of Template.
Parameters: MongoDB query params HASH, current existsing collection indexes HASH
Returns a list of all params passed which do not match the given indexes. If the list is empty, the params are good.
Note: only looks at keys in $params, not their values.
$params
Parameters: HASH form parameters for a MongoDB query, ARRAY of index keys whose values are always ObjectIds, excluding _id.
Turns fields from an HTTP request into a query suitable for MongoDB::Collection.
Skips key/value pairs where the value is the empty string.
If a key is id or is in $oid_keys, turns the value(s) which should be hex strings into MongoDB::OID objects.
$oid_keys
Otherwise if a value (or first element of an ARRAY value) looks like a number, ensures the value (or elements) is a Perl number.
Any values which are ARRAYs are turned into queries joined by $or.
$or
If more than one key/value pair, they are joined into an $and query.
$and
Returns a query to pass a MongoDB::Collection object.
Performs a MongoDB query (count or find).
count
find
Parameters: HTTP params (HASH), options (HASH), title (string), OID keys (ARRAY), MongoDB::Collection object, form action path (string), return raw result (boolean), indexes (ARRAY of arrays).
Form action path should be from request->{path}.
request->{path}
Options can be .count, .fields to return, .limit, and .skip.
.count
.fields
.limit
.skip
Raw and indexes key are optional -- raw defaults to 0, and indexes are queried if undef.
undef
Returns the result of the query as a HASH or ARRAY, or an error HASH.
NOTE: I hate this code. Read it to determine the formats it might return.
NOTE: all JSON routes use send_json_options, documented above.
send_json_options
Parameters: none.
Returns an object with the following fields: database (the name of the MongoDB database used), web_extensions (an array of configured web extensions for custom routes), and routes (an object where fields are HTTP verbs and values are routes in the ordered configured).
database
web_extensions
routes
Note: new in Disbatch 4.2
Returns an Array of node Objects defined (with id the stringified _id) on success, { "error": "Could not get current nodes: $_" } on error.
{ "error": "Could not get current nodes: $_" }
Sets HTTP status to 400 on error.
Note: new in Disbatch 4
URL: :node is the _id if it matches /\A[0-9a-f]{24}\z/, or node name if it does not.
:node
/\A[0-9a-f]{24}\z/
node
Returns node Object (with id the stringified _id) on success, { "error": "Could not get node $node: $_" } on error.
{ "error": "Could not get node $node: $_" }
Parameters: { "maxthreads": maxthreads }
{ "maxthreads": maxthreads }
"maxthreads" is a non-negative integer or null
Returns { ref $res: Object } or { ref $res: Object, "error": error_string_or_reponse_object }
{ ref $res: Object }
{ ref $res: Object, "error": error_string_or_reponse_object }
Returns an Array of allowed plugin names.
Should never fail.
Note: replaces /queue-prototypes-json
Returns an Array of queue Objects on success, { "error": "Could not get current queues: $_" } on error.
{ "error": "Could not get current queues: $_" }
Each item has the following keys: id, plugin, name, threads, queued, running, completed
Note: replaces /scheduler-json
URL: :queue is the _id if it matches /\A[0-9a-f]{24}\z/, or name if it does not.
:queue
name
Returns a queue Object on success, { "error": "Could not get current queues: $_" } on error.
Create a new queue.
Parameters: { "name": name, "plugin": plugin }
{ "name": name, "plugin": plugin }
name is the desired name for the queue (must be unique), plugin is the plugin name for the queue.
plugin
Returns: { ref $res: Object, "id": $inserted_id } on success; { "error": "name and plugin required" }, { "error": "Invalid param", "param": $param }, or { "error": "Unknown plugin", "plugin": $plugin } on input error; or { ref $res: Object, "id": null, "error": "$res" } on MongoDB error.
{ ref $res: Object, "id": $inserted_id }
{ "error": "name and plugin required" }
{ "error": "Invalid param", "param": $param }
{ "error": "Unknown plugin", "plugin": $plugin }
{ ref $res: Object, "id": null, "error": "$res" }
Note: replaces /start-queue-json
Parameters: { "name": name, "plugin": plugin, "threads": threads }
{ "name": name, "plugin": plugin, "threads": threads }
name is the new name for the queue (must be unique), plugin is the new plugin name for the queue (must be defined in the config file), threads must be a non-negative integer. Only one of name, plugin, and threads is required, but any combination is allowed.
threads
Returns { ref $res: Object } or { "error": error }
{ "error": error }
Note: replaces /set-queue-attr-json
Deletes the specified queue.
Returns: { ref $res: Object } on success, or { ref $res: Object, "error": "$res" } on error.
{ ref $res: Object, "error": "$res" }
Note: replaces /delete-queue-json
Parameters: anything indexed on the tasks collection, as well as any dot options.
tasks
Options can be .count, .fields to return, query .limit and .skip, .terse or .full output, dates as .epoch, and .pretty print JSON result.
.pretty
Performs a search of tasks, returning either JSON or a web page.
If want_json() (based on the Accept header), returns a JSON array (which may be pretty-printed if specified in the parameters) of task documents, or on error an object with an error field (and possibly other fields).
want_json()
error
Otherwise, if no parameters returns a web form to perform a search of indexed fields. If parameters, returns a web page of results or error.
Note: new in 4.2, replaces POST /tasks/search
POST /tasks/search
Parameters: Task OID in URL
Returns the task matching OID as JSON, or { "error": "no task with id :id" } and status 404 if OID not found. Or, via a web browser (based on Accept header value), returns the task matching OID with some formatting, or No tasks found matching query if OID not found.
{ "error": "no task with id :id" }
404
No tasks found matching query
Parameters: { "queue": queue, "params": [single_task_params, another_task_params, ...] } or { "queue": queue, "params": generic_task_params, "collection": collection, "filter": filter }.
{ "queue": queue, "params": [single_task_params, another_task_params, ...] }
{ "queue": queue, "params": generic_task_params, "collection": collection, "filter": filter }
queue is the _id if it matches /\A[0-9a-f]{24}\z/, or name if it does not.
queue
collection is a MongoDB collection name.
collection
filter is a filter expression (query) object for the :collection collection.
filter
:collection
params depends on if passing a collection and filter or not.
If not, params is an array of objects, each of which will be inserted as-is as the params value in created tasks.
Otherwise, params is an object of generic task params. To insert a document value from a query into the params, prefix the desired key name with document. as a value.
document.
Returns: { ref $res: Object } on success; { "error": "params must be a JSON array of task params" }, { "error": "filter and params required and must be name/value objects" } or { "error": "queue not found" } on input error; { "error": "Could not iterate on collection $collection: $error" } on query error, or { ref $res: Object, "error": "Unknown error" } on MongoDB error.
{ "error": "params must be a JSON array of task params" }
{ "error": "filter and params required and must be name/value objects" }
{ "error": "queue not found" }
{ "error": "Could not iterate on collection $collection: $error" }
{ ref $res: Object, "error": "Unknown error" }
Note: new in 4.2, replaces POST /tasks/:queue and POST /tasks/:queue/:collection
POST /tasks/:queue
POST /tasks/:queue/:collection
Returns a web page to view and update Queue Balance settings if the Accept header wants text/html, otherwise returns a pretty JSON result of get_balance
get_balance
Parameters: { "max_tasks": max_tasks, "queues": queues, "disabled": disabled }
{ "max_tasks": max_tasks, "queues": queues, "disabled": disabled }
max_tasks is a HASH where keys match /^[*0-6] (?:[01]\d|2[0-3]):[0-5]\d$/ (that is, 0..6 or * for DOW, followed by a space and a 24-hour time) and values are non-negative integers.
max_tasks
/^[*0-6] (?:[01]\d|2[0-3]):[0-5]\d$/
0..6
*
queues is an ARRAY of ARRAYs of queue names which must exist
queues
disabled is a timestamp which must be in the future (optional)
disabled
Sets the balance document fields given in the above parameters to the given values.
Returns JSON {"status":"success: queuebalance modified"} on success, or {"status":"failed: invalid json passed " . $_} with HTTP status of 400 on error.
{"status":"success: queuebalance modified"}
{"status":"failed: invalid json passed " . $_}
Monitoring is controlled by setting config.monitoring and config.balance.enabled.
Returns as JSON the result of checks(), documented above.
checks()
You can set an object of package names and arguments (can be null) to web_extensions in the config file to load any custom routes and call init($disbatch, $arguments) if available. Note that if a request which matches your custom route is also matched by an above route, your custom route will never be called. If a custom route package needs to interface with Disbatch or have any arguments passed to it, it should have the following:
null
init($disbatch, $arguments)
my $disbatch; sub init { ($disbatch, my $args) = @_; # do whatever you may need to do with $args }
For examples see Disbatch::Web::Files (which is automatically loaded at the end of init(), after any custom routes) and Disbatch::Web::Tasks (not loaded by default).
init(), after any custom routes) and Disbatch::Web::Tasks (not loaded by default).
Returns the contents of "/index.html" – the queue browser page.
Returns the contents of the request path.
Note: this is loaded from Disbatch::Web::Files.
Disbatch
Disbatch::Roles
Disbatch::Plugin::Demo
disbatchd
disbatch.pl
task_runner
disbatch-create-users
Ashley Willis <awillis@synacor.com>
Matt Busigin
This software is Copyright (c) 2015, 2016, 2019 by Ashley Willis.
This is free software, licensed under:
The Apache License, Version 2.0, January 2004
1 POD Error
The following errors were encountered while parsing the POD:
Unterminated C<...> sequence
To install Disbatch, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Disbatch
CPAN shell
perl -MCPAN -e shell install Disbatch
For more information on module installation, please visit the detailed CPAN module installation guide.