MediaCloud::JobManager::Job - An abstract class for a "function".
MediaCloud::JobManager::Job
function
A function to be run by locally or remotely, e.g. add_default_feeds.
add_default_feeds
job
An instance of the function doing the actual job with specific parameters.
The following subroutines must be implemented by the subclasses of this class.
run($self, $args)
Run the job.
Parameters:
$self, a reference to the instance of the function class
$self
(optional) $args (hashref), arguments needed for running the function
$args
An instance (object) of the class will be created before each run. Class instance variables (e.g. $self->_my_variable) will be discarded after each run.
$self->_my_variable
Returns result on success (serializable by the JSON module). The result will be discarded if the job is added as a background process.
Provides progress reports when available:
by calling $self->set_progress($numerator, $denominator)
$self->set_progress($numerator, $denominator)
die()s on error.
die()
Writes log to STDOUT or STDERR (preferably the latter).
STDOUT
STDERR
retries()
Return the number of retries for each job.
Returns a number of retries each job will be attempted at. For example, if the number of retries is set to 3, the job will be attempted 4 four times in total.
Returns 0 if the job should not be retried (attempted only once).
Default implementation of this subroutine returns 0 (no retries).
lazy_queue()
Return true if RabbitMQ should create a "lazy" queue for this function.
Returns true if the job queue is expected to grow very large so RabbitMQ should create a "lazy" queue (https://www.rabbitmq.com/lazy-queues.html) for this type of job.
Default implementation of this subroutine returns 0 ("default" type of queue).
publish_results()
Return true if worker should publish results back to a results RabbitMQ queue.
Returns true if client that added job to the queue might be interested in the results of the job (whether or not it has failed, what has run() returned) so RabbitMQ should keep a result of the job and send it back to client when requested.
One might want to disable this if distinct results of many jobs aren't that important and you'd like to make job broker a little bit faster.
This subroutine will only be used when calling add_to_queue().
Default implementation of this subroutine returns 1 (results will be collected, stored and sent back to clients if requested).
configuration()
Return an instance or a subclass of MediaCloud::JobManager::Configuration to be used as default configuration by both workers and clients.
MediaCloud::JobManager::Configuration
Workers and clients will still be able to override this configuration by passing their own config argument. This configuration will be used if no such argument is present.
config
Default implementation of this subroutine returns an instance of MediaCloud::JobManager::Configuration (default configuration).
Jobs in a single queue can have different priorities ("low", "normal" or "high") in order for them to be run in desirable order:
$MJM_JOB_PRIORITY_LOW, if the job is considered of "low priority".
$MJM_JOB_PRIORITY_LOW
$MJM_JOB_PRIORITY_NORMAL if the job is considered of "normal priority".
$MJM_JOB_PRIORITY_NORMAL
$MJM_JOB_PRIORITY_HIGH if the job is considered of "high priority".
$MJM_JOB_PRIORITY_HIGH
run_remotely() and add_to_queue() both accept the job priority argument.
run_remotely()
add_to_queue()
By default, jobs are being run with a "normal" priority.
The following subroutines can be used by the deriving class.
Provide progress report while running the task (from run()).
run()
Examples:
$self->set_progress(3, 10)
3 out of 10 subtasks are complete.
$self->set_progress(45, 100)
45 out of 100 subtasks are complete (or 45% complete).
The following subroutines can be used by clients to run a function.
$class->run_locally([$args, $config])
Run locally and right away, blocking the parent process until the job is finished.
(optional) $args (hashref), arguments required for running the function (serializable by the JSON module)
(optional, internal) job handle to be later used by send_progress()
Returns result (may be false of undef) on success, die()s on error
undef
$class->run_remotely([$args])
Run remotely, wait for the task to complete, return the result; block the process until the job is complete.
(optional) $args (hashref), arguments needed for running the function (serializable by the JSON module)
$class->add_to_queue([$args, $config])
Add to queue remotely, do not wait for the task to complete, return immediately; do not block the parent process until the job is complete.
Returns job ID if the job was added to queue successfully, die()s on error.
name()
Returns function's name (e.g. NinetyNineBottlesOfBeer).
NinetyNineBottlesOfBeer
Usage:
NinetyNineBottlesOfBeer->name();
Class or class instance
To install MediaCloud::JobManager, copy and paste the appropriate command in to your terminal.
cpanm
cpanm MediaCloud::JobManager
CPAN shell
perl -MCPAN -e shell install MediaCloud::JobManager
For more information on module installation, please visit the detailed CPAN module installation guide.