Proc::JobQueue - generic job queue base class
use Proc::JobQueue; $queue = Proc::JobQueue->new(%parameters); $queue->addhost($host, %parameters); $queue->add($job); $queue->add($job, $host); $queue->startmore(); $queue->hold($new_value); $queue->checkjobs(); $queue->jobdone($job, $do_startmore, @exit_code); $queue->alldone() $queue->status() $queue->startjob($host, $jobnum, $job);
Generic queue of "jobs". Most likely to be subclassed for different situations. Jobs are registered. Hosts are registered. Jobs may or may not be tied to particular hosts. Jobs are started on hosts.
Jobs are started with:
$job->jobnum($jobnum); $jobnum = $job->jobnum(); $job->queue($queue); $job->host($host); $job->start();
When jobs complete, they must call:
$queue->jobdone($job, $do_startmore, @exit_code);
Hosts are added with:
$queue->addhost($host, name => $hostname, jobs_per_host => $number_to_run_on_this_host)
The parameters for new are:
new
Default number of jobs to run on each host simultaneously. This can be overridden on a per-host basis.
If any one host has more than this many jobs waiting for it, no can-run-on-any-host jobs will be started. This is to prevent the queue for this one overloaded host from getting too large.
This is the starting job number. Job numbers are sometimes displayed. They increment for each new job.
If true, prevent any jobs from starting until $queue->hold(0) is called.
$queue->hold(0)
Adjusts the same parameters that can be set with new.
Register a new host. Parameters are:
The hostname
The number of jobs that can be run at once on this host. This defaults to the jobs_per_host parameter of the $queue.
$queue
Add a job object to the queue. The job object must be a Proc::JobQueue::Job or subclass of Proc::JobQueue::Job. The $host parameter is optional: if not set, the job can be run on any host.
$host
The $job object is started with:
$job
When the job complets, it must call:
When jobs complete, they must call jobdone. If $do_startmore is true, then startmore() will be called. A true exit code signals an error and it is used by Proc::JobQueue::CommandQueue.
$do_startmore
startmore()
This checks the job queue. It returns true if all jobs have completed and the queue is empty.
This prints a queue status to STDERR showing what's running on which hosts. Printing is supressed unless $Proc::JobQueue::status_frequency seconds have passed since the last call to status().
status()
This will start more jobs if possible. The return value is true if there are no more jobs to start.
This starts a single job. It is used by startmore() and probably should not be used otherwise.
This is used be Proc::JobQueue::BackgroundQueue.
Get (or set if $new_value is defined) the queue's hold-all-jobs parameter.
Proc::JobQueue needs canonical hostnames. It gets them by default with Proc::JobQueue::CanonicalHostnames. You can override this default by overriding $Proc::JobQueue::host_canonicalizer with the name of a perl module to use instead of Proc::JobQueue::CanonicalHostnames.
$Proc::JobQueue::host_canonicalizer
Helper functions are provided by Proc::JobQueue and are available via explicit import:
use Proc::JobQueue qw(my_hostname canonicalize is_remote_host);
Proc::JobQueue::Job Proc::JobQueue::DependencyQueue Proc::JobQueue::BackgroundQueue
Copyright (C) 2007-2008 SearchMe, Inc. Copyright (C) 2008-2010 David Sharnoff. Copyright (C) 2011 Google, Inc. This package may be used and redistributed under the terms of either the Artistic 2.0 or LGPL 2.1 license.
To install Proc::JobQueue, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Proc::JobQueue
CPAN shell
perl -MCPAN -e shell install Proc::JobQueue
For more information on module installation, please visit the detailed CPAN module installation guide.