NAME

Parallel::Queue - fork or thread a list of closures N-way parallel

SYNOPSIS

    # example queue:
    # only squish files larger than 8KB in size.  figure
    # that the system can handle four copies of squish
    # running at the same time without them interfering
    # with one another.

    my @queue = map { -s > 8192 ? sub{ squish $_ } : () } @filz;

    # functional: pass in the count and list of coderefs.
    #
    # adding 'runqueue' exports the subroutine into
    # the current package. useful for non-OO situations.
    #
    # run the queue 4 way parallel.

    use Parallel::Queue qw( runqueue verbose fork );

    runqueue 4, @queue;

    die "Incomplete jobs" if @queue;

    # OO: generate queue manager and use without the 
    # 'runqueue' arguments, construct a queue manager,
    # and use it to run the jobs

    use Parallel::Queue;

    my $quemgr = Parallel::Queue->construct( thread );

    $quemgr->runqueue( 4, @queue );

    die "Incomplete jobs" if @queue;

    # call Parallel::Queue with the default configuration
    # (fork quietly).

    require Parallel::Queue;

    Parallel::Queue->runqueue( 4, @queue );

    # pre-define defaults for the objects: leave
    # out runqueue, set the rest, and construct 
    # an object. the one here gets verbose, thread,
    # and debug all set to true.

    use Parallel::Queue qw( verbose thread );

    my $quemgr = Parallel::Queue->construct( debug );

    $quemgr->runqueue( 4, @queue );

    # avoid forking with a zero job count.
    # this dispatches the closures one by one
    # via $sub->() for easier debugging.
    #
    # this mode is also used regardless of 
    # the job count if $^P is set (i.e., 
    # if the que is run through the debugger).

    runqueue 0, @queue;

    $quemgr->runqueue( 0, @queue );

DESCRIPTION

Given a count and an array of coderefs (most likely closures), runqueue will run the jobs in parallel. The jobs can be run via fork or detached threads [see known issues for threading]. Jobs on the queue are executed until one of them exits non-zero, the fork/thread operation fails, or all of them are dispatched (i.e., the queue is empty).

Functional

Parallel::Queue does not export the runqueue function by default. Adding 'runqueue' to the argument list exports the subroutine into the caller's space and additinally stores the remaining arguments indexed by the caller's package (i.e., multiple modules can use Parallel::Queue and get different behavior).

    # get the subroutine installed locally, run the
    # queue 8-ways parallel.

    use Parallel::Queue qw( runqueue verbose );

    runqueue 8, @queue;

Objective

If runqueue is not exported it can be accessed as an object or class method. Objects are constructed with the same arguments as use, after which runqueue is called via the object. This allows pushing things like fork vs. thread decisions into runtime code:

    use Parallel::Queue;

    ...

    my $how = $threadly ? 'thread' : 'fork';

    ...

    my $que_mgr = Parallel::Queue->construct( $how, $verbose );

    $que_mgr->runqueue( 8, @queue );

If runqueue is called as a class method then it will run with the default configuration: forking quietly.

    require Parallel::Queue;

    Parallel::Queue->runqueue( 8, @queue );

this is the equivalent of using:

    use Parallel::Queue qw( runqueue );

    runqueue  8, @queue;

but may be easier in cases where the module is being included at runtime (usually via require).

Constructor defaults via use.

Arguments passed to use are installed as defaults for the calling class. This allows a class to determine defaults for the objects created, which can be useful for reducing the amount of data thrown around in the construction.

    # default has verbose turned on.

    use Parallel::Queue qw( verbose );

    ...

    # gets verbose and fork turned on.

    my $quemgr = Parallel::Queue->construct( fork ); 

KNOWN ISSUES

Non-numeric count arguments.

The runqueue sub uses Scalar::Util::looks_like_number to determine if it is being called as a function or method. Basically, if the first argument looks like a number then it isn't being used via an OO call.

If a class that doesn't look like a number with numeric overloading is used for the count then this could cause problems. This will show up as runqueue complaining that the first argument is not a Parallel::Queue (i.e., that $_[0]->isa( __PACKAGE__ ) is false).

Threads are unstested.

Forks work; threads are still in progress. The code may work but I have not tested it yet.

SEE ALSO

Debugging forks.

<http://qs321.pair.com/~monkads/index.pl?node_id=128283>

COPYRIGHT

This code is released under the same terms as Perl-5.8 or any later version of Perl.

AUTHOR

Steven Lembark <lembark@wrkhors.com>