MCE::Hobo - MCE model for running code asynchronously
This document describes MCE::Hobo version 1.699_002
use MCE::Hobo; MCE::Hobo->create( sub { print "Hello from hobo\n" } )->join(); sub parallel { my ($arg1) = @_; print "Hello again, $arg1\n"; } MCE::Hobo->create( \¶llel, $_ ) for 1 .. 3; my @hobos = MCE::Hobo->list(); my @running = MCE::Hobo->list_running(); my @joinable = MCE::Hobo->list_joinable(); $_->join() for @hobos; my $hobo = mce_async { foreach (@files) { ... } }; $hobo->join(); if (my $err = $hobo->error()) { warn("Hobo error: $err\n"); } # Get a hobo's object $hobo = MCE::Hobo->self(); # Get a hobo's ID $tid = MCE::Hobo->tid(); # "$$.tid" $tid = $hobo->tid(); $pid = MCE::Hobo->pid(); # $$ $pid = $hobo->pid(); # Test hobo objects if ($hobo1 == $hobo2) { ... } # Give other hobos a chance to run MCE::Hobo->yield(); MCE::Hobo->yield(0.05); # Wantarray-aware return context my ($value1, $value2) = $hobo->join(); my $value = $hobo->join(); # Check hobo's state if ($hobo->is_running()) { sleep 1; } if ($hobo->is_joinable()) { $hobo->join(); } # Send a signal to a hobo $hobo->kill('SIGUSR1'); # Exit a hobo MCE::Hobo->exit();
MCE::Hobo provides threads-like capability for running code asynchronously via child processes. The underlying IPC is managed by MCE::Shared, which runs on all major platforms including Cygwin.
MCE::Hobo
threads
MCE::Shared
This module may be used as a standalone or together with MCE including running alongside threads.
The following is a parallel demonstration.
use strict; use warnings; use MCE::Hobo; use MCE::Shared Sereal => 1; # Serialization via Sereal if available. use MCE::Shared::Ordhash; # Ordhash for non-shared use below. # usage: head -20 file.txt | perl script.pl # my $aa = MCE::Shared->array(); # OO with on-demand deref @{} # my $ha = MCE::Shared->hash(); # OO with on-demand deref %{} # my $oh = MCE::Shared->ordhash(); # OO with on-demand deref %{} # my $va = MCE::Shared->scalar(); # OO interface only: ->set, ->get my $ifh = MCE::Shared->handle( "<", \*STDIN ); # shared my $ofh = MCE::Shared->handle( ">", \*STDOUT ); my $ary = MCE::Shared->array(); sub parallel_task { my ( $id ) = @_; while ( <$ifh> ) { printf {$ofh} "[ %4d ] %s", $., $_; # $ary->{ $. - 1 } = "[ ID $id ] read line $.\n" ); # on-demand $ary->set( $. - 1, "[ ID $id ] read line $.\n" ); # OO faster } } my $hobo1 = MCE::Hobo->new( "parallel_task", 1 ); my $hobo2 = MCE::Hobo->new( \¶llel_task, 2 ); my $hobo3 = MCE::Hobo->new( sub { parallel_task(3) } ); $_->join for MCE::Hobo->list(); my $search = MCE::Shared::Ordhash->new( # non-shared $ary->find( "val =~ / ID 2 /" ) # search array (one IPC call) ); # print {*STDERR} join( "", values %{ $search } ); # on-demand print {*STDERR} join( "", $search->values ); # OO faster
The following list options which may be overridden when loading the module.
use Sereal qw( encode_sereal decode_sereal ); use CBOR::XS qw( encode_cbor decode_cbor ); use JSON::XS qw( encode_json decode_json ); use MCE::Hobo freeze => \&encode_sereal, ## \&Storable::freeze thaw => \&decode_sereal ## \&Storable::thaw ;
There is a simpler way to enable Sereal. The following will attempt to use Sereal if available, otherwise defaults to Storable for serialization.
use MCE::Hobo Sereal => 1; use MCE::Shared Sereal => 1; # <-- supports Sereal only, at this time
This will create a new hobo that will begin execution with function as the entry point, and optionally ARGS for list of parameters. It will return the corresponding MCE::Hobo object, or undef if hobo creation failed.
FUNCTION may either be the name of a function, an anonymous subroutine, or a code ref.
my $hobo = MCE::Hobo->create( "func_name", ... ); # or my $hobo = MCE::Hobo->create( sub { ... }, ... ); # or my $hobo = MCE::Hobo->create( \&func, ... ); The C<new()> method is an alias for C<create()>.
mce_async runs the block asynchronously similarly to MCE::Hobo-create()>. It returns the hobo object, or undef if hobo creation failed.
mce_async
MCE::Hobo-
my $hobo = mce_async { foreach (@files) { ... } }; $hobo->join(); if (my $err = $hobo->error()) { warn("Hobo error: $err\n"); }
This will wait for the corresponding hobo to complete its execution. In non-voided context, join() will return the value(s) of the entry point function.
join()
The context (void, scalar or list) for the return value(s) for join is determined at the time of joining and mostly wantarray-aware.
join
my $hobo1 = MCE::Hobo->create( sub { my @results = qw(foo bar baz); return (@results); }); my @res1 = $hobo1->join; # ( foo, bar, baz ) my $res1 = $hobo1->join; # baz my $hobo2 = MCE::Hobo->create( sub { return 'foo'; }); my @res2 = $hobo2->join; # ( foo ) my $res2 = $hobo2->join; # foo
Tests if two hobo objects are the same hobo or not. This is overloaded to the more natural forms:
if ($hobo1 == $hobo2) { print("Hobos are the same\n"); } # or if ($hobo1 != $hobo2) { print("Hobos differ\n"); }
(Hobo comparison is based on process IDs.)
Hobos are executed in an eval context. This method will return undef if the hobo terminates normally. Otherwise, it returns the value of $@ associated with the hobo's execution status in its eval context.
eval
undef
$@
This sends 'SIGQUIT' to the hobo, notifying the hobo to exit. It returns the hobo object to allow for method chaining.
'SIGQUIT'
$hobo->exit()->join();
A hobo can be exited at any time by calling MCE::Hobo-exit()>. This behaves the same as exit(status) when called from the main process.
exit(status)
This class method is called automatically by END, but may be called explicitly. Two shared objects to MCE::Shared are destroyed. An error is emmitted via croak if there are active hobos not yet joined.
END
MCE::Hobo->create( 'task1', $_ ) for 1 .. 4; $_->join for MCE::Hobo->list(); MCE::Hobo->create( 'task2', $_ ) for 1 .. 4; $_->join for MCE::Hobo->list(); MCE::Hobo->create( 'task3', $_ ) for 1 .. 4; $_->join for MCE::Hobo->list(); MCE::Hobo->finish();
Returns true if a hobo is still running.
Returns true if the hobo has finished running and not yet joined.
Sends the specified signal to the hobo. Returns the hobo object to allow for method chaining.
$hobo->kill('SIG...')->join();
Returns a list of all hobos not yet joined.
Returns a list of all hobos that are still running.
Returns a list of all hobos that have completed running. Thus, ready to be joined without blocking.
Class method that allows a hobo to obtain it's own MCE::Hobo object.
Returns the ID of the hobo. TID is composed of process and thread IDs together as a string value.
PID: $$ TID: "$$.tid"
Class methods that allows a hobo to obtain its own ID.
Let this hobo yield CPU time to other hobos. By default, the class method calls sleep(0.0002) on Unix including Cygwin and sleep(0.001) on Windows.
sleep(0.0002)
sleep(0.001)
MCE::Hobo->yield(); MCE::Hobo->yield(0.05);
The inspiration for MCE::Hobo comes from wanting threads-like behavior for processes. Both can run side-by-side including safe-use by MCE workers. Likewise, the documentation resembles threads.
DBM::Deep
forks
forks::BerkeleyDB
Thread::Tie
MCE, MCE::Core, MCE::Shared
Mario E. Roy, <marioeroy AT gmail DOT com>
To install MCE, copy and paste the appropriate command in to your terminal.
cpanm
cpanm MCE
CPAN shell
perl -MCPAN -e shell install MCE
For more information on module installation, please visit the detailed CPAN module installation guide.