The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.


Parallel::parallel_map - map in parallel using all CPU cores


  use Parallel::parallel_map;
  $Parallel::parallel_map::INIT_DATA_PROCESSOR = \&srand; # seed random generator for each process
  my @m2 = parallel_map {$_*2} 1..10000000;


It does the same as map in perl, but using all CPU/cores. Take into account that each iteration is executed as separate process i.e. changes of memory inside iterator will not affect memory state of main thread. Still there is some overhead on IPC, so to calculate $_*2 in parallel does not make practical sense. Although if your iteration is more complex then that - you will win. Do benchmarks to figure out.

It is implemented using Parallel::DataPipe.




I have found Parallel::Iterator module at CPAN which basically does the same thing. But it has much bigger overhead on IPC than this module.

 perl -MParallel::Iterator=iterate_as_array -MBenchmark -e'@a=1..10000;timethis(10,q{@m2=iterate_as_array(sub {$_[1]*2}, \@a)})'
 timethis 10:  4 wallclock secs ( 2.85 usr  0.67 sys +  2.83 cusr  0.83 csys =  7.19 CPU) @  1.39/s (n=10)

 perl -MParallel::parallel_map -MBenchmark -e'timethis(10,q{@m2=parallel_map {$_*2} 1..10000_00})'
 timethis 10:  5 wallclock secs ( 2.81 usr  0.19 sys +  4.51 cusr  2.13 csys =  9.64 CPU) @  1.04/s (n=10)

It's almost 100 times faster than Parallel::Iterator on this simple iteration.


Parallel::DataPipe, Parallel::Loops, MCE, POE, Parallel::Iterator


Oleksandr Kharchenko, <>


Copyright (C) 2013 by Oleksandr Kharchenko

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.16.3 or, at your option, any later version of Perl 5 you may have available.