++ed by:
RSCHABER JGAMBLE KEEDI
2 non-PAUSE users
Author image Salvador Fandiño García

NAME

Array::Shuffle - fast shuffling of arrays in-place

SYNOPSIS

  use Array::Shuffle qw(shuffle_array);
  shuffle_array(@a);

DESCRIPTION

This module provide some functions for shuffling arrays in-place efficiently.

API

shuffle_array @a

Shuffles the given array in-place using the Fisher-Yates algorithm that is O(N).

This function is an order of magnitude faster than the shuffle function from List::Util.

Note: that was true a long, long, long time ago. If you are worried about performance, you should check it for yourself.

In most cases you should probably use "shuffle" in List::Utils instead of this obscure module!

shuffle_huge_array @a

Shuffles the given array in-place using an algorithm that is O(NlogN) but more cache friendly than Fisher-Yates. In some extreme cases, when shuffling huge arrays that do not find in the available RAM it may perform better.

You would like to do some benchmarking to find out which one is better suited for your particular case.

SEE ALSO

List::Util.

The following thread on PerlMonks for a discussion on the topic: http://perlmonks.org/?node_id=953607.

COPYRIGHT AND LICENSE

Copyright (C) 2012, 2021 by Salvador Fandiño (sfandino@yahoo.com)

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.14.2 or, at your option, any later version of Perl 5 you may have available.

BENCHMARKS

Included in this package you can find the script samples/benchmark.pl that compares the performance of the following shuffle algorithms and implementations:

pp

uses a Fisher Yates shuffle implemented in pure perl

ls

uses the shuffle function from List::Util

sa

uses the shuffle_array method from this module

sha

uses the shuffle_huge_array method from this module

What follows is the output of this script running on an i386 virtual machine with 64MB of RAM and several GB of swap running Ubuntu.

Note that the algorithms are selectively switched down when they become too slow and in the sa case, when it starts using swap memory as at this point it just becomes unbearably slow.

  Generating array with 100 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+     0:00     21  1416  5843  3076  5.6 benchmark.pl
        Rate    pp    lu   sha    sa
  pp   354/s    --  -77%  -94%  -94%
  lu  1534/s  333%    --  -73%  -73%
  sha 5697/s 1509%  271%    --   -0%
  sa  5697/s 1509%  271%    0%    --
  Generating array with 158 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+     0:57     21  1416  5843  3212  5.8 benchmark.pl
        Rate    pp    lu   sha    sa
  pp   188/s    --  -83%  -95%  -95%
  lu  1109/s  489%    --  -69%  -72%
  sha 3618/s 1821%  226%    --  -10%
  sa  4003/s 2025%  261%   11%    --
  Generating array with 251 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+     1:34     21  1416  5843  3212  5.8 benchmark.pl
        Rate    pp    lu   sha    sa
  pp   333/s    --  -76%  -94%  -95%
  lu  1365/s  310%    --  -77%  -78%
  sha 5843/s 1656%  328%    --   -7%
  sa  6263/s 1783%  359%    7%    --
  Generating array with 398 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+     2:50     21  1416  5843  3224  5.8 benchmark.pl
        Rate    pp    lu    sa   sha
  pp   260/s    --  -79%  -92%  -94%
  lu  1227/s  371%    --  -61%  -72%
  sa  3176/s 1120%  159%    --  -28%
  sha 4402/s 1591%  259%   39%    --
  Generating array with 630 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+     3:34     21  1416  5843  3224  5.8 benchmark.pl
        Rate    pp    lu    sa   sha
  pp   309/s    --  -80%  -95%  -95%
  lu  1540/s  398%    --  -74%  -75%
  sa  5966/s 1830%  287%    --   -1%
  sha 6054/s 1858%  293%    1%    --
  Generating array with 1000 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+     4:24     21  1416  5843  3224  5.8 benchmark.pl
        Rate    pp    lu    sa   sha
  pp   214/s    --  -74%  -93%  -94%
  lu   823/s  285%    --  -73%  -77%
  sa  3000/s 1304%  265%    --  -16%
  sha 3583/s 1577%  336%   19%    --
  Generating array with 1584 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+     4:47     21  1416  5979  3280  5.9 benchmark.pl
        Rate    pp    lu   sha    sa
  pp  76.9/s    --  -85%  -94%  -96%
  lu   526/s  584%    --  -58%  -71%
  sha 1245/s 1520%  137%    --  -31%
  sa  1796/s 2238%  242%   44%    --
  Generating array with 2511 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+     5:08     21  1416  5979  3280  5.9 benchmark.pl
        Rate    pp    lu   sha    sa
  pp  48.1/s    --  -82%  -94%  -94%
  lu   271/s  464%    --  -66%  -67%
  sha  801/s 1565%  195%    --   -4%
  sa   834/s 1634%  207%    4%    --
  Generating array with 3981 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+     5:29     21  1416  6111  3388  6.1 benchmark.pl
        Rate    pp    lu    sa   sha
  pp  48.0/s    --  -79%  -95%  -95%
  lu   228/s  375%    --  -75%  -78%
  sa   921/s 1818%  304%    --  -13%
  sha 1059/s 2105%  364%   15%    --
  Generating array with 6309 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+     5:52     21  1416  6243  3508  6.3 benchmark.pl
        Rate    pp    lu    sa   sha
  pp  32.9/s    --  -77%  -93%  -94%
  lu   140/s  327%    --  -68%  -76%
  sa   445/s 1254%  217%    --  -24%
  sha  587/s 1684%  318%   32%    --
  Generating array with 10000 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+     6:17     21  1416  6407  3608  6.5 benchmark.pl
        Rate    pp    lu    sa   sha
  pp  20.4/s    --  -68%  -92%  -95%
  lu  62.8/s  208%    --  -75%  -85%
  sa   254/s 1147%  305%    --  -38%
  sha  413/s 1925%  557%   62%    --
  Generating array with 15848 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+     6:41     21  1416  6547  3880  7.0 benchmark.pl
        Rate    pp    lu    sa   sha
  pp  14.7/s    --  -76%  -94%  -96%
  lu  61.1/s  317%    --  -77%  -83%
  sa   263/s 1696%  331%    --  -25%
  sha  353/s 2307%  478%   34%    --
  Generating array with 25118 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+     7:12     21  1416  7023  4288  7.8 benchmark.pl
        Rate    pp    lu    sa   sha
  pp  9.26/s    --  -74%  -94%  -95%
  lu  35.5/s  283%    --  -78%  -82%
  sa   161/s 1644%  355%    --  -20%
  sha  202/s 2081%  469%   25%    --
  Generating array with 39810 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+     7:36     21  1416  7843  5052  9.2 benchmark.pl
        Rate    pp    lu   sha    sa
  pp  5.77/s    --  -74%  -94%  -95%
  lu  21.8/s  278%    --  -78%  -80%
  sha  100/s 1633%  358%    --   -6%
  sa   107/s 1749%  389%    7%    --
  Generating array with 63095 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+     7:56     21  1416  8563  5888 10.7 benchmark.pl
        Rate    pp    lu   sha    sa
  pp  3.00/s    --  -72%  -94%  -95%
  lu  10.9/s  263%    --  -77%  -82%
  sha 46.7/s 1458%  329%    --  -23%
  sa  60.8/s 1927%  458%   30%    --
  Generating array with 100000 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+     8:13     21  1416 10379  7524 13.7 benchmark.pl
        Rate    pp    lu    sa   sha
  pp  2.21/s    --  -51%  -89%  -92%
  lu  4.46/s  102%    --  -78%  -84%
  sa  20.5/s  831%  360%    --  -26%
  sha 27.8/s 1159%  522%   35%    --
  Generating array with 158489 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+     8:31     21  1416 12919 10208 18.6 benchmark.pl
        Rate    pp    lu   sha    sa
  pp  1.59/s    --  -52%  -88%  -93%
  lu  3.31/s  108%    --  -76%  -85%
  sha 13.6/s  756%  311%    --  -37%
  sa  21.5/s 1254%  550%   58%    --
  Generating array with 251188 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+     8:56     21  1416 16623 13692 24.9 benchmark.pl
         Rate    pp    lu   sha    sa
  pp  0.826/s    --  -69%  -88%  -94%
  lu   2.65/s  221%    --  -61%  -81%
  sha  6.73/s  714%  154%    --  -51%
  sa   13.6/s 1550%  414%  103%    --
  Generating array with 398107 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+     9:16     28  1416 23611 20172 36.7 benchmark.pl
         Rate    pp    lu   sha    sa
  pp  0.714/s    --  -56%  -82%  -92%
  lu   1.64/s  130%    --  -59%  -82%
  sha  4.00/s  460%  144%    --  -55%
  sa   8.91/s 1148%  444%  123%    --
  Generating array with 630957 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+     9:44     45  1416 33775 30632 55.8 benchmark.pl
      s/iter    pp    lu   sha    sa
  pp    3.10    --  -59%  -77%  -92%
  lu    1.27  144%    --  -43%  -80%
  sha  0.725  328%   75%    --  -65%
  sa   0.255 1116%  398%  184%    --
  Generating array with 1000000 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+    11:35   5038  1416 48455 34376 62.6 benchmark.pl
      s/iter  sha   sa
  sha   1.21   -- -64%
  sa   0.440 175%   --
  Generating array with 1584893 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+    12:00   6007  1416 52551 32100 58.5 benchmark.pl
      s/iter  sha   sa
  sha   1.36   -- -53%
  sa   0.645 111%   --
  Generating array with 1995262 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+    12:11   6382  1416 60603 34600 63.0 benchmark.pl
      s/iter  sha   sa
  sha   2.02   -- -65%
  sa   0.700 189%   --
  Generating array with 2511886 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+    12:24   6885  1416 78959 35356 64.4 benchmark.pl
      s/iter  sha   sa
  sha   2.10   -- -61%
  sa   0.820 156%   --
  Generating array with 3162277 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+    12:41   7331  1416 91631 35200 64.1 benchmark.pl
      s/iter  sha   sa
  sha   2.79   -- -56%
  sa    1.22 129%   --
  Generating array with 3981071 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+    12:57   7918  1416 107735 35176 64.1 benchmark.pl
      s/iter  sha   sa
  sha   3.78   -- -62%
  sa    1.43 164%   --
  Generating array with 5011872 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+    13:28   9455  1416 144447 34692 63.2 benchmark.pl
      s/iter  sha   sa
  sha   5.11   -- -48%
  sa    2.67  91%   --
  Generating array with 6309573 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+    14:15  12666  1416 169923 35112 64.0 benchmark.pl
      s/iter  sha   sa
  sha   6.77   -- -69%
  sa    2.11 221%   --
  Generating array with 7943282 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+    15:06  16655  1416 202131 35140 64.0 benchmark.pl
      s/iter  sha   sa
  sha   8.26   -- -66%
  sa    2.84 191%   --
  Generating array with 10000000 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+    16:27  25072  1416 275291 34468 62.8 benchmark.pl
      s/iter sha
  sha   27.6  --
  Generating array with 12589254 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+    18:10  32251  1416 326243 34044 62.0 benchmark.pl
      s/iter sha
  sha   46.9  --
  Generating array with 15848931 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+    20:44  44029  1416 390395 33768 61.5 benchmark.pl
      s/iter sha
  sha   91.6  --
  Generating array with 19952623 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+    25:26  69013  1416 536583 32964 60.1 benchmark.pl
      s/iter sha
  sha    330  --
  Generating array with 25118864 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+    35:12 123761  1416 638223 33072 60.2 benchmark.pl
      s/iter sha
  sha    393  --
  Generating array with 31622776 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+    50:09 248018  1416 766131 33504 61.0 benchmark.pl
      s/iter sha
  sha    499  --
  Generating array with 39810717 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+    70:41 446728  1416 1058111 32608 59.4 benchmark.pl
      s/iter sha
  sha    388  --
  Generating array with 50118723 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+    86:00 603755  1416 1260863 32960 60.0 benchmark.pl
      s/iter sha
  sha    531  --
  Generating array with 63095734 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+   106:33 839278  1416 1516151 31488 57.4 benchmark.pl
      s/iter sha
  sha    816  --
  Generating array with 79432823 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+   141:37 1224208 1416 2099583 31336 57.1 benchmark.pl
      s/iter sha
  sha    970  --
  Generating array with 100000000 elements...
    PID TTY      STAT   TIME  MAJFL   TRS   DRS   RSS %MEM COMMAND
   2551 pts/0    S+   178:54 1625198 1416 2504775 30196 55.0 benchmark.pl
      s/iter sha
  sha   1282  --
  Generating array with 125892541 elements...
      s/iter sha
  sha   2268  --