NAME

Bencher::Scenario::ComparingArrays - Modules that compare arrays

VERSION

This document describes version 0.002 of Bencher::Scenario::ComparingArrays (from Perl distribution Bencher-Scenario-ComparingArrays), released on 2019-03-24.

SYNOPSIS

To run benchmark with default option:

 % bencher -m ComparingArrays

To run module startup overhead benchmark:

 % bencher --module-startup -m ComparingArrays

For more options (dump scenario, list/include/exclude/add participants, list/include/exclude/add datasets, etc), see bencher or run bencher --help.

DESCRIPTION

Packaging a benchmark script as a Bencher scenario makes it convenient to include/exclude/add participants/datasets (either via CLI or Perl code), send the result to a central repository, among others . See Bencher and bencher (CLI) for more details.

BENCHMARKED MODULES

Version numbers shown below are the versions used when running the sample benchmark.

Array::Compare 3.0.2

Arrays::Same 0.002

Cpanel::JSON::XS 3.0239

Data::Cmp 0.006

Sereal::Encoder 4.005

Storable 2.62

BENCHMARK PARTICIPANTS

  • Data::Cmp::cmp_data (perl_code) [int, str]

    Function call template:

     Data::Cmp::cmp_data(<array1>, <array2>)
  • Array::Compare (perl_code) [int, str]

    Code template:

     Array::Compare->new()->compare(<array1>, <array2>)
  • Arrays::Same::arrays_same_i (perl_code) [int]

    Function call template:

     Arrays::Same::arrays_same_i(<array1>, <array2>)
  • Arrays::Same::arrays_same_s (perl_code) [str]

    Function call template:

     Arrays::Same::arrays_same_s(<array1>, <array2>)
  • num eq op loop (perl_code) [int]

    Code template:

     Bencher::Scenario::ComparingArrays::_loop(<array1>, <array2>)
  • str eq op loop (perl_code) [str]

    Code template:

     Bencher::Scenario::ComparingArrays::_loopstr(<array1>, <array2>)
  • Storable (perl_code) [int, str]

    Code template:

     Storable::freeze(<array1>) eq Storable::freeze(<array2>)
  • Sereal::Encoder (perl_code) [int, str]

    Code template:

     Sereal::Encoder::encode_sereal(<array1>) eq Sereal::Encoder::encode_sereal(<array2>)
  • Cpanel::JSON::XS (perl_code) [int, str]

    Code template:

     Cpanel::JSON::XS::encode_json(<array1>) eq Cpanel::JSON::XS::encode_json(<array2>)

BENCHMARK DATASETS

  • empty

  • 10-int-same

  • 10-str-same

  • 100-int-same

  • 100-str-same

  • 1000-int-same

  • 1000-str-same

SAMPLE BENCHMARK RESULTS

Run on: perl: v5.26.1, CPU: Intel(R) Core(TM) M-5Y71 CPU @ 1.20GHz (2 cores), OS: GNU/Linux LinuxMint version 18.3, OS kernel: Linux version 4.10.0-38-generic.

Benchmark with default options (bencher -m ComparingArrays):

 #table1#
 {dataset=>"10-int-same"}
 +-----------------------------+----------+-----------+-----------+------------+---------+---------+
 | participant                 | p_tags   | rate (/s) | time (μs) | vs_slowest |  errors | samples |
 +-----------------------------+----------+-----------+-----------+------------+---------+---------+
 | Array::Compare              | int, str |     44800 |  22.3     |     1      | 6.7e-09 |      20 |
 | Data::Cmp::cmp_data         | int, str |     88400 |  11.3     |     1.97   |   1e-08 |      34 |
 | Storable                    | int, str |    142000 |   7.03    |     3.18   | 2.8e-09 |      28 |
 | num eq op loop              | int      |    520000 |   1.92    |    11.6    | 6.5e-10 |      33 |
 | Sereal::Encoder             | int, str |    599717 |   1.66745 |    13.3943 |   0     |      24 |
 | Cpanel::JSON::XS            | int, str |    670000 |   1.5     |    15      | 1.7e-09 |      20 |
 | Arrays::Same::arrays_same_i | int      |   1210000 |   0.824   |    27.1    | 4.5e-10 |      20 |
 +-----------------------------+----------+-----------+-----------+------------+---------+---------+

 #table2#
 {dataset=>"10-str-same"}
 +-----------------------------+----------+-----------+-----------+------------+---------+---------+
 | participant                 | p_tags   | rate (/s) | time (μs) | vs_slowest |  errors | samples |
 +-----------------------------+----------+-----------+-----------+------------+---------+---------+
 | Array::Compare              | int, str |     43000 |  23       |    1       | 5.3e-08 |      20 |
 | Data::Cmp::cmp_data         | int, str |     81100 |  12.3     |    1.89    | 3.3e-09 |      21 |
 | Storable                    | int, str |    130605 |   7.65669 |    3.03704 |   0     |      20 |
 | str eq op loop              | str      |    348000 |   2.87    |    8.1     | 8.3e-10 |      20 |
 | Cpanel::JSON::XS            | int, str |    420000 |   2.4     |    9.8     | 5.3e-09 |      32 |
 | Sereal::Encoder             | int, str |    430000 |   2.3     |   10       | 3.1e-09 |      23 |
 | Arrays::Same::arrays_same_s | str      |    590000 |   1.7     |   14       | 3.3e-09 |      20 |
 +-----------------------------+----------+-----------+-----------+------------+---------+---------+

 #table3#
 {dataset=>"100-int-same"}
 +-----------------------------+----------+-----------+-----------+------------+---------+---------+
 | participant                 | p_tags   | rate (/s) | time (μs) | vs_slowest |  errors | samples |
 +-----------------------------+----------+-----------+-----------+------------+---------+---------+
 | Data::Cmp::cmp_data         | int, str |     11500 |     87.1  |        1   | 2.7e-08 |      20 |
 | Array::Compare              | int, str |     15000 |     67    |        1.3 | 3.5e-07 |      20 |
 | Storable                    | int, str |     63000 |     16    |        5.5 | 1.1e-07 |      20 |
 | num eq op loop              | int      |     69000 |     15    |        6   | 1.1e-07 |      20 |
 | Cpanel::JSON::XS            | int, str |     90000 |     11    |        7.9 | 1.6e-08 |      23 |
 | Sereal::Encoder             | int, str |    127000 |      7.84 |       11.1 | 3.3e-09 |      20 |
 | Arrays::Same::arrays_same_i | int      |    180000 |      5.5  |       16   | 6.5e-09 |      21 |
 +-----------------------------+----------+-----------+-----------+------------+---------+---------+

 #table4#
 {dataset=>"100-str-same"}
 +-----------------------------+----------+-----------+-----------+------------+---------+---------+
 | participant                 | p_tags   | rate (/s) | time (μs) | vs_slowest |  errors | samples |
 +-----------------------------+----------+-----------+-----------+------------+---------+---------+
 | Data::Cmp::cmp_data         | int, str |   11000   |   94      |    1       |   1e-07 |      21 |
 | Array::Compare              | int, str |   16000   |   63      |    1.5     | 1.1e-07 |      20 |
 | Storable                    | int, str |   37200   |   26.9    |    3.48    | 1.3e-08 |      20 |
 | str eq op loop              | str      |   47175.7 |   21.1973 |    4.41521 |   0     |      20 |
 | Cpanel::JSON::XS            | int, str |   60000   |   17      |    5.6     | 2.7e-08 |      20 |
 | Sereal::Encoder             | int, str |   66000   |   15      |    6.2     | 4.9e-08 |      24 |
 | Arrays::Same::arrays_same_s | str      |   77000   |   13      |    7.2     |   2e-08 |      21 |
 +-----------------------------+----------+-----------+-----------+------------+---------+---------+

 #table5#
 {dataset=>"1000-int-same"}
 +-----------------------------+----------+-----------+-----------+------------+---------+---------+
 | participant                 | p_tags   | rate (/s) | time (μs) | vs_slowest |  errors | samples |
 +-----------------------------+----------+-----------+-----------+------------+---------+---------+
 | Data::Cmp::cmp_data         | int, str |      1200 |     870   |        1   |   2e-06 |      21 |
 | Array::Compare              | int, str |      2100 |     470   |        1.8 | 9.1e-07 |      20 |
 | num eq op loop              | int      |      6900 |     150   |        6   | 2.1e-07 |      20 |
 | Storable                    | int, str |      7400 |     140   |        6.4 | 1.6e-07 |      20 |
 | Cpanel::JSON::XS            | int, str |     10000 |      96   |        9   | 1.1e-07 |      20 |
 | Sereal::Encoder             | int, str |     15000 |      69   |       13   | 2.1e-07 |      20 |
 | Arrays::Same::arrays_same_i | int      |     16500 |      60.5 |       14.4 | 2.7e-08 |      20 |
 +-----------------------------+----------+-----------+-----------+------------+---------+---------+

 #table6#
 {dataset=>"1000-str-same"}
 +-----------------------------+----------+-----------+-----------+------------+---------+---------+
 | participant                 | p_tags   | rate (/s) | time (μs) | vs_slowest |  errors | samples |
 +-----------------------------+----------+-----------+-----------+------------+---------+---------+
 | Data::Cmp::cmp_data         | int, str |   1100    |   900     |    1       | 2.7e-06 |      22 |
 | Array::Compare              | int, str |   2180    |   458     |    1.97    | 2.7e-07 |      20 |
 | Storable                    | int, str |   4660    |   215     |    4.21    | 1.8e-07 |      27 |
 | str eq op loop              | str      |   4836.24 |   206.772 |    4.37298 |   0     |      20 |
 | Cpanel::JSON::XS            | int, str |   6539.15 |   152.925 |    5.91276 |   0     |      20 |
 | Sereal::Encoder             | int, str |   7100    |   140     |    6.4     | 2.5e-07 |      23 |
 | Arrays::Same::arrays_same_s | str      |   7800    |   130     |    7       | 2.1e-07 |      20 |
 +-----------------------------+----------+-----------+-----------+------------+---------+---------+

 #table7#
 {dataset=>"empty"}
 +-----------------------------+----------+-----------+-----------+------------+---------+---------+
 | participant                 | p_tags   | rate (/s) | time (μs) | vs_slowest |  errors | samples |
 +-----------------------------+----------+-----------+-----------+------------+---------+---------+
 | Array::Compare              | int, str |     57000 |  17       |    1       | 2.7e-08 |      20 |
 | Storable                    | int, str |    173526 |   5.76282 |    3.02255 |   0     |      20 |
 | Data::Cmp::cmp_data         | int, str |    390000 |   2.6     |    6.8     | 3.3e-09 |      20 |
 | Sereal::Encoder             | int, str |   1130000 |   0.888   |   19.6     | 4.1e-10 |      21 |
 | str eq op loop              | str      |   1700000 |   0.6     |   29       | 8.1e-10 |      21 |
 | num eq op loop              | int      |   1800000 |   0.55    |   32       | 2.6e-09 |      21 |
 | Cpanel::JSON::XS            | int, str |   1800000 |   0.55    |   32       | 7.8e-10 |      24 |
 | Arrays::Same::arrays_same_s | str      |   4400000 |   0.23    |   76       | 4.2e-10 |      20 |
 | Arrays::Same::arrays_same_i | int      |   4800000 |   0.21    |   83       | 4.2e-10 |      20 |
 +-----------------------------+----------+-----------+-----------+------------+---------+---------+

Benchmark module startup overhead (bencher -m ComparingArrays --module-startup):

 #table8#
 +---------------------+-----------+------------------------+------------+---------+---------+
 | participant         | time (ms) | mod_overhead_time (ms) | vs_slowest |  errors | samples |
 +---------------------+-----------+------------------------+------------+---------+---------+
 | Array::Compare      |      55   |                   49.1 |        1   | 9.8e-05 |      20 |
 | Storable            |      14   |                    8.1 |        4   | 2.5e-05 |      20 |
 | Sereal::Encoder     |      12   |                    6.1 |        4.5 |   2e-05 |      20 |
 | Cpanel::JSON::XS    |      11   |                    5.1 |        4.9 | 1.8e-05 |      20 |
 | Arrays::Same        |      10   |                    4.1 |        5.4 | 1.1e-05 |      24 |
 | Data::Cmp           |       9.5 |                    3.6 |        5.8 | 2.3e-05 |      20 |
 | perl -e1 (baseline) |       5.9 |                    0   |        9.3 | 4.3e-05 |      20 |
 +---------------------+-----------+------------------------+------------+---------+---------+

To display as an interactive HTML table on a browser, you can add option --format html+datatables.

HOMEPAGE

Please visit the project's homepage at https://metacpan.org/release/Bencher-Scenario-ComparingArrays.

SOURCE

Source repository is at https://github.com/perlancar/perl-Bencher-Scenario-ComparingArrays.

BUGS

Please report any bugs or feature requests on the bugtracker website https://rt.cpan.org/Public/Dist/Display.html?Name=Bencher-Scenario-ComparingArrays

When submitting a bug or request, please include a test-file or a patch to an existing test-file that illustrates the bug or desired feature.

AUTHOR

perlancar <perlancar@cpan.org>

COPYRIGHT AND LICENSE

This software is copyright (c) 2019 by perlancar@cpan.org.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.