NAME

Bencher::Scenario::PerinciSubValidateArgs::Overhead - Measure validation overhead

VERSION

This document describes version 0.004 of Bencher::Scenario::PerinciSubValidateArgs::Overhead (from Perl distribution Bencher-Scenarios-PerinciSubValidateArgs), released on 2016-05-25.

SYNOPSIS

To run benchmark with default option:

 % bencher -m PerinciSubValidateArgs::Overhead

To run module startup overhead benchmark:

 % bencher --module-startup -m PerinciSubValidateArgs::Overhead

For more options (dump scenario, list/include/exclude/add participants, list/include/exclude/add datasets, etc), see bencher or run bencher --help.

BENCHMARKED MODULES

Version numbers shown below are the versions used when running the sample benchmark.

Bencher::ScenarioUtil::PerinciSubValidateArgs::NoValidate

Bencher::ScenarioUtil::PerinciSubValidateArgs::ValidateManually

Bencher::ScenarioUtil::PerinciSubValidateArgs::ValidateManualDataSah

Bencher::ScenarioUtil::PerinciSubValidateArgs::ValidateUsingPSW

Bencher::ScenarioUtil::PerinciSubValidateArgs::ValidateUsingPSV

Bencher::ScenarioUtil::PerinciSubValidateArgs::ValidateUsingTypeTiny

BENCHMARK PARTICIPANTS

• none (perl_code)

  Function call template:

   Bencher::ScenarioUtil::PerinciSubValidateArgs::NoValidate::foo(a1=><a1>, a2=><a2>)

• manual (perl_code)

  Function call template:

   Bencher::ScenarioUtil::PerinciSubValidateArgs::ValidateManually::foo(a1=><a1>, a2=><a2>)

• manual+dsah (perl_code)

  Function call template:

   Bencher::ScenarioUtil::PerinciSubValidateArgs::ValidateManualDataSah::foo(a1=><a1>, a2=><a2>)

• DZPRV (perl_code) (not included by default)

  Function call template:

   Bencher::ScenarioUtil::PerinciSubValidateArgs::ValidateUsingPSW::foo(a1=><a1>, a2=><a2>)

• DZPRW (perl_code) (not included by default)

  Function call template:

   Bencher::ScenarioUtil::PerinciSubValidateArgs::ValidateUsingPSV::foo(a1=><a1>, a2=><a2>)

• PSW (perl_code)

  Function call template:

   Bencher::ScenarioUtil::PerinciSubValidateArgs::ValidateUsingTypeTiny::foo(a1=><a1>, a2=><a2>)

• PSV (perl_code)

  Bencher::ScenarioUtil::PerinciSubValidateArgs::ValidateUsingPSV::foo

• Type::Tiny (perl_code)

  Bencher::ScenarioUtil::PerinciSubValidateArgs::ValidateUsingTypeTiny::foo

BENCHMARK DATASETS

• {a1=>1,a2=>[1]}

SAMPLE BENCHMARK RESULTS

Run on: perl: v5.22.1, CPU: Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz (4 cores), OS: GNU/Linux Debian version 8.0, OS kernel: Linux version 3.16.0-4-amd64.

Benchmark with default options (bencher -m PerinciSubValidateArgs::Overhead):

 +-------------+------------+-----------+------------+---------+---------+
 | participant | rate (/s)  | time (μs) | vs_slowest | errors  | samples |
 +-------------+------------+-----------+------------+---------+---------+
 | PSW         | 7.6e+04    | 13        | 1          | 2.2e-08 | 30      |
 | PSV         | 1.0787e+05 | 9.2703    | 1.4252     | 5.7e-11 | 20      |
 | manual+dsah | 1.18e+05   | 8.5       | 1.55       | 3.3e-09 | 20      |
 | Type::Tiny  | 1.3e+05    | 8         | 1.7        | 1.2e-08 | 23      |
 | manual      | 2.4e+05    | 4.2       | 3.1        | 5e-09   | 20      |
 | none        | 1.969e+06  | 0.508     | 26.01      | 4.7e-11 | 21      |
 +-------------+------------+-----------+------------+---------+---------+

Benchmark module startup overhead (bencher -m PerinciSubValidateArgs::Overhead --module-startup):

 +----------------------------------------------------------------------+-----------+------------------------+------------+---------+---------+
 | participant                                                          | time (ms) | mod_overhead_time (ms) | vs_slowest | errors  | samples |
 +----------------------------------------------------------------------+-----------+------------------------+------------+---------+---------+
 | Bencher::ScenarioUtil::PerinciSubValidateArgs::ValidateUsingPSW      | 78        | 73                     | 1          | 0.00014 | 20      |
 | Bencher::ScenarioUtil::PerinciSubValidateArgs::ValidateManualDataSah | 68        | 63                     | 1.1        | 0.00015 | 20      |
 | Bencher::ScenarioUtil::PerinciSubValidateArgs::ValidateUsingTypeTiny | 47        | 42                     | 1.7        | 9.7e-05 | 20      |
 | Bencher::ScenarioUtil::PerinciSubValidateArgs::ValidateUsingPSV      | 11        | 6                      | 6.8        | 4.1e-05 | 21      |
 | Bencher::ScenarioUtil::PerinciSubValidateArgs::ValidateManually      | 8.3       | 3.3                    | 9.3        | 2.5e-05 | 20      |
 | Bencher::ScenarioUtil::PerinciSubValidateArgs::NoValidate            | 8.1       | 3.1                    | 9.7        | 5.5e-05 | 20      |
 | perl -e1 (baseline)                                                  | 5         | 0                      | 2e+01      | 5.1e-05 | 20      |
 +----------------------------------------------------------------------+-----------+------------------------+------------+---------+---------+

DESCRIPTION

Packaging a benchmark script as a Bencher scenario makes it convenient to include/exclude/add participants/datasets (either via CLI or Perl code), send the result to a central repository, among others . See Bencher and bencher (CLI) for more details.

BENCHMARK NOTES

The Type::Tiny, Data::Sah, Perinci::Sub::ValidateArgs participants (except none, obviously) all should be in the same order of magnitude because both Data::Sah and Type::Tiny work by generating Perl code validator and then compiling (eval()) them then execute the compiled result. The differences are in the details: how the generated Perl code is structured, what the code for the type checks are (e.g. checking for number can be done with a regex or Scalar::Util's looks_like_number() or Scalar::Util::Numeric, and so on).

Perinci::Sub::Wrapper (PSW) and Dist::Zilla::Plugin::Rinci::Wrap is slower because it does more.

HOMEPAGE

Please visit the project's homepage at https://metacpan.org/release/Bencher-Scenarios-PerinciSubValidateArgs.

SOURCE

Source repository is at https://github.com/perlancar/perl-Bencher-Scenarios-PerinciSubValidateArgs.

BUGS

Please report any bugs or feature requests on the bugtracker website https://rt.cpan.org/Public/Dist/Display.html?Name=Bencher-Scenarios-PerinciSubValidateArgs

When submitting a bug or request, please include a test-file or a patch to an existing test-file that illustrates the bug or desired feature.

AUTHOR

perlancar <perlancar@cpan.org>

COPYRIGHT AND LICENSE

This software is copyright (c) 2016 by perlancar@cpan.org.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.

cpanm Bencher::Scenarios::PerinciSubValidateArgs

perl -MCPAN -e shell
install Bencher::Scenarios::PerinciSubValidateArgs