NAME
Bencher::Scenario::ExceptionHandling - Benchmark various ways to do exception handling in Perl
VERSION
This document describes version 0.001 of Bencher::Scenario::ExceptionHandling (from Perl distribution Bencher-Scenario-ExceptionHandling), released on 2024-02-23.
SYNOPSIS
To run benchmark with default option:
% bencher -m ExceptionHandling
To run module startup overhead benchmark:
% bencher --module-startup -m ExceptionHandling
For more options (dump scenario, list/include/exclude/add participants, list/include/exclude/add datasets, etc), see bencher or run bencher --help
.
DESCRIPTION
Keywords: try-catch, eval, die
TODO: benchmark other try-catch modules.
BENCHMARKED MODULES
Version numbers shown below are the versions used when running the sample benchmark.
Try::Tiny 0.31
BENCHMARK PARTICIPANTS
builtin-try (perl_code)
Code template:
use feature 'try'; use experimental 'try'; try { <code_try:raw> } catch($e) { <code_catch:raw> }
Requires perl 5.34+.
naive-eval (perl_code)
Code template:
eval { <code_try:raw> }; if ($@) { <code_catch:raw> }
eval-localize-die-signal-and-eval-error (perl_code)
Code template:
{ local $@; local $SIG{__DIE__}; eval { <code_try:raw> }; if ($@) { <code_catch:raw> } }
Try::Tiny (perl_code)
Code template:
use Try::Tiny; try { <code_try:raw> } catch { <code_catch:raw> }
BENCHMARK DATASETS
empty try, empty catch
die in try, empty catch
BENCHMARK SAMPLE RESULTS
Sample benchmark #1
Run on: perl: v5.38.2, CPU: Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz (2 cores), OS: GNU/Linux Ubuntu version 20.04, OS kernel: Linux version 5.4.0-164-generic.
Benchmark command (default options):
% bencher -m ExceptionHandling
Result formatted as table (split, part 1 of 2):
#table1#
{dataset=>"die in try, empty catch"}
+-----------------------------------------+-----------+-----------+-----------------------+-----------------------+---------+---------+
| participant | rate (/s) | time (μs) | pct_faster_vs_slowest | pct_slower_vs_fastest | errors | samples |
+-----------------------------------------+-----------+-----------+-----------------------+-----------------------+---------+---------+
| Try::Tiny | 310000 | 3.23 | 0.00% | 602.22% | 2.1e-09 | 8 |
| eval-localize-die-signal-and-eval-error | 1000000 | 1 | 207.56% | 128.32% | 1.2e-08 | 9 |
| builtin-try | 1700000 | 0.6 | 438.20% | 30.48% | 1.1e-09 | 7 |
| naive-eval | 2170000 | 0.46 | 602.22% | 0.00% | 3.6e-10 | 8 |
+-----------------------------------------+-----------+-----------+-----------------------+-----------------------+---------+---------+
The above result formatted in Benchmark.pm style:
Rate T:T e b n
T:T 310000/s -- -69% -81% -85%
e 1000000/s 223% -- -40% -54%
b 1700000/s 438% 66% -- -23%
n 2170000/s 602% 117% 30% --
Legends:
T:T: participant=Try::Tiny
b: participant=builtin-try
e: participant=eval-localize-die-signal-and-eval-error
n: participant=naive-eval
The above result presented as chart:
#IMAGE: share/images/bencher-result-1.png|/tmp/VHOUgvh_oa/bencher-result-1.png
Result formatted as table (split, part 2 of 2):
#table2#
{dataset=>"empty try, empty catch"}
+-----------------------------------------+-----------+-----------+-----------------------+-----------------------+---------+---------+
| participant | rate (/s) | time (μs) | pct_faster_vs_slowest | pct_slower_vs_fastest | errors | samples |
+-----------------------------------------+-----------+-----------+-----------------------+-----------------------+---------+---------+
| Try::Tiny | 397000 | 2.52 | 0.00% | 6147.90% | 5.8e-10 | 8 |
| eval-localize-die-signal-and-eval-error | 1700000 | 0.59 | 328.63% | 1357.65% | 6.5e-10 | 7 |
| builtin-try | 8400000 | 0.12 | 2022.06% | 194.43% | 1.9e-10 | 7 |
| naive-eval | 25000000 | 0.04 | 6147.90% | 0.00% | 8e-11 | 7 |
+-----------------------------------------+-----------+-----------+-----------------------+-----------------------+---------+---------+
The above result formatted in Benchmark.pm style:
Rate T:T e b n
T:T 397000/s -- -76% -95% -98%
e 1700000/s 327% -- -79% -93%
b 8400000/s 2000% 391% -- -66%
n 25000000/s 6200% 1374% 200% --
Legends:
T:T: participant=Try::Tiny
b: participant=builtin-try
e: participant=eval-localize-die-signal-and-eval-error
n: participant=naive-eval
The above result presented as chart:
#IMAGE: share/images/bencher-result-2.png|/tmp/VHOUgvh_oa/bencher-result-2.png
Sample benchmark #2
Benchmark command (benchmarking module startup overhead):
% bencher -m ExceptionHandling --module-startup
Result formatted as table:
#table3#
+---------------------+-----------+-------------------+-----------------------+-----------------------+---------+---------+
| participant | time (ms) | mod_overhead_time | pct_faster_vs_slowest | pct_slower_vs_fastest | errors | samples |
+---------------------+-----------+-------------------+-----------------------+-----------------------+---------+---------+
| Try::Tiny | 14 | 8.4 | 0.00% | 145.39% | 1.8e-05 | 7 |
| perl -e1 (baseline) | 5.6 | 0 | 145.39% | 0.00% | 1.6e-05 | 7 |
+---------------------+-----------+-------------------+-----------------------+-----------------------+---------+---------+
The above result formatted in Benchmark.pm style:
Rate T:T perl -e1 (baseline)
T:T 71.4/s -- -60%
perl -e1 (baseline) 178.6/s 150% --
Legends:
T:T: mod_overhead_time=8.4 participant=Try::Tiny
perl -e1 (baseline): mod_overhead_time=0 participant=perl -e1 (baseline)
The above result presented as chart:
#IMAGE: share/images/bencher-result-3.png|/tmp/VHOUgvh_oa/bencher-result-3.png
To display as an interactive HTML table on a browser, you can add option --format html+datatables
.
HOMEPAGE
Please visit the project's homepage at https://metacpan.org/release/Bencher-Scenario-ExceptionHandling.
SOURCE
Source repository is at https://github.com/perlancar/perl-Bencher-Scenario-ExceptionHandling.
AUTHOR
perlancar <perlancar@cpan.org>
CONTRIBUTING
To contribute, you can send patches by email/via RT, or send pull requests on GitHub.
Most of the time, you don't need to build the distribution yourself. You can simply modify the code, then test via:
% prove -l
If you want to build the distribution (e.g. to try to install it locally on your system), you can install Dist::Zilla, Dist::Zilla::PluginBundle::Author::PERLANCAR, Pod::Weaver::PluginBundle::Author::PERLANCAR, and sometimes one or two other Dist::Zilla- and/or Pod::Weaver plugins. Any additional steps required beyond that are considered a bug and can be reported to me.
COPYRIGHT AND LICENSE
This software is copyright (c) 2024 by perlancar <perlancar@cpan.org>.
This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.
BUGS
Please report any bugs or feature requests on the bugtracker website https://rt.cpan.org/Public/Dist/Display.html?Name=Bencher-Scenario-ExceptionHandling
When submitting a bug or request, please include a test-file or a patch to an existing test-file that illustrates the bug or desired feature.