Revision history for Bencher-Backend
1.017 2016-07-13 (PERLANCAR)
- Add chart_result().
1.016 2016-07-13 (PERLANCAR)
- Include permutation in result metadata (func.permute).
1.015 2016-07-12 (PERLANCAR)
- Handle negative time in results.
- [Bugfix] Still allow stringy matching of --include with name even
though value is a number.
1.014 2016-06-30 (PERLANCAR)
- Add function: split_result().
1.013 2016-06-29 (PERLANCAR)
- [Bugfix] Determine number of significant digits from 'time', not from
'rate' because 'rate' is removed in module startup result.
1.012 2016-06-28 (PERLANCAR)
- format_result(): Add a convenient way to skip using the
RenderAsTextTable formatter by passing option render_as_text_table=>0.
- Avoid some perl undef warnings, e.g. for older bencher results where
'perl' field is not defined.
1.011 2016-06-23 (PERLANCAR)
- More tweaks/fixes for alignments.
1.010 2016-06-17 (PERLANCAR)
- [Bugfix] DeleteConstantFields formatter didn't use
delete_fields() yet.
1.009 2016-06-17 (PERLANCAR)
- [ux] Add column alignment to result.
- [ux] Use decimal notation by default, except for 'errors' field
or when option --scientific-notation is given.
1.008 2016-05-20 (PERLANCAR)
- Follow Sah::Schemas specification, rename
Sah::Schema::Bencher::Scenario to Sah::Schema::bencher::scenario.
- Avoid perl undef warning.
1.007 2016-04-19 (PERLANCAR)
- [Bugfix] save_result was turned *on* all the time, it should be
only when --save-result or (--result-dir or BENCHER_RESULT_DIR
was set).
1.006 2016-04-13 (PERLANCAR)
- Add --{in,ex}clude-{participant,dataset,item}-{name,seq} to
unambiguously include/exclude by name or sequence.
- [ux] Warn when there is a participant/dataset/item which has name that
is also a sequence number, which can be confusing when
including/excluding.
- [Bugfix] Defeat the annoying compile errors when using Capture::Tiny
due to mismatching prototypes.
1.005 2016-04-08 (PERLANCAR)
[ENHANCEMENTS]
- Add --include-process-size to show process memory stats.
- Add output capturing (--show-items-outputs, --capture-stdout,
--capture-stderr).
[BUG FIXES]
- Regression: --show-items-results and --show-items-results-sizes
returned JSON by default when it should return a formatted text.
- When trying to come up with a name for a command participant, also
take a look at cmdline_template/perl_cmdline/perl_cmdline_template in
addition to just cmdline.
1.004 2016-04-08 (PERLANCAR)
- Eval code template in 'main' package.
- Allow <VARNAME:raw> in template to avoid escaping.
1.003 2016-04-03 (PERLANCAR)
- [Bugfix] completer for participant/dataset name didn't include
generated names.
- [Bugfix] save_result was not turned on automatically when user
sets result_dir.
1.002 2016-04-01 (PERLANCAR)
- [Bug] Logic bug: --return-meta accidentally defaulted to off when
--save-result is not enabled.
1.001 2016-04-01 (PERLANCAR)
- Turn off include_result_size when in module startup mode.
- Add feature to save result to a JSON file (--save-result,
--result-dir, --result-filename).
- Grok scenario property 'extra_modules' to allow extra modules to
be loaded and their versions recorded in benchmark result
metadata.
1.000 2016-03-31 (PERLANCAR)
- No functional changes.
- Bump version number to 1.000.
0.46 2016-03-31 (PERLANCAR)
- No functional changes.
- [doc] Add description, fixes.
0.45 2016-03-30 (PERLANCAR)
- [ux] Check that what's specified in
--{in,ex}clude-{dataset,participant,item} are actually known, to
avoid silent surprise.
- [Bugfix] Fix typo that caused sometimes specifying
--include-participant by name doesn't work.
- [Bugfix] RoundNumbers formatter: no need to round result_size
except when it's not a round number.
0.44 2016-03-26 (PERLANCAR)
- Show result size (as measured by Devel::Size), add
--include-result-size (& include_result_size scenario property),
add action show-items-results-sizes.
- [Bugfix] Fix completion of -m when there is -I specified.
0.43 2016-03-24 (PERLANCAR)
- [ux] Directory specified in -I (--include-path) is now also used
for searching and loading scenario module, this is so we can say
something like 'bencher -Ilib -m Foo' instead of 'PERL5OPT=-Ilib
bencher -m Foo'.
- [ux] When a dataset has no name, try to assign a name from
argument value.
0.42 2016-03-22 (PERLANCAR)
- Allow specifying 'modules' in participant to benchmark startup
overhead of several modules at once.
0.41 2016-03-18 (PERLANCAR)
- No functional changes.
- [Bugfix] Include modver & perl in result items even when in not
multiperl/multimodver mode (ModuleStartup formatter module uses
the perl key).
- Add workaround for RT#113117 (List::MoreUtils sometimes return
undef max).
0.40 2016-03-16 (PERLANCAR)
- Add action: list-perls; by default only use perls that have
Bencher::Backend installed.
- [Bugfix] Avoid benchmarking duplicate items, e.g. 'perl -e1
(baseline)' when in module_startup mode + multimodver mode.
- [Bugfix] fix calculating mod_overhead_time when in multiperl mode.
0.39 2016-03-15 (PERLANCAR)
[ENHANCEMENTS]
- Support benchmarking against multiple perls (introduce options
`--multiperl`, `--include-perl`, `--exclude-perl`). Limitation:
Currently this is implemented by dumping (using Data::Dmp, which means
using B::Deparse) the items' codes into a temporary scenario file,
then executing the scenario file using each perl executable. This
means codes that contains closures (references to outside variable)
won't work correctly.
- Support benchmarking multiple versions of a module (`--multimodver`).
To do this, first add more module search paths using `--include-path`
(`-I`).
- In participant, introduce 'perl_cmdline' and 'perl_cmdline_template'
in addition to 'cmdline' and 'cmdline_template'. They are the same as
'cmdline*' except there is implicit first argument which is the perl
executable. They are for executing commands using perl, without having
to hardcode/include the perl executable location.
0.38 2016-03-15 (PERLANCAR)
- First release, split from Bencher distribution to reduce dependencies
when installing Bencher to a perl installation (this is part of
developing support for benchmarking against multiple perls/module
versions).