Revision history for Algorithm-Classifier-IsolationForest
0.7.0 2026-07-08/14:30
- Add explain_samples() / explain_sample_tagged(), reporting which
features drove a sample's anomaly score, to both the batch class
and ::Online. Two methods: 'ablation' (the default) substitutes
each feature in turn with a stored baseline and reports the score
drop -- fit() now learns per-feature training-data medians for
this and persists them with the model (::Online uses the medians
of its retained window instead, tracking drift for free);
'path' apportions credit over the splits each tree walk crossed
(local DIFFI; Carletti, Terzi & Susto 2023, see REFERENCES),
needing nothing beyond the trees, so it also serves models saved
before baseline support. Ablation is the default because it
answers the counterfactual for any scored sample; path credit is
only sharp for samples that were in the training data.
- Add the `iforest explain` CLI command exposing explain_samples:
one output line per (row, feature) pair, most responsible
feature first, with --method path|ablation, -n for top-N
features per row, and -t to score first and explain only the
rows clearing the cutoff.
- Add fit_from_csv(), which trains directly from a CSV file without
loading it into RAM (for data sets too large to slurp). Streams
the file in a census pass, a gather pass that keeps only the rows
the trees sampled (Floyd's algorithm, O(n_trees*sample_size)
memory), and -- when contamination is set -- a scoring pass whose
min-heap yields the exact same threshold the in-RAM learner would.
The census and gather passes only parse the cells they need (row
count / column width, and the sampled rows), and the contamination
scoring pass runs through the C backend when use_c is on, so a
learned threshold over a large file is seconds rather than minutes.
The first CSV line is skipped automatically when it holds feature
names (any non-numeric cell, or a match of stored feature_names);
header => 1 still forces it.
- fit_from_csv() now defaults to an "index" gather: a fast block-scan
census records each row's byte offset so the second pass seeks
straight to the sampled rows instead of re-scanning the whole file,
cutting a no-contamination fit of a 2M-row file from ~15s to ~3s.
The offset table costs 8*n bytes and is dropped (falling back to the
streaming reader) once it would exceed index_max (default 256 MiB);
pass index => 0 to force streaming. The contamination scoring pass
defaults to c_scan => 1, letting the C packer coerce cells instead
of validating each in Perl (identical result on valid data; a
non-numeric scored cell becomes 0.0 rather than dying). Under
missing => 'die', a missing cell is now rejected when it lands in a
sampled training row rather than during a full up-front scan.
- fit()/from_json(): _pack_tree, which flattens each tree into the
packed buffers the C scorer walks, now runs in the C backend
(pack_tree_xs) instead of a recursive Perl closure that built an
arrayref and six SVs per node and then flattened the lot through
a map for pack(). It had grown into the largest single phase of
an axis-mode fit: 100 trees repack in 0.6ms rather than 20.5ms
(4.2ms rather than 62ms in extended mode), taking a 10k x 8 fit
from 44ms to 16ms. Same DFS pre-order numbering, same dense-pack
rule, byte-identical buffers. Skipped on wide-NV perls, where
c(size) computed in C doubles would differ from _c() in the last
ulp -- those keep the pure-Perl packer, as _NV_IS_DOUBLE guards
elsewhere.
- fit(): under missing => 'die' (the default) the up-front scan for
undef cells now runs in the C backend when use_c is on, instead of
a per-cell Perl loop over the whole training set -- 183ms -> 17ms
on 400k rows x 4 features, where it had been ~68% of fit() time.
Same row-major order, so the same offending cell is reported. A
row that is not an arrayref now reads as missing at column 0 on
both the C and pure-Perl paths (previously a Perl deref error),
matching how pack_input_xs already treats one.
- Doc cleanup.
- Add t/81-sklearn-real-data.t and the four UCI datasets it uses
(glass, ionosphere, seeds, wdbc; CC BY 4.0, see t/data/README
for citations). The existing sklearn comparison only ever ran
against synthetic blobs, and only on machines with Python.
sklearn's scores are now checked in beside each dataset, so the
comparison runs everywhere; where Python is present an extra arm
re-runs sklearn live and catches drift against the checked-in
reference. Agreement is required to be no worse than our own
seed-to-seed agreement, rather than against a per-dataset floor:
an Isolation Forest is a random estimator, so that spread is the
ceiling, and measuring against it keeps the thresholds from
encoding how hard a given dataset is to rank. Also checks that C
and pure-Perl score identically on 30+ correlated columns, and
that fit_from_csv detects a real header.
0.6.0 2026-07-09/08:15
- Implement Online Isolation Forest (Filippo Leveni, Guilherme
Weigert Cassales, Bernhard Pfahringer, Albert Bifet, Giacomo
Boracchi (2024)) as the new companion class
Algorithm::Classifier::IsolationForest::Online and releated
CLI commands.
- initial prototype support
- munging via Algorithm::ToNumberMunger
0.5.0 2026-07-04/14:30
- Wide-NV perls (-Duselongdouble / -Dusequadmath(maybe? does this
even exist? but possibly some idiot on github... but would fix
it in this case as well): the pure-Perl tree builder now rounds
every value it stores (split points, hyperplane coefficients and
offsets, impute fills) to C double precision at the same points
the C builder rounds preserving the seed-for-seed bit-identical
guarantee across backends. Possible breakages for the tests for
extended mode where -Duselongdouble is in play may exist so tests
for those systems where it is are skipped for now.
- Implement Majority Voting Isolation Forest (MVIForest --
Chabchoub, Togbe, Boly & Chiky 2022, IEEE Access,
doi:10.1109/ACCESS.2022.3144425) as new(voting => 'majority'):
Since this only affects how the trees are walked etc and not build,
the voting method can be switched between majority and mean.
- adjust how iforest info displays tag info
0.4.0 2026-07-03/22:45
- Implement named features and methods for testing single rows using
tagged data.
- The C backend can be built and installed at install time, meaning
nothing needs built unless changing the opts. See the docs for
more details.
- new IF_NO_OPENMP=1 selects/builds the serial C backend: no
libgomp linkage and no OpenMP runtime in the process at all
(unlike OMP_NUM_THREADS=1, which just caps the thread count);
IF_NO_OPENMP=0 re-enables OpenMP over a serial install default
- new IF_RUNTIME_BUILD=1 ignores the prebuilt object and forces
the classic runtime build even when the flags match
- iforest accel updated to reflect various changes to the C backend
- scoring: the per-leaf path-length adjustment c(size) is now
precomputed at tree-pack time and stored in the (previously
unused) third slot of packed leaf records, removing a log()
call per point per tree from the C scoring hot loop -- about
25% faster axis-mode scoring; results are bit-identical
- scoring: score_all_xs now picks between two loop shapes based
on total forest size: small forests keep the point-major loop
(whole forest stays cache-resident anyway), while forests
over 4 MB switch to a tree-blocked loop that walks a block of
points through one tree at a time so each tree stays hot in
L1/L2 instead of being re-streamed from memory per point --
measured ~3.2x faster extended-mode scoring at 400 trees
(20k points, 16 features); both shapes add the same terms in
the same order, so scores are bit-identical either way
- any -march (IF_ARCH / IF_NATIVE, configure- or run-time) now
automatically adds -ffp-contract=off: with FMA available the
compiler otherwise contracts a*b+c into fused multiply-adds
whose different rounding silently broke the use_c bit-parity
guarantee (one ulp in a split value builds a different tree);
the -march win comes from vectorization, so this costs ~nothing
- scoring: oblique nodes now prefetch both child records before
the dot product resolves which branch is taken, hiding the
next node's memory latency under the FMA work (~7-10% faster
extended-mode scoring on top of the tiling; axis path is
untouched -- its single compare has no work to hide a
prefetch under); purely a hint, results unchanged
0.3.0 2026-07-02/23:00
- lots of POD fixes/cleanup
- various further C optmizations
- fit() can now handle training data with missing (undef) feature
cells, selectable via the new `missing =>` constructor option:
die :: croak on undef in the training data (default)
zero :: treat a missing cell as the value 0
impute :: fill with the per-feature mean/median (see `impute_with`)
nan :: range over present values and route missing rows to the
right child, consistently at fit and score time
- new `impute_with => 'mean'|'median'` option for impute mode ...
missing strategy + impute fill vector are persisted in saved models;
models from older releases load as `zero` (the prior undef -> 0
scoring behaviour)
- the C build is now tunable via environment variables read at first
module load: IF_ARCH=<value> adds -march=<value>, IF_NATIVE=1 is
shorthand for IF_ARCH=native, IF_OPT overrides the default -O3,
and IF_NO_C=1 skips building the C backend entirely; values are
validated (bad ones warn and fall back to the defaults) and the
flags actually used are exposed via $OPT_LEVEL
- Benchmarking: bench-sklearn-scoring.pl now compares pure Perl, C,
and C+OpenMP fit/score paths against sklearn side by side
0.2.1 2026-06-30/14:30
- derp... actually update MANIFEST so a bunch of files from last release
are actually included
0.2.0 2026-06-30/14:15
- C acceleration via Inline::C for core fit and predict ops
- OpenMP support for parallel multi-threaded fitting and predict ops
- SIMD (AVX/SSE) acceleration where available
- Data packing support for compact model storage (new `pack` CLI command)
- Parallel fit capability
- New `score_predict_split` method
- New `accel` CLI command for querying available acceleration flags
- New `bench` CLI command for running built-in benchmarks
- New `info` CLI command with expanded model introspection
- Benchmarking scripts covering fit, predict, scoring, and accel modes
- Tests: accel flag detection, accel selection, undef column handling,
data packing, parallel fit, sklearn comparison (including undef),
and CLI
- minor tweaks to `csv2plot` for a bit nicer rendering
- minor POD fixes
0.1.0 2026-06-23/03:15
- add csv2plot helper command for graphing
0.0.1 2026-06-21/21:45
- initial release