dbcolmovingstats - compute moving statistics over a window of a column of data
dbcolmovingstats [-am] [-w window-width] [-e EmptyValue] column
Compute moving statistics over a COLUMN of data. Records containing non-numeric data are considered null do not contribute to the stats (optionally they are treated as zeros with -a).
-a
Currently we compute mean and sample standard deviation. (Note we only compute sample standard deviation, not full population.) Optionally, with -m we also compute median. (Currently there is no support for generalized quantiles.)
-m
Values before a sufficient number have been accumulated are given the empty value (if specified with -e). If no empty value is given, stats are computed on as many are possible if no empty value is specified.
-e
Dbcolmovingstats runs in O(1) memory, but must buffer a full window of data. Quantiles currently will repeatedly sort the window and so may perform poorly with wide windows.
Compute stats over all records (treat non-numeric records as zero rather than just ignoring them).
WINDOW of how many items to accumulate (defaults to 10). (For compatibility with fsdb-1.x, -n is also supported.)
Show median of the window in addition to mean.
Give value E as the value for empty (null) records. This null value is then output before a full window is accumulated.
Specify a printf(3)-style format for output mean and standard deviation. Defaults to %.5g.
%.5g
Eventually we expect to support other options of dbcolstats.
This module also supports the standard fsdb options:
Enable debugging output.
Read from InputSource, typically a file name, or - for standard input, or (if in Perl) a IO::Handle, Fsdb::IO or Fsdb::BoundedQueue objects.
-
Write to OutputDestination, typically a file name, or - for standard output, or (if in Perl) a IO::Handle, Fsdb::IO or Fsdb::BoundedQueue objects.
By default, programs process automatically, but Fsdb::Filter objects in Perl do not run until you invoke the run() method. The --(no)autorun option controls that behavior within Perl.
--(no)autorun
Show help.
Show full manual.
#fsdb date epoch count 19980201 886320000 6 19980202 886406400 8 19980203 886492800 19 19980204 886579200 53 19980205 886665600 20 19980206 886752000 18 19980207 886838400 5 19980208 886924800 9 19980209 887011200 22 19980210 887097600 22 19980211 887184000 36 19980212 887270400 26 19980213 887356800 23 19980214 887443200 6
cat data.fsdb | dbmovingstats -e - -w 4 count
#fsdb date epoch count moving_mean moving_stddev 19980201 886320000 6 - - 19980202 886406400 8 - - 19980203 886492800 19 - - 19980204 886579200 53 21.5 21.764 19980205 886665600 20 25 19.442 19980206 886752000 18 27.5 17.02 19980207 886838400 5 24 20.445 19980208 886924800 9 13 7.1647 19980209 887011200 22 13.5 7.8528 19980210 887097600 22 14.5 8.8129 19980211 887184000 36 22.25 11.026 19980212 887270400 26 26.5 6.6081 19980213 887356800 23 26.75 6.3966 19980214 887443200 6 22.75 12.473 # | dbcolmovingstats -e - -n 4 count
Fsdb. dbcolstats. dbmultistats. dbrowdiff.
Currently there is no support for generalized quantiles.
$filter = new Fsdb::Filter::dbcolmovingstats(@arguments);
Create a new dbcolmovingstats object, taking command-line arguments.
$filter->set_defaults();
Internal: set up defaults.
$filter->parse_options(@ARGV);
Internal: parse command-line arguments.
$filter->setup();
Internal: setup, parse headers.
$filter->run();
Internal: run over each rows.
Copyright (C) 1991-2018 by John Heidemann <johnh@isi.edu>
This program is distributed under terms of the GNU general public license, version 2. See the file COPYING with the distribution for details.
To install Fsdb, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Fsdb
CPAN shell
perl -MCPAN -e shell install Fsdb
For more information on module installation, please visit the detailed CPAN module installation guide.