++ed by:
EMAZEP AERO TSIBLEY PERLANCAR PJFL

7 PAUSE users
3 non-PAUSE users.

Benjamin Bernard
and 1 contributors

NAME

recs-decollate

recs-decollate --help-all

 Help from: --help-basic:
 Usage: recs-decollate <args> [<files>]
    Decollate records of input (or records from <files>) into output records.
 
 Arguments:
    --dldeaggregator ...                Specify a domain language aggregate. See "Domain Language Integration" below.
    --deaggregator|-d <deaggregators>   Colon separated list of aggregate field specifiers. See "Deaggregates" section below.
    --list-deaggregators                Bail and output a list of deaggregators.
    --show-deaggregator <deaggregator>  Bail and output this deaggregator's detailed usage.
    --filename-key|fk <keyspec>         Add a key with the source filename (if no filename is applicable will put NONE)
 
   Help Options:
       --help-all             Output all help for this script
       --help                 This help screen
       --help-deaggregators   List the deaggregators
       --help-domainlanguage  Help on the recs domain language, a [very complicated] way of specifying valuations (which act like
                              keys) or aggregators
 
 Deaggregates:
    Deaggregates are specified as <deaggregator>[,<arguments>]. See --list-deaggregators for a list of available deaggregators.
 
    In general, key name arguments to deaggregators may be key specs, but not key groups
 
 Domain Lanuage Integration:
    The normal mechanism for specifying keys and aggregators allows one to concisely instantiate the objects that back them in the
    platform and is certainly the easiest way to use recs. The record stream domain language allows the creation of these objects in
    a programmatic way, with neither the syntactic issues of the normal way nor its guiding hand.
 
    The domain language is itself just PERL with a collection of library functions for creating platform objects included. Your
    favorite aggregators are all here with constructors matching their normal token. For convenience of e.g. last, aggregators are
    also included with a prefixed underscore.
 
    Below you can find documentation on all the "built in" functions. Most aggregators and deaggregators should be present with
    arguments comparable to their normal instantiation arugments, but with keyspec parameters replaced with valuations parameters.
 
    Deaggregates may be specified using the recs domain language. --dldeaggregator requires the code evaluate as a deaggregator.
 
    See --help-domainlanguage for a more complete description of its workings and a list of available functions.
 
    See the examples below for a more gentle introduction.
 
 Examples:
    Split the "hosts" field into individual "host" fields
       recs-decollate --dldeaggregator '_split(hosts,qr/, */,host)'
 
 Help from: --help-deaggregators:
 split: split the provided field
 unarray: split the provided array
 unhash: split the provided hash
 
 Help from: --help-domainlanguage:
 DOMAIN LANGUAGE
    The normal mechanism for specifying keys and aggregators allows one to concisely instantiate the objects that back them in the
    platform and is certainly the easiest way to use recs. The record stream domain language allows the creation of these objects in
    a programmatic way, with neither the syntactic issues of the normal way nor its guiding hand.
 
    The domain language is itself just PERL with a collection of library functions for creating platform objects included. Your
    favorite aggregators are all here with constructors matching their normal token. For convenience of e.g. last, aggregators are
    also included with a prefixed underscore.
 
    Below you can find documentation on all the "built in" functions. Most aggregators and deaggregators should be present with
    arguments comparable to their normal instantiation arugments, but with keyspec parameters replaced with valuations parameters.
 
 Special Syntax
    Where one sees a <snippet> argument below, a string scalar is expected, however quoting these can get fairly difficult and they
    can be confused with non-<snippet> scalars.
 
    Example:
      --dla "silly= uconcat(',', snip('{{x}} * 2'))"
 
    To remedy this, one may use <<CODE>> to inline a snippet which will be immediately understood by the typing mechanism as being
    code. Escaping inside this is as single quotes in PERL.
 
    Example With <<CODE>>
      --dla 'silly= uconcat(",", <<{{x}} * 2>>)'
 
    Furthermore one may mark variables to be propagated in by prefixing CODE like <<var1,var2,var3|CODE>>:
      --dla 'silly= $f=2; uconcat(",", <<f|{{x}} * $f>>)'
 
 Function Library
    ii_agg(<snippet>, <snippet>[, <snippet>])
    ii_aggregator(<snippet>, <snippet>[, <snippet>])
    inject_into_agg(<snippet>, <snippet>[, <snippet>])
    inject_into_aggregator(<snippet>, <snippet>[, <snippet>])
       Take an initial snippet, a combine snippet, and an optional squish snippet to produce an ad-hoc aggregator based on inject
       into. The initial snippet produces the aggregate value for an empty collection, then combine takes $a representing the
       aggregate value so far and $r representing the next record to add and returns the new aggregate value. Finally, the squish
       snippet takes $a representing the final aggregate value so far and produces the final answer for the aggregator.
 
       Example(s):
          Track count and sum to produce average:
             ii_agg(<<[0, 0]>>, <<[$a->[0] + 1, $a->[1] + {{ct}}]>>, <<$a->[1] / $a->[0]>>)
 
    for_field(qr/.../, <snippet>)
       Takes a regex and a snippet of code. Creates an aggregator that creates a map. Keys in the map correspond to fields chosen by
       matching the regex against the fields from input records. Values in the map are produced by aggregators which the snippet
       must act as a factory for ($f is the field).
 
       Example(s):
          To aggregate the sums of all the fields beginning with "t"
             for_field(qr/^t/, <<sum($f)>>)
 
    for_field(qr/.../, qr/.../, <snippet>)
       Takes two regexes and a snippet of code. Creates an aggregator that creates a map. Keys in the map correspond to pairs of
       fields chosen by matching the regexes against the fields from input records. Values in the map are produced by aggregators
       which the snippet must act as a factory for ($f1 is the first field, $f2 is the second field).
 
       Example(s):
          To find the covariance of all x-named fields with all y-named fields:
             for_field(qr/^x/, qr/^y/, <<covar($f1, $f2)>>)
 
    map_reduce_agg(<snippet>, <snippet>[, <snippet>])
    map_reduce_aggregator(<snippet>, <snippet>[, <snippet>])
    mr_agg(<snippet>, <snippet>[, <snippet>])
    mr_aggregator(<snippet>, <snippet>[, <snippet>])
       Take a map snippet, a reduce snippet, and an optional squish snippet to produce an ad-hoc aggregator based on map reduce. The
       map snippet takes $r representing a record and returns its mapped value. The reduce snippet takes $a and $b representing two
       mapped values and combines them. Finally, the squish snippet takes a mapped value $a representing all the records and
       produces the final answer for the aggregator.
 
       Example(s):
          Track count and sum to produce average:
             mr_agg(<<[1, {{ct}}]>>, <<[$a->[0] + $b->[0], $a->[1] + $b->[1]]>>, <<$a->[1] / $a->[0]>>)
 
    rec()
    record()
       A valuation that just returns the entire record.
 
    snip(snip)
       Takes a snippet and returns both the snippet and the snippet as a valuation. Used to distinguished snippets from scalars in
       cases where it matters, e.g. min('{{x}}') interprets it is a keyspec when it was meant to be a snippet (and then a
       valuation), min(snip('{{x}}')) does what is intended. This is used internally by <<...>> and in fact <<...>> just translates
       to snip('...').
 
    subset_agg(<snippet>, <aggregator>)
    subset_aggregator(<snippet>, <aggregator>)
       Takes a snippate to act as a record predicate and an aggregator and produces an aggregator that acts as the provided
       aggregator as run on the filtered view.
 
       Example(s):
           An aggregator that counts the number of records with a time not above 6 seconds:
              subset_agg(<<{{time_ms}} <= 6000>>, ct())
 
    type_agg(obj)
    type_scalar(obj)
    type_val(obj)
       Force the object into a specific type. Can be used to force certain upconversions (or avoid them).
 
    valuation(sub { ... })
    val(sub { ... })
       Takes a subref, creates a valuation that represents it. The subref will get the record as its first and only argument.
 
       Example(s):
          To get the square of the "x" field:
             val(sub{ $[0]->{x} ** 2 })
 
    xform(<aggregator>, <snippet>)
       Takes an aggregator and a snippet and produces an aggregator the represents invoking the snippet on the aggregator's result.
 
       Example(s):
          To take the difference between the first and second time fields of the record collection:
             xform(recs(), <<{{1/time}} - {{0/time}}>>)
 

See Also

RecordStream(3) - Overview of the scripts and the system
recs-examples(3) - A set of simple recs examples
recs-story(3) - A humorous introduction to RecordStream
SCRIPT --help - every script has a --help option, like the output above