The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.
 / 
Data-Crumbr-0.1.1
River stage zero No dependents
++
/ Data::Crumbr

NAME

Data::Crumbr - Render data structures for easy searching and parsing

VERSION

version 0.1.1

SYNOPSIS

   use Data::Crumber; # imports `crumbr`

   # some data to work with
   my $data = { what => 'ever', hey => 'you' };

   # crumbr provides an anonyous sub back. This has defaults
   my $csub = crumbr();

   # use it to encode the data
   my $encoded = $crumbr->($data);
   # {"here"}{"hey"}:"you"
   # {"here"}{"what"}:"ever"

   # URI profile simplifies things but loses something
   $encoded = crumbr(profile => 'URI')->($data);
   # here/hey "you"
   # here/what "ever"

   # JSON profile produces valid JSON "slices"
   $encoded = crumbr(profile => 'JSON')->($data);
   # {"here":{"hey":"you"}}
   # {"here":{"what":"ever"}}

   # Object Oriented Interface
   my $crobj = Data::Crumbr->new();
   $encoded = $crobj->encode($data); # same as default

DESCRIPTION

Data::Crumbr lets you render data structures in a way that can then be easily searched and parsed in "slices". The basic idea is that data shaped in this way will then be easily filtered in the shell for extracting interesting parts.

The input data structure is traversed is if it is a tree (so no circular structures please!), and a record is generated for each leaf in the tree. Depending on the backend and the configurations, the full path from the root to the parent of the leaf is represented as a sequence of keys (which can be hash keys or array indexes) followed by the value. This should make your life easier e.g. in the shell, so that you can specify the full path to the data structure part you're interested into with common Unix tools like grep and/or sed.

Example

Suppose you have the following data structure in Perl:

   my $data = {
      one => '1',
      two => 2,
      three => 3.1,
      four => '4.0',
      true => \1,
      false => \0,
      array => [
         qw< what ever >,
         { inner => 'part', empty => [] }
       ],
      hash => {
         'with ♜' => {},
         ar => [ 1..3 ],
         something => "funny \x{263A} ☻",
      },
   };

If you encode this e.g. in JSON, it will be easy to parse with the right program, but not from the shell, even if you pretty print it:

   {
      "hash" : {
         "something" : "funny ☺ ☻",
         "with ♜" : {},
         "ar" : [
            1,
            2,
            3
         ]
      },
      "one" : "1",
      "array" : [
         "what",
         "ever",
         {
            "inner" : "part",
            "empty" : []
         }
      ],
      "four" : "4.0",
      "true" : true,
      "two" : 2,
      "three" : 3.1,
      "false" : false
   }

How do you get the second item in the array ari inside the hash hash? Would you do better with YAML instead?

   ---
   array:
     - what
     - ever
     - empty: []
       inner: part
   false: !!perl/ref
     =: 0
   four: 4.0
   hash:
     ar:
       - 1
       - 2
       - 3
     something: funny ☺ ☻
     with ♜: {}
   one: 1
   three: 3.1
   true: !!perl/ref
     =: 1
   two: 2

Not really. Data::Crumbr lets you represent the data in a more verbose but easily consumable way for the shell. Hence, this:

   use Data::Crumbr;
   print crumbr()->($data), "\n";

will give you this:

   {"array"}[0]:"what"
   {"array"}[1]:"ever"
   {"array"}[2]{"empty"}:[]
   {"array"}[2]{"inner"}:"part"
   {"false"}:false
   {"four"}:"4.0"
   {"hash"}{"ar"}[0]:1
   {"hash"}{"ar"}[1]:2
   {"hash"}{"ar"}[2]:3
   {"hash"}{"something"}:"funny \u263A \u263B"
   {"hash"}{"with \u265C"}:{}
   {"one"}:"1"
   {"three"}:3.1
   {"true"}:true
   {"two"}:2

Now it should pretty easy for a shell program to get at the data, e.g. with this sed substitution:

   sed -ne 's/^{"hash"}{"ar"}\[2\]://p'

Profiles

If you don't like the default encoding, you can get a different one by using a profile. This is a set of configurations for Data::Crumbr::Default, which is a pretty generic class for representing a wide class of possible record-oriented encodings.

A Data::Crumbr::Default encoder is defined in terms of the following parameters:

array_open

sequence to put when an array is opened

array_close

sequence to put when an array is closed

array_key_prefix

sequence to put before an array's index

array_key_suffix

sequence to put after an array's index

array_key_encoder

a reference to a function that encodes an array's index

hash_open

sequence to put when a hash is opened

hash_close

sequence to put when a hash is closed

hash_key_prefix

sequence to put before a hash's key

hash_key_suffix

sequence to put after a hash's key

hash_key_encoder

a reference to a function that encodes a hash's key

value_encoder

a reference to a function that encodes a leaf value

keys_separator

sequence to separate the keys breadcrumb

value_separator

sequence to separate the keys from the value

By default, Data::Crumbr ships with the following profiles:

Default

i.e. the profile you get by default, and what you saw in action in the example above. It has the following settings:

  • no openers and closers:

       array_open  => ''
   array_close => ''
   hash_open   => ''
   hash_close  => ''

  • array keys are printed verbatim, surrounded by square brackets:

       array_key_prefix  => '['
   array_key_suffix  => ']'
   array_key_encoder => Data::Crumbr::Util::id_encoder

  • hash keys encoded as JSON strings, surrounded by curly brackets:

       hash_key_prefix   => '['
   hash_key_suffix   => ']'
   hash_key_encoder  => Data::Crumbr::Util::json_leaf_encoder

  • no separator between keys (because they already stand out very clearly, but a colon to separate the sequence of keys from the value:

       keys_separator  => ''
   value_separator => ':'

  • leaf values encoded as JSON scalars:

       value_encoder => Data::Crumbr::Util::json_leaf_encoder

This is quite verbose, but lets you specify very precisely what you are looking for because the hash keys stand out clearly with respect to array identifiers, i.e. there's no chance that you will mistake an array index for a hash key (because they are embedded in different bracket types).

JSON

this profile always provides you compact JSON-compliant string representations that contain only one single leaf value.

It has the following characteristics:

  • openers and closers are what you would expect for JSON objects and arrays:

       array_open  => '['
   array_close => ']'
   hash_open   => '{'
   hash_close  => '}'

  • there is only one non-empty suffix, i.e. the hash key suffix, so that we can separate the hash key from the value with : according to JSON:

       array_key_prefix => ''
   array_key_suffix => ''
   hash_key_prefix  => ''
   hash_key_suffix  => ':'

  • array keys are not printed:

       array_key_encoder => sub { }

  • hash keys are JSON encoded:

       hash_key_encoder  => Data::Crumbr::Util::json_encoder()

  • no separators are needed:

       keys_separator  => ''
   value_separator => ''

  • leaf values encoded as JSON scalars:

       value_encoder => Data::Crumbr::Util::json_leaf_encoder
URI

this is the simplest of the profiles, and sacrifices the possibility to distinguish between hash and array keys to the altar of simplicity.

It has the following characteristics:

  • no openers, closers, prefixes or suffixes:

       array_open  => ''
   array_close => ''
   array_key_prefix => ''
   array_key_suffix => ''

   hash_open   => ''
   hash_close  => ''
   hash_key_prefix => ''
   hash_key_suffix => ''

  • array keys are printed verbatim

  • hash keys are URI encoded

       hash_key_encoder  => Data::Crumbr::Util::uri_encoder

  • keys are separated by a slash character / and values are separated by a single space :

       keys_separator  => '/'
   value_separator => ' '

  • leaf values encoded as JSON scalars:

       value_encoder => Data::Crumbr::Util::json_leaf_encoder

INTERFACE

There are two ways to use Data::Crumber: a function crumbr, that is exported by default, and the object-oriented interface.

crumbr
   $subref = crumbr(%args); # OR
   $subref = crumbr(\%args);

get a crumbr generator based on provided %args.

Returns a reference to a sub, which can then be called upon a data structure in order to get the crumbed version.

The input arguments can be:

encoder

details about the encoder, see "Profiles" for the available key-value pairs. In addition, you can also set the following:

output

the output channel to use for sending encoded data. This can be:

  • filename

    this will be opened in raw mode and used to send the output

  • filehandle

    used directly

  • array reference

    each output line will be pushed as a new element in the array

  • object reference

    which is assumed to support the print() method, that will be called with each generated line

  • sub reference

    which will be called with each generated line

profile

the name of a profile to use as a base - see "Profiles". Settings in the profile are always overridden by corresponding ones in the provided encoder, if any.

encode
   $dc->encode($data_structure);

generate the encoding for the provided $data_structure. Output is generated depending on how it is specified, see "crumbr" above.

new
   my $dc = Data::Crumber->new(encoder => \%args);

create a new instance of Data::Crumbr. Data provided for the encoder parameter (i.e. %args) are those discussed in "Profiles".

The new instance can then be used to encode data using the /encode method.

AUTHOR

Flavio Poletti <polettix@cpan.org>

COPYRIGHT AND LICENSE

Copyright (C) 2015 by Flavio Poletti <polettix@cpan.org>

This module is free software. You can redistribute it and/or modify it under the terms of the Artistic License 2.0.

This program is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose.