The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.
 / 
Text-xSV-Slurp-0.11
River stage one • 1 direct dependent • 4 total dependents
1 ++
/ Text::xSV::Slurp

NAME

Text::xSV::Slurp - Convert xSV data to common data shapes.

VERSION

Version 0.11

SYNOPSIS

Text::xSV::Slurp converts between xSV data and a variety of nested data shapes, allowing both column and row filtering using user defined functions.

This brief example creates an array of hashes from a file, where each array record corresponds to a line of the file, and each line is represented as a hash of header-to-value pairs.

    use Text::xSV::Slurp 'xsv_slurp';
    
    my $aoh = xsv_slurp( 'foo.csv' );
    
    ## if foo.csv contains:
    ##
    ##   uid,name
    ##   342,tim
    ##   939,danboo
    ##
    ## then $aoh contains:
    ##
    ##   [
    ##     { uid => '342', name => 'tim' },
    ##     { uid => '939', name => 'danboo' },
    ##   ]

FUNCTIONS

xsv_slurp()

xsv_slurp() converts an xSV data source to one of a variety of nested data shapes. It allows both column and row filtering using user defined functions.

Option summary:

  • file - file name to be opened

  • handle - file handle to be iterated

  • string - string to be parsed

  • shape - target data structure (aoa, aoh, hoa or hoh)

  • col_grep - skip a subset of columns based on user callback

  • row_grep - skip a subset of rows based on user callback

  • key - xSV string or ARRAY used to build the keys of the hoh shape

  • text_csv - option hash for Text::CSV constructor

The file, handle and string options are mutually exclusive. Only one source parameter may be passed in each call to xsv_slurp(), otherwise a fatal exception will be raised.

The source can also be provided implicitly, without the associated key, and the source type will be guessed by examining the first item in the option list. If the item is a reference type, it is treated as a handle source. If the item contains a newline or carriage return, it is treated as a string source. If the item passes none of the prior tests, it is treated as a file source.

   ## implicit C<handle> source
   my $aoa = xsv_slurp( \*STDIN, shape => 'aoa' );

   ## implicit C<string> source
   my $aoh = xsv_slurp( "h1,h2\n" . "d1,d2\n" );

   ## implicit C<file> source
   my $aoh = xsv_slurp( 'foo.csv' );

The shape parameter supports values of aoa, aoh, hoa or hoh. The default shape is aoh. Each shape affects certain parameters differently (see below).

The text_csv option can be used to control Text::CSV parsing. The given HASH reference is passed to the Text::CSV constructor. If the text_csv option is undefined, the default Text::CSV constructor is called. For example, to change the separator to a colon, you could do the following:

   my $aoh = xsv_slurp( file => 'foo.csv',
                    text_csv => { sep_char => ':' } );

shape-specific option details:

       ## examples below assume the following data

   h1,h2,h3
   l,m,n
   p,q,r

aoa

    example data structure:

       [
      [ qw/ h1 h2 h3 / ],
      [ qw/ l  m  n  / ],
      [ qw/ p  q  r  / ],
   ]

    shape specifics:

    • col_grep - passed an ARRAY reference of indexes, should return a list of indexes to be included

    • row_grep - passed an ARRAY reference of values, should return true or false whether the row should be included or not

    full example:

       ## - convert xSV example to an array of arrays
   ## - include only rows containing values matching /[nr]/
   ## - include only the first and last columns 

   my $aoa = xsv_slurp( string   => $xsv_data,
                        shape    => 'aoa',
                        col_grep => sub { return @( shift() }[0,-1] },
                        row_grep => sub { return grep /[nr]/, @{ $_[0] } },
                      );

   ## $aoa contains:
   ##
   ##   [
   ##      [ 'l',  'n' ],
   ##      [ 'p',  'r' ],
   ##   ]

aoh

    example data structure:

       [
      { h1 => 'l', h2 => 'm', h3 => 'n' },
      { h1 => 'p', h2 => 'q', h3 => 'r' },
   ]

    shape specifics:

    • col_grep - passed an ARRAY reference of column names, should return a list of column names to be included

    • row_grep - passed a HASH reference of column name / value pairs, should return true or false whether the row should be included or not

    full example:

       ## - convert xSV example to an array of hashes
   ## - include only rows containing values matching /n/
   ## - include only the h3 column 

   my $aoh = xsv_slurp( string   => $xsv_data,
                        shape    => 'aoh',
                        col_grep => sub { return 'h3' },
                        row_grep => sub { return grep /n/, values %{ $_[0] } },
                      );

   ## $aoh contains:
   ##
   ##   [
   ##      { h3 => 'n' },
   ##   ]

hoa

    example data structure:

       {
      h1 => [ qw/ l p / ],
      h2 => [ qw/ m q / ],
      h3 => [ qw/ n r / ],
   }

    shape specifics:

    • col_grep - passed an ARRAY reference of column names, should return a list of column names to be included

    • row_grep - passed a HASH reference of column name / value pairs, should return true or false whether the row should be included or not

    full example:

       ## - convert xSV example to a hash of arrays
   ## - include only rows containing values matching /n/
   ## - include only the h3 column 

   my $hoa = xsv_slurp( string   => $xsv_data,
                        shape    => 'hoa',
                        col_grep => sub { return 'h3' },
                        row_grep => sub { return grep /n/, values %{ $_[0] } },
                      );

   ## $hoa contains:
   ##
   ##   {
   ##      h3 => [ qw/ n r / ],
   ##   }

hoh

    example data structure (assuming a key of 'h2,h3'):

       {
   m => { n => { h1 => 'l' } },
   q => { r => { h1 => 'p' } },
   }

    shape specifics:

    • key - an xSV string or ARRAY specifying the indexing column names

    • col_grep - passed an ARRAY reference of column names, should return a list of column names to be included

    • row_grep - passed a HASH reference of column name / value pairs, should return true or false whether the row should be included or not

    full example:

       ## - convert xSV example to a hash of hashes
   ## - index using h1 values
   ## - include only rows containing values matching /n/
   ## - include only the h3 column 

   my $hoh = xsv_slurp( string   => $xsv_data,
                        shape    => 'hoh',
                        key      => 'h1',
                        col_grep => sub { return 'h3' },
                        row_grep => sub { return grep /n/, values %{ $_[0] } },
                      );

   ## $hoh contains:
   ##
   ##   {
   ##      l => { h3 => 'n' },
   ##      p => { h3 => 'r' },
   ##   }

AUTHOR

Dan Boorstein, <dan at boorstein.net>

TODO

  • agg specific actions by key

  • add average, weighted-average and count agg keys and tests

  • add test for warn agg key

  • die and warn agg keys should include the value path in the output

  • document hoh 'agg' predefined keys

  • document hoh 'agg' custom keys

  • add a recipes/examples section to cover grep and agg examples

BUGS

Please report any bugs or feature requests to bug-text-xsv-slurp at rt.cpan.org, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Text-xSV-Slurp. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.

SUPPORT

You can find documentation for this module with the perldoc command.

    perldoc Text::xSV::Slurp

You can also look for information at:

ACKNOWLEDGEMENTS

COPYRIGHT & LICENSE

Copyright 2009 Dan Boorstein.

This program is free software; you can redistribute it and/or modify it under the terms of either: the GNU General Public License as published by the Free Software Foundation; or the Artistic License.

See http://dev.perl.org/licenses/ for more information.