Text::AutoCSV - helper module to automate the use of Text::CSV
version 1.1.9
By default, Text::AutoCSV will detect the following characteristics of the input:
- The separator, among ",", ";" and "\t" (tab)
- The escape character, among '"' (double-quote) and '\\' (backslash)
- Try UTF-8 and if it fails, fall back on latin1
- Read the header line and compute field names
- If asked to (see "fields_dates_auto"), detect any field that contains a DateTime value, trying 20 date formats, possibly followed by a time (6 time formats tested)
- If asked to (see "fields_dates"), detect DateTime format of certain fields, croak if no DateTime format can be worked out
- Fields identified as containing a DateTime value ("fields_dates_auto" or "fields_dates") are stored as DateTime objects by default
Text::AutoCSV also provides methods to search on fields (using cached hash tables) and it can populate the value of "remote" fields, made from joining 2 CSV files with a key-value search
use Text::AutoCSV; Text::AutoCSV->new()->write(); # Read CSV data from std input, write to std output Text::AutoCSV->new(in_file => 'f.csv')->write(); # Read CSV data from f.csv, write to std output # Read CSV data from f.csv, write to g.csv Text::AutoCSV->new(in_file => 'f.csv', out_file => 'g.csv')->write(); # "Rewrite" CSV file by printing out records as a list (separated by line breaks) of field # name followed by its value. my $csv = Text::AutoCSV->new(in_file => 'in.csv', walker_hr => \&walk); my @cols = $csv->get_fields_names(); $csv->read(); sub walk { my %rec = %{$_[0]}; for (@cols) { next if $_ eq ''; print("$_ => ", $rec{$_}, "\n"); } print("\n"); }
# Identify column internal names with more flexibility as the default mechanism my $csv = Text::AutoCSV->new(in_file => 'zips.csv', fields_hr => {'CITY' => '^(city|town)', 'ZIPCODE' => '^zip(code)?$'}); # Get zipcode of Claix my $z = $csv->vlookup('CITY', ' claix ', 'ZIPCODE'); my $csv = Text::AutoCSV->new(in_file => 'zips.csv'); # Get zipcode of Claix my $z = $csv->vlookup('CITY', ' claix ', 'ZIPCODE'); # Same as above, but vlookup is strict for case and spaces around my $csv = Text::AutoCSV->new(in_file => 'zips.csv', search_case => 1, search_trim => 0); my $z = $csv->vlookup('CITY', 'Claix', 'ZIPCODE'); # Create field 'MYCITY' made by taking pers.csv' ZIP column value, looking it up in the # ZIPCODE columns of zips.csv, taking CITY colmun value and naming it 'MYCITY'. Output is # written in std output. # If a zipcode is ambiguous, say it. Text::AutoCSV->new(in_file => 'pers.csv') ->field_add_link('MYCITY', 'ZIP->ZIPCODE->CITY', 'zips.csv', { ignore_ambiguous => 0, value_if_ambiguous => '<duplicate zipcode found!>' })->write(); # Note the above can also be written using Text::AutoCSV level attributes: Text::AutoCSV->new(in_file => 'pers.csv', search_ignore_ambiguous => 0, search_value_if_ambiguous => '<duplicate zipcode found!>') ->field_add_link('MYCITY', 'ZIP->ZIPCODE->CITY', 'zips.csv')->write(); # Create 'MYCITY' field as above, then display some statistics my $nom_compose = 0; my $zip_not_found = 0; Text::AutoCSV->new(in_file => 'pers.csv', walker_hr => \&walk) ->field_add_link('MYCITY', 'ZIP->ZIPCODE->CITY', 'zips.csv')->read(); sub walk { my $hr = shift; $nom_compose++ if $hr->{'NAME'} =~ m/[- ]/; $zip_not_found++ unless defined($hr->{'MYCITY'}); } print("Number of persons with a multi-part name: $nom_compose\n"); print("Number of persons with unknown zipcode: $zip_not_found\n");
Text::AutoCSV->new(in_file => 'names.csv', out_file => 'ucnames.csv', read_post_update_hr => \&updt)->write(); sub updt { $_[0]->{'LASTNAME'} =~ s/^.*$/\U&/; } Text::AutoCSV->new(in_file => 'squares.csv', out_file => 'checkedsquares.csv', out_filter => \&wf)->write(); sub wf { return ($_[0]->{'X'} ** 2 == $_[0]->{'SQUAREOFX'}); } # Add a field for the full name, made of the concatenation of the # first name and the last name. # Also display stats about empty full names. Text::AutoCSV->new(in_file => 'dirpeople.csv', out_file => 'dirwithfn.csv', verbose => 1) ->field_add_computed('FULLNAME', \&calc_fn)->write(); sub calc_fn { my ($field, $hr, $stats) = @_; my $fn = $hr->{'FIRSTNAME'} . ' ' . uc($hr->{'LASTNAME'}); $stats->{'empty full name'}++ if $fn eq ' '; return $fn; } # Read a file with a lot of columns and keep only 2 columns in output Text::AutoCSV->new(in_file => 'big.csv', out_file => 'addr.csv', out_fields => ['NAME', 'ADDRESS']) ->out_header('ADDRESS', 'Postal Address') ->write();
# Detect any field containing a DateTime value and convert it to yyyy-mm-dd whatever the # input format is. Text::AutoCSV->new(in_file => 'in.csv', out_file => 'out.csv', fields_dates_auto => 1, out_dates_format => '%F')->write(); # Detect any field containing a DateTime value and convert it to a US DateTime whatever the # input format is. Text::AutoCSV->new(in_file => 'in.csv', out_file => 'out.csv', fields_dates_auto => 1, out_dates_format => '%b %d, %Y, %I:%M:%S %p', out_dates_locale => 'en')->write(); # Find dates of specific formats and convert it into yyyy-mm-dd Text::AutoCSV->new(in_file => 'raw.csv', out_file => 'cooked.csv', dates_formats_to_try => ['%d_%m_%Y', '%m_%d_%Y', '%Y_%m_%d'], out_dates_format => '%F')->write(); # Take the dates on columns 'LASTLOGIN' and 'CREATIONDATE' and convert it into French dates # (day/month/year). # Text::AutoCSV will croak if LASTLOGIN or CREATIONDATE do not contain a DateTime format. # By default, Text::AutoCSV will try 20 different formats. Text::AutoCSV->new(in_file => 'in.csv', out_file => 'out.csv', fields_dates => ['LASTLOGIN', 'CREATIONDATE'], out_dates_format => '%d/%m/%Y')->write(); # Convert 2 DateTime fields into unix standard epoch # Write -1 if DateTime is empty. sub toepoch { return $_->epoch() if $_; -1; } Text::AutoCSV->new(in_file => 'stats.csv', out_file => 'stats-epoch.csv', fields_dates => ['ATIME', 'MTIME']) ->in_map('ATIME', \&toepoch) ->in_map('MTIME', \&toepoch) ->write(); # Do the other way round from above: convert 2 fields containing unix standard epoch into a # string displaying a human-readable DateTime. my $formatter = DateTime::Format::Strptime->new(pattern => 'DATE=%F, TIME=%T'); sub fromepoch { return $formatter->format_datetime(DateTime->from_epoch(epoch => $_)) if $_ >= 0; ''; } $csv = Text::AutoCSV->new(in_file => 'stats-epoch.csv', out_file => 'stats2.csv') ->in_map('ATIME', \&fromepoch) ->in_map('MTIME', \&fromepoch) ->write();
use Text::AutoCSV 'remove_accents'; # Output 'Francais: etre elementaire, Tcheque: sluzba dum' followed by a new line. print remove_accents("Français: être élémentaire, Tchèque: služba dům"), "\n";
my $csv = Text::AutoCSV->new(%attr);
(Class method) Returns a new instance of Text::AutoCSV. The object attributes are described by the hash %attr (can be empty).
%attr
Currently the following attributes are available:
By default, Text::AutoCSV assumes the input has a header and will use the field values of this first line (the header) to work out the column internal names. These internal names are used everywhere in Text::AutoCSV to designate columns.
The values are transformed as follows:
- All accents are removed using the exportable function "remove_accents".
- Any non-alphanumeric character is removed (except underscore) and all letters are switched to upper case. The regex to do this is
s/[^[:alnum:]_]//gi; s/^.*$/\U$&/;
Thus a header line of
'Office Number 1,Office_2,Personal Number'
will produce the internal column names
'OFFICENUMBER1' (first column) 'OFFICE_2' (second column) 'PERSONALNUMBER' (third column).
The attribute "fields_hr", "fields_ar" or "fields_column_names" (only one of the three is useful at a time) allows to change this behavior.
NOTE
The removal of accents is *not* a conversion to us-ascii, see "remove_accents" for details.
Functions that are given a field name ("get_cell", "vlookup", "field_add_copy", ...) raise an error if the field requested does not exist.
SO WILL THE HASHREFS GIVEN BY Text::AutoCSV: when a function returns a hashref ("search_1hr", "get_row_hr", ...), the hash is locked with the lock_keys function of Hash::Util. Any attempt to read a non-existing key from the hash causes a croak. This feature is de-activated if you specified croak_if_error => 0 when creating Text::AutoCSV object.
lock_keys
Hash::Util
croak_if_error => 0
The name of the file to read CSV data from.
If not specified or empty, read standard input.
Example:
my $csv = Text::AutoCSV->new(in_file => 'in.csv');
File handle to read CSV data from. Normally you don't want to specify this attribute.
inh is useful if you don't like the way Text::AutoCSV opens the input file for you.
inh
open my $inh, "producecsv.sh|"; my $csv = Text::AutoCSV->new(inh => $inh);
Comma-separated list of encodings to try to read input.
Note that finding the correct encoding of any given input is overkill. This script just tries encodings one after the other, and selects the first one that does not trigger a warning during reading of input. If all produce warnings, select the first one.
The encoding chosen is used in output, unless attribute "out_encoding" is specified.
Value by default: 'UTF-8,latin1'
IMPORTANT
If one tries something like encoding => 'latin1,UTF-8', it'll almost never detect UTF-8 because latin1 rarely triggers warnings during reading. It tends to be also true with encodings like UTF-16 that can remain happy with various inputs (sometimes resulting in Western languages turned into Chinese text).
encoding => 'latin1,UTF-8'
Ultimately this attribute should be used with a unique value. The result when using more than one value can produce weird results and should be considered experimental.
my $csv = Text::AutoCSV->new(in_file => 'w.csv', encoding => 'UTF-16');
Adds a via to the file opening instruction performed by Text::AutoCSV. You don't want to use it under normal circumstances.
via
The value should start with a ':' character (Text::AutoCSV won't add one for you).
Value by default: none
my $csv = Text::AutoCSV->new(in_file => 'in.csv', via => ':raw:perlio:UTF-32:crlf');
If true, just ignore completely encoding and don't try to alter I/O operations with encoding considerations (using binmode instruction). Note that if inh attribute is specified, then Text::AutoCSV will consider the caller manages encoding for himself and dont_mess_with_encoding will be automatically set, too.
binmode
This attribute does not mean perl will totally ignore encoding and would consider character strings as bytes for example. The meaning of "dont_mess_with_encoding" is that Text::AutoCSV itself will totally ignore encoding matters, and leave it entirely to Perl' default.
Value by default:
0 if inh attribute is not set 1 if inh attribute is set
my $csv = Text::AutoCSV->new(in_file => 'in.csv', dont_mess_with_encoding => 1);
Specify the CSV separator character. Turns off separator auto-detection. This attribute is passed as is to Text::CSV->new().
Text::CSV->new()
my $csv = Text::AutoCSV->new(in_file => 'in.csv', sep_char => ';');
Specify the field quote character. This attribute is passed as is to Text::CSV->new().
Value by default: double quote ('"')
my $csv = Text::AutoCSV->new(in_file => 'in.csv', quote_char => '\'');
Specify the escape character. Turns off escape character auto-detection. This attribute is passed as is to Text::CSV->new().
Value by default: backslash ('\\')
my $csv = Text::AutoCSV->new(in_file => 'in.csv', escape_char => '"');
Text::CSV object to use. Normally you don't want to specify this attribute.
By default, Text::AutoCSV will manage creating such an object and will work hard to detect the parameters it requires.
Defining in_csvobj attribute turns off separator character and escape character auto-detection.
in_csvobj
Using this attribute workarounds Text::AutoCSV philosophy a bit, but you may need it in case Text::AutoCSV behavior is not suitable for Text::CSV creation.
my $tcsv = Text::CSV->new(); my $acsv = Text::AutoCSV->new(in_file => 'in.csv', in_csvobj => $tcsv);
If true, Text::AutoCSV assumes the input has a header line.
Value by default: 1
my $csv = Text::AutoCSV->new(in_file => 'in.csv', has_headers => 0);
(Only if input has a header line) Hash ref that contains column internal names along with a regular expression to find it in the header line. For example if you have:
my $csv = Text::AutoCSV->new(in_file => 'in.csv', fields_hr => {'PHONE OFFICE' => '^office phone nu', 'PHONE PERSONAL' => '^personal phone nu'});
And the header line is
'Personal Phone Number,Office Phone Number'
the column name 'PHONE OFFICE' will designate the second column and the column name 'PHONE PERSONAL' will designate the first column.
You can choose column names like 'Phone Office' and 'Phone Personal' as well.
The regex search is case insensitive.
(Only if input has a header line) Array ref that contains column internal names. The array is used to create a hash ref of the same kind as "fields_hr", by wrapping the column name in a regex. The names are surrounded by a leading '^' and a trailing '$', meaning, the name must match the entire field name.
For example
fields_ar => ['OFFICENUMBER', 'PERSONALNUMBER']
is strictly equivalent to
fields_hr => {'OFFICENUMBER' => '^officenumber$', 'PERSONALNUMBER' = '^personalnumber$'}
fields_ar is useful if the internal names are identical to the file column names. It avoids repeating the names over and over as would happen if using "fields_hr" attribute.
fields_ar
You might wonder why using fields_ar as opposed to Text::AutoCSV default' mechanism. There are two reasons for that:
1- Text::AutoCSV removes spaces from column names, and some people may want another behavior. A header name of 'Phone Number' will get an internal column name of 'PHONENUMBER' (default behavior, if none of fields_hr, fields_ar and fields_column_names attributes is specified), and one may prefer 'PHONE NUMBER' or 'phone number' or whatsoever.
2- By specifying a list of columns using either of fields_hr or fields_ar, you not only map column names as found in the header line to internal column names: you also request these columns to be available. If one of the requested columns cannot be found, Text::AutoCSV will croak (default) or print an error and return an undef object (if created with croak_if_error => 0).
Array ref of column internal names, in the order of columns in file. This attribute works like the column_names attribute of Text::CSV. It'll just assign names to columns one by one, regardless of what the header line contains. It'll work also if the file has no header line.
column_names
my $csv = Text::AutoCSV->new(in_file => 'in.csv', fields_column_names => ['My COL1', '', 'My COL3']);
Output file when executing the "write" method.
If not specified or empty, write to standard output.
my $csv = Text::AutoCSV->new(in_file => 'in.csv', out_file => 'out.csv');
File handle to write CSV data to when executing the "write" method. Normally you don't want to specify this attribute.
outh is useful if you don't like the way Text::AutoCSV opens the output file for you.
outh
my $outh = open "myin.csv', ">>"; my $csv = Text::AutoCSV->new(in_file => 'in.csv', has_headers => 0, outh => $outh);
Enforce the encoding of output.
Value by default: input encoding
my $csv = Text::AutoCSV->new(in_file => 'in.csv', out_file => 'out.csv', out_encoding => 'UTF-16');
Enforce BOM (Byte-Order-Mark) on output, when it is UTF8. If output encoding is not UTF-8, this attribute is ignored.
UTF-8 needs no BOM (there is no Byte-Order in UTF-8), and in practice, UTF8-encoded files rarely have a BOM.
Using this attribute is not recommended. It is provided for the sake of completeness, and also to produce Unicode files Microsoft EXCEL will be happy to read.
At first sight it would seem more logical to make EXCEL happy with something like this:
out_encoding => 'UTF-16'
But... While EXCEL will identify UTF-16 and read it as such, it will not take into account the BOM found at the beginning. In the end the first cell will have 2 useless characters prepended. The only solution the author knows to workaround this issue if to use UTF-8 as output encoding, and enforce a BOM. That is, use:
..., out_encoding => 'UTF-8', out_utf8_bom => 1, ...
Enforce the output CSV separator character.
Value by default: input separator
my $csv = Text::AutoCSV->new(in_file => 'in.csv', out_file => 'out.csv', out_sep_char => ',');
Enforce the output CSV quote character.
Value by default: input quote character
my $csv = Text::AutoCSV->new(in_file => 'in.csv', out_file => 'out.csv', out_quote_char => '"');
Enforce the output CSV escape character.
Value by default: input escape character
my $csv = Text::AutoCSV->new(in_file => 'in.csv', out_file => 'out.csv', out_escape_char_char => '\\');
If true, quote all fields of output (set always_quote of Text::CSV).
If false, don't quote all fields of output (don't set always_quote of Text::CSV).
always_quote
Value by default: same as what is found in input
While reading input, Text::AutoCSV works out whether or not all fields were quoted. If yes, then the output Text::CSV object has the always_quote attribute set, if no, then the output Text::CSV object does not have this attribute set.
my $csv = Text::AutoCSV->new(in_file => 'in.csv', out_file => 'out.csv', out_always_quote => 1);
If true, when writing output, write a header line on first line.
If false, when writing output, don't write a header line on first line.
Value by default: same as has_headers attribute
Example 1
Read standard input and write to standard output, removing the header line.
Text::AutoCSV->new(out_has_headers => 0)->write();
Example 2
Read standard input and write to standard output, adding a header line.
Text::AutoCSV->new(fields_column_names => ['MYCOL1', 'MYCOL2'], out_has_headers => 1)->write();
If true, non-existent column values are set to an empty string instead of undef. It is also done on extra fields that happen to have an undef value (for example when the target of a linked field is not found).
Note this attribute does not work on callback functions output set with "in_map": for example empty DateTime values (on fields identified as containing a date/time, see dates_* attributes below) are set to undef, even while no_undef is set. Indeed setting it to an empty string while non-empty values would contain a Datetime object would not be clean. An empty value in a placeholder containing an object must be undef.
dates_*
undef
no_undef
Since version 1.1.5 of Text::AutoCSV, no_undef is examined when sending parameter ($_) to "in_map" callback: an undef value is now passed as is (as undef), unless no_undef is set. If no_undef is set, and field value is undef, then $_ is set to the empty string ('') when calling callback defined by "in_map". This new behavior was put in place to be consistent with what is being done with DateTime values.
Value by default: 0
my $csv = Text::AutoCSV->new(in_file => 'in.csv', no_undef => 1);
To be set to a ref sub. Each time a record is read from input, call read_post_update_hr to update the hash ref of the record. The sub is called with 2 arguments: the hash ref to the record value and the hash ref to stats.
read_post_update_hr
The stats allow to count events and are printed in the end of reading in case Text::AutoCSV is called in verbose mode (verbose => 1).
verbose => 1
For example, the read_post_update_hr below will turn column 'CITY' values in upper case and count occurences of empty cities in stat display:
Text::AutoCSV->new(in_file => 'addresses.csv', read_post_update_hr => \&updt, verbose => 1) ->write(); sub updt { my ($hr, $stats) = @_; $hr->{'CITY'} =~ s/^.*$/\U$&/; $stats->{'empty city encountered'}++ if $hr->{'CITY'} eq ''; }
You cannot create a field this way. To create a field, you have to use the member functions "field_add_link", "field_add_copy" or "field_add_computed".
If you wish to manage some updates at field level, consider registering update functions with "in_map" and "out_map" member functions. These functions register callbacks that work at field level and with $_ variable (thus the callback function invoked is AutoCSV-agnostic).
"in_map" updates a field after read, "out_map" updates the field content before writing it.
To set to a sub ref that'll be executed each time a record is read from input. It is executed after "read_post_update_hr". The sub is called with 2 arguments: the hash ref to the record value and the hash ref to stats.
Note "read_post_update_hr" is meant for updating record fields just after reading, whereas "walker_hr" is read-only.
The stats allow to count events and are printed in the end of reading in case Text::AutoCSV is called in verbose mode (verbose => 1). If the "verbose" attribute is not set, the stats are not displayed, however you can get stats by calling the get_stats function.
The example below will count in the stats the number of records where the 'CITY' field is empty. Thanks to verbose => 1 attribute, at the end of reading the stats are displayed.
my $csv = Text::AutoCSV->new(in_file => 'addresses.csv', walker_hr => \&walk1, verbose => 1)->read(); sub walk1 { my ($hr, $stats) = @_; $stats->{'empty city'}++ if $hr->{'CITY'} eq ''; }
To set to a sub ref that'll be executed each time a record is read from input. It is executed after "read_post_update_hr". The sub is called with 2 arguments: the array ref to the record value and the hash ref to stats.
Note "read_post_update_hr" is meant for updating record fields just after reading, whereas walker_hr is read-only.
walker_hr
The stats allow to count events and are printed in the end of reading in case Text::AutoCSV is called in verbose mode (verbose => 1). If the "verbose" attribute is not set, the stats are lost.
The array ref contains values in their natural order in the CSV. To be used with the column names, you have to use "get_fields_names" member function.
The example below will count in the stats the number of records where the 'CITY' field is empty. Thanks to verbose => 1 attribute, at the end of reading the stats are displayed. It produces the exact same result as the example in walker_hr attribute, but it uses walker_ar.
use List::MoreUtils qw(first_index); my $csv = Text::AutoCSV->new(in_file => 'addresses.csv', walker_ar => \&walk2, verbose => 1); my @cols = $csv->get_fields_names(); my $idxCITY = first_index { /^city$/i } @cols; die "No city field!??" if $idxCITY < 0; $csv->read(); sub walk2 { my ($ar, $stats) = @_; $stats->{'empty city'}++ if $ar->[$idxCITY] eq ''; }
Alias of "out_filter".
To be set to a ref sub. Before writing a record to output, out_filter is called and the record gets writen only if out_filter return value is true. The sub is called with 1 argument: the hash ref to the record value.
out_filter
For example, if you want to output only records where the 'CITY' column value is Grenoble:
Text::AutoCSV->new(in_file => 'addresses.csv', out_file => 'grenoble.csv', out_filter => \&filt)->write(); sub filt { my $hr = shift; return 1 if $hr->{'CITY'} =~ /^grenoble$/i; return 0; }
Alias of "out_fields".
Set to an array ref. List fields to write to output.
Fields are written in their order in the array ref, the first CSV column being the first element in the array, and so on. Fields not listed in out_fields are not written in output.
You can use empty field names to have empty columns in output.
Value by default: none, meaning, all fields are output in their natural order. What is natural order? It is the input order for fields that were read from input, and the order in which they got created for created fields.
Text::AutoCSV->new(in_file => 'allinfos.csv', out_file => 'only-addresses.csv', out_fields => [ 'NAME', 'ADDRESS' ] )->write();
Array reference to a list of fields to sort output with.
At the moment this feature is a bit of a hack (no option to make sort descending or ascending, numeric or text, and it is not part of test plan).
Text::AutoCSV->new(in_file => 'names.csv', out_file => 'sortednames.csv', out_orderby => [ 'LASTNAME', 'FIRSTNAME']);
If true, searches are case sensitive by default. Searches are done by the member functions "search", "search_1hr", "vlookup", and linked fields ("field_add_link").
The search functions can also be called with the option "case", that takes precedence over the object-level search_case attribute value. See "vlookup" help.
search_case
Value by default: 0 (by default searches are case insensitive)
my $csv = Text::AutoCSV->new(in_file => 'in.csv', search_case => 1);
If true, searches ignore the presence of leading or trailing spaces in values.
The search functions can also be called with the option "trim", that takes precedence over the object-level search_trim attribute value. See "vlookup" help.
search_trim
Value by default: 1 (by default searches ignore leading and trailing spaces)
my $csv = Text::AutoCSV->new(in_file => 'in.csv', search_trim => 0);
If true, empty fields are not included in the search indexes.
The search functions can also be called with the option "ignore_empty", that takes precedence over the object-level search_ignore_empty attribute value. See "vlookup" help.
search_ignore_empty
Value by default: 1 (by default, search of the value '' will find nothing)
my $csv = Text::AutoCSV->new(in_file => 'in.csv', search_ignore_empty => 0);
If true, accents are ignored by search indexes.
The search functions can also be called with the option "ignore_accents", that takes precedence over the object-level search_ignore_accents attribute value. See "vlookup" help.
search_ignore_accents
Value by default: 1 (by default, accents are ignored by search functions)
my $csv = Text::AutoCSV->new(in_file => 'in.csv', search_ignore_accents => 0);
When a search is done with a unique value to return (field_add_link member function behavior or return value of vlookup), default value of option "value_if_not_found". See "vlookup".
When a search is done with a unique value to return (field_add_link member function behavior or return value of vlookup), default value of option "value_if_found". See "vlookup".
This attribute is extremly unusual. Once you've provided it, all vlookups and the target field value of fields created with field_add_link will all be populated with the value provided with this option.
Don't use it unless you know what you are doing.
When a search is done with a unique value to return (field_add_link member function behavior or return value of search_1hr and vlookup), default value of option "ignore_ambiguous". See "vlookup".
When a search is done with a unique value to return (field_add_link member function behavior or return value of vlookup), default value of option "value_if_ambiguous". See "vlookup".
Array ref of field names that contain a date.
Once the formats of these fields is known (auto-detection by default), each of these fields will get a specific "in_map" sub that converts the text in a DateTime object and a "out_map" sub that converts back from DateTime to text.
The "out_map" given to a DateTime field is "defensive code": normally, "in_map" converts text into a DateTime object and "out_map" does the opposite, it takes a DateTime object and converts it to text. If ever "out_map" encounters a value that is not a DateTime object, it'll just stringify it (evaluation in a string context), without calling its DateTime formatter.
If the format cannot be detected for a given field, output an error message and as always when an error occurs, croak (unless "croak_if_error" got set to 0).
my $csv = Text::AutoCSV->new(in_file => 'logins.csv', fields_dates => ['LASTLOGIN', 'CREATIONDATE']);
Boolean value. If set to 1, will detect dates formats on all fields. Fields in which a DateTime format got detected are then managed as if they had been being listed in "fields_dates" attribute: they get an appropriate "in_map" sub and a "out_map" sub to convert to and from DateTime (see "fields_dates" attribute above).
fields_dates_auto looks for DateTime on all fields, but it expects nothing: it won't raise an error if no field is found that contains DateTime.
fields_dates_auto
my $csv = Text::AutoCSV->new(in_file => 'logins.csv', fields_dates_auto => 1);
Relevant only if "fields_dates_auto" is set.
Normally when "fields_dates_auto" is set, the input is read completely to make sure auto-detection produces a reliable result. If fields_dates_auto_optimize is set, this reading pass will stop as soon as there is no ambiguity left. That is, for every fields in input, the date format (or the fact that no date format is suitable) is known.
fields_dates_auto_optimize
Using this option is a bit risky because it could trigger a date format detection that later in the input, would turn out to be wrong. Should that be the case, strange errors will occur, that are not easy to understand. Use it at your own risk.
my $csv = Text::AutoCSV->new(in_file => 'logins.csv', fields_dates_auto => 1, fields_dates_auto_optimize => 1);
Array ref of string formats.
Text::AutoCSV has a default built-in list of 20 date formats to try and 6 time formats (also it'll combine any date format with any time format).
dates_formats_to_try will replace Text::AutoCSV default format-list will the one you specify, in case the default would not produce the results you expect.
dates_formats_to_try
The formats are written in Strptime format.
Value by default (see below about the role of the pseudo-format ''):
[ '', '%Y-%m-%d', '%Y.%m.%d', '%Y/%m/%d', '%m.%d.%y', '%m-%d-%Y', '%m.%d.%Y', '%m/%d/%Y', '%d-%m-%Y', '%d.%m.%Y', '%d/%m/%Y', '%m-%d-%y', '%m/%d/%y', '%d-%m-%y', '%d.%m.%y', '%d/%m/%y', '%Y%m%d%H%M%S', '%b %d, %Y', '%b %d %Y', '%b %d %T %Z %Y', '%d %b %Y', '%d %b, %Y' ]
The empty format (empty string) has a special meaning: when specified, Text::AutoCSV will be able to identify fields that contain only a time (not preceeded by a date).
Note
Format identification is over only when there is no more ambiguity. So the usual pitfall of US versus French dates (month-day versus day-month) gets resolved only when a date is encountered that disambiguates it (a date of 13th of the month or later).
Example with a weird format that uses underscores to separate elements, using either US (month, day, year), French (day, month, year), or international (year, month, day) order:
my $csv = Text::AutoCSV->new(in_file => 'logins.csv', dates_formats_to_try => ['%d_%m_%Y', '%m_%d_%Y', '%Y_%m_%d']);
Same as "dates_formats_to_try" but instead of replacing the default list of formats used during detection, it is added to this default list.
You want to use this attribute if you need a specific DateTime format while continuing to benefit from the default list.
Text::AutoCSV will identify a given Datetime format only when there is no ambiguity, meaning, one unique Datetime format matches (all other failed). Adding a format that already exists in the default list will prevent the format from being identified, as it'll always be ambiguous. See "dates_formats_to_try" for the default list of formats.
my $csv = Text::AutoCSV->new(in_file => 'logins.csv', dates_formats_to_try_supp => ['%d_%m_%Y', '%m_%d_%Y', '%Y_%m_%d']);
If set to 1, DateTime auto-detection will ignore trailing text that may follow detected DateTime-like text.
Value by default: 1 (do ignore trailing chars)
my $csv = Text::AutoCSV->new(in_file => 'logins.csv', dates_ignore_trailing_chars => 0);
If set to 1, look for times when detecting DateTime format. That is, whenever a date format candidate is found, a longer candidate that also contains a time (after the date) is tested.
Value by default: 1 (do look for times when auto-detecting DateTime formats)
my $csv = Text::AutoCSV->new(in_file => 'logins.csv', dates_search_time => 0);
Comma-separated string of locales to test when detecting DateTime formats. Ultimately, Text::AutoCSV will try all combinations of date formats, times and locales.
Value by default: none (use perl default locale)
my $csv = Text::AutoCSV->new(in_file => 'logins.csv', dates_locales => 'fr,de,en');
Boolean. If true, a date made only of 0s is regarded as being empty.
For example if dates_zeros_ok is False, then a date like 0000-00-00 will be always incorrect (as the day and month are out of bounds), therefore a format like '%Y-%m-%d' will never match for the field.
dates_zeros_ok
Conversely if dates_zeros_ok is true, then a date like 0000-00-00 will be processed as if being the empty string, thus the detection of format will work and when parsed, this "full of zeros" dates will be processed the same way as the empty string (= value will be undef).
"0s dates" are evaluated to undef when parsed, thus when converted back to text (out_map), they are set to an empty string, not to the original value.
my $csv = Text::AutoCSV->new(in_file => 'in.csv', dates_zeros_ok => 0);
Enforce the format of dates in output, for all fields that contain a DateTime value.
The format is written in Strptime format.
Value by default: none (by default, use format detected on input)
# Detect any field containing a DateTime value and convert it to yyyy-mm-dd whatever the # input format is. Text::AutoCSV->new(in_file => 'in.csv', out_file => 'out.csv', fields_dates_auto => 1, out_dates_format => '%F')->write();
Taken into account only if "out_dates_format" is used.
Sets the locale to apply on "out_dates_format".
Value by default: none (by default, use the locale detected on input)
# Detect any field containing a DateTime value and convert it to a US DateTime whatever the # input format is. Text::AutoCSV->new(in_file => 'in.csv', out_file => 'out.csv', fields_dates_auto => 1, out_dates_format => '%b %d, %Y, %I:%M:%S %p', out_dates_locale => 'en')->write();
If true, stops the program execution in case of error.
If set to zero (croak_if_error => 0), errors are displayed as warnings. This printing can then be affected by setting the "quiet" attribute.
If true, get Text::AutoCSV to be a bit talkative instead of speaking only when warnings and errors occur. Verbose output is printed to STDERR by default, this can be tuned with the "infoh" attribute.
my $csv = Text::AutoCSV->new(in_file => 'in.csv', verbose => 1);
File handle to display program's verbose output. Has effect *mainly* with attribute verbose => 1.
Note infoh is used to display extra information in case of error (if a field does not exist, Text::AutoCSV will display the list of existing fields). If you don't want such output, you can set infoh to undef.
infoh
Value by default: \*STDERR
open my $infoh, ">", "log.txt"; my $csv = Text::AutoCSV->new(in_file => 'in.csv', infoh => $infoh);
If true, don't display warnings and errors, unless croaking.
If "croak_if_error" attribute is set (as per default), still, Text::AutoCSV will produce output (on STDERR) when croaking miserably.
When using croak_if_error => 0, errors are processed as warnings and if "quiet" is set (in addition to "croak_if_error" being set to 0), there'll be no output. Note this way of working is not recommended, as things can go wrong without any notice to the caller.
my $csv = Text::AutoCSV->new(in_file => 'in.csv', quiet => 1);
If true, Text::AutoCSV will perform one reading of the input. If other readings are triggered, it'll raise an error and no reading will be done. Should that be the case (you ask Text::AutoCSV to do something that'll trigger more than one reading of input), Text::AutoCSV will croak as is always the case if an error occurs.
Normally Text::AutoCSV will do multiple reads of input to work out certain characteristics of the CSV: guess of encoding and guess of escape character.
Also if member functions like "field_add_link", "field_add_copy", "field_add_computed", "read" or "write" are called after input has already been read, it'll trigger further reads as needed.
If one wishes a unique read of the input to occur, one_pass attribute is to be set.
When true, encoding will be assumed to be the first one in the provided list ("encoding" attribute), if no encoding attribute is provided, it'll be the first one in the default list, to date, it is UTF-8.
When true, and if attribute "escape_char" is not set, escape_char will be assumed to be '\\' (backslash).
By default, one_pass is set if inh attribute is set (caller provides the input file handle of input) or if input file is stdin (in_file attribute not set or set to an empty string).
0 if inh attribute is not set and in_file attribute is set to a non empty string 1 if inh attribute is set or in_file is not set or set to an empty string
my $csv = Text::AutoCSV->new(in_file => 'in.csv', one_pass => 1);
$csv->read();
Read input entirely.
Return value
Returns the object itself in case of success. Returns undef if error.
Callback functions (when defined) are invoked, in the following order:
"read_post_update_hr", intended to do updates on fields values after each record read
"walker_ar", called after each record read, with an array ref of fields values
"walker_hr", called after each record read, with a hash ref of fields values
# Do nothing - just check CSV can be read successfully Text::AutoCSV->new(in_file => 'in.csv')->read();
$csv->read_all_in_mem();
Created in version 1.1.5. Before, existed only as _read_all_in_mem, meaning, was private.
Read input entirely, as with "read" function, but enforcing content to be kept in-memory.
Having the content kept in-memory is implied by search functions ("vlookup" for example). With read_all_in_mem you can enforce this behavior without doing a fake search.
read_all_in_mem
$csv->reset_next_record_hr();
Reset the internal status to start from the beginning with "get_next_record_hr". Used in conjunction with "get_next_record_hr".
my $hr = $csv->get_next_record_hr(\$opt_key);
Get the next record content as a hash ref. $hr is undef when the end of records has been reached.
$hr
When specified, $opt_key is set to the current (returned) record key.
$opt_key
You do not need to call "reset_next_record_hr" once before using get_next_record_hr.
get_next_record_hr
Therefore "reset_next_record_hr" is useful only if you wish to restart from the beginning before you've reached the end of the records.
NOTE bis
"walker_hr" allows to execute some code each time a record is read, and it better fits with Text::AutoCSV philosophy. Using a loop with get_next_record_hr is primarily meant for Text::AutoCSV internal usage. Also when using this mechanism, you get very close to original Text::CSV logic, that makes Text::AutoCSV less useful.
A hashref of the record, or undef once there's no more record to return.
while (my $hr = $csv->get_next_record_hr()) { say Dumper($hr); }
$csv->write();
Write input into output.
- If the content is not in-memory at the time write() is called:
Each record is read (with call of "read_post_update_hr", "walker_ar" and "walker_hr") and then written. The read-and-write is done in sequence, each record is written to output before the next record is read from input.
- If the content is in-memory at the time write() is called:
No "read" operation is performed, instead, records are directly written to output.
If defined, "out_filter" is called for each record. If the return value of "out_filter" is false, the record is not written.
# Copy input to output. # As CSV is parsed in-between, this copy also checks a number of characteristics about the # input, as opposed to a plain file copy operation. Text::AutoCSV->new(in_file => 'in.csv', out_file => 'out.csv')->write();
$csv->out_header($field, $header);
Set the header text of $field to $header.
$field
$header
By default, the input header value is rewritten as is to output. out_header allows you to change it.
out_header
Returns the object itself.
Text::AutoCSV->new(in_file => 'in.csv', out_file => 'out.csv') ->out_header('LOGIN', 'Login') ->out_header('FULLNAME', 'Full Name') ->write();
$csv->print_id();
Print out a description of input. Write to \*STDERR by default or to "infoh" attribute if set.
The description consists in a list of a few characteristics (CSV separator and the like) followed by the list of columns with the details of each.
Example of output:
If you go to the utils directory of this module and execute the following:
utils
./csvcopy.pl -i f1.csv -l "1:,A->B,f2.csv" --id
You will get this output:
-- f1.csv: sep_char: , escape_char: \ in_encoding: UTF-8 is_always_quoted: no # FIELD HEADER EXT DATA DATETIME FORMAT DATETIME LOCALE - ----- ------ -------- --------------- --------------- 0 TIMESTAMP timestamp %Y%m%d%H%M%S 1 A a 2 B b 3 C c 4 D d %d/%m/%Y 5 1:SITE 1:SITE link: f2.csv, chain: A->B->* (SITE) 6 1:B 1:B link: f2.csv, chain: A->B->* (B)
$csv->field_add_computed($new_field, $subref);
$new_field is the name of the created field.
$new_field
$subref is a reference to a sub that'll calculate the new field value.
$subref
Add a field calculated from other fields values. The subref runs like this:
sub func { # $new_field is the name of the field (allows to use one subref for more than one field # calculation). # $hr is a hash ref of fields values. # $stats is a hash ref that gets printed (if Text::AutoCSV is created with verbose => 1) # in the end. my ($new_field, $hr, $stats) = @_; my $field_value; # ... compute $field_value return $field_value; }
# Add a field for the full name, made of the concatenation of the # first name and the last name. Text::AutoCSV->new(in_file => 'dirpeople.csv', out_file => 'dirwithfn.csv', verbose => 1) ->field_add_computed('FULLNAME', \&calc_fn)->write(); sub calc_fn { my ($new_field, $hr, $stats) = @_; die "Man, you are in serious trouble!" unless $new_field eq 'FULLNAME'; my $fn = $hr->{'FIRSTNAME'} . ' ' . uc($hr->{'LASTNAME'}); $stats->{'empty full name'}++ if $fn eq ' '; return $fn; }
$csv->field_add_copy($new_field, $src_field, $opt_subref);
$new_field if the name of the new field.
$src_field is the name of the field being copied.
$src_field
$opt_subref is optional. It is a reference to a sub that takes one string (the value of $src_field) and returns a string (the value assigned to $new_field).
$opt_subref
field_add_copy is a special case of "field_add_computed". The advantage of field_add_copy is that it relies on a sub that is Text::AutoCSV "unaware", just taking one string as input and returning another string as output.
field_add_copy
The current field value is passed to field_add_copy in $_.
A call to
$csv->field_add_copy($new_field, $src_field, $subref);
is equivalent to
$csv->field_add_computed($new_field, \&subref2); sub subref2 { my (undef, $hr) = @_; local $_ = $hr->{$src_field}; return $subref->(); }
Example of a field copy + pass copied field in upper case and surround content with <<>>:
my $csv = Text::AutoCSV->new(in_file => 'dirpeople.csv', out_file => 'd2.csv'); $csv->field_add_copy('UCLAST', 'LASTNAME', \&myfunc); $csv->write(); sub myfunc { s/^.*$/<<\U$&>>/; $_; }
Note that the calls can be chained as most member functions return the object itself upon success. The example above is equivalent to:
Text::AutoCSV->new(in_file => 'dirpeople.csv', out_file => 'd2.csv') ->field_add_copy('UCLAST', 'LASTNAME', \&myfunc) ->write(); sub myfunc { s/^.*$/<<\U$&>>/; $_; }
$csv->field_add_link($new_field, $chain, $linked_file, \%opts);
$new_field is the name of the new field.
$chain is the CHAIN of the link, that is: 'LOCAL->REMOTE->PICK' where:
$chain
LOCAL is the field name to read the value from.
LOCAL
REMOTE is the linked field to find the value in. This field belongs to $linked_file.
REMOTE
PICK is the field from which to read the value of, in the record found by the search. This field belongs to $linked_file.
PICK
If $new_field is undef, the new field name is the name of the third field of $chain (PICK).
$linked_file is the name of the linked file, that gets read in a Text::AutoCSV object created on-the-fly to do the search on. $linked_file can also be a Text::AutoCSV object that you created yourself, allowing for more flexibility. Example:
$linked_file
my $lcsv = Text::AutoCSV->new(in_file => 'logins.csv', case => 1); $csv->field_add_link('NAME', 'ID->LOGIN->DISPLAYNAME', $lcsv);
\%opts is a hash ref of optional attributes. The same values can be provided as with vlookup.
\%opts
If set to 1, searches will ignore leading and trailing spaces. That is, a LOCAL value of ' x ' will match with a REMOTE value of 'x'.
If option is not present, use "search_value_if_not_found" attribute of object (default value: 1).
$csv->field_add_link('NAME', 'ID->LOGIN->DISPLAYNAME', 'logins.csv', { trim => 0 });
If set to 1, searches will take the case into account. That is, a LOCAL value of 'X' will not match with a REMOTE value of 'x'.
If option is not present, use "search_case" attribute of object (default value: 0).
$csv->field_add_link('NAME', 'ID->LOGIN->DISPLAYNAME', 'logins.csv', { case => 1 });
If set to 1, empty values won't match. That is, a LOCAL value of '' will not match with a REMOTE value of ''.
If option is not present, use "search_ignore_empty" attribute of object (default value: 1).
$csv->field_add_link('NAME', 'ID->LOGIN->DISPLAYNAME', 'logins.csv', { ignore_empty => 0 });
If the searched value is not found, the value of the field is undef, that produces an empty string at write time. Instead, you can specify the value.
If option is not present, use "search_value_if_not_found" attribute of object (default value: undef).
$csv->field_add_link('NAME', 'ID->LOGIN->DISPLAYNAME', 'logins.csv', { value_if_not_found => '<not found!>' });
If the searched value is found, you can specify the value to return.
If option is not present, use "search_value_if_found" attribute of object (default value: none).
Although the PICK field is ignored when using this option, you must specify it any way.
$csv->field_add_link('NAME', 'ID->LOGIN->DISPLAYNAME', 'logins.csv', { value_if_not_found => '0', value_if_found => '1' });
If the searched value is found in more than one record, the value of the field is undef, that produces an empty string at write time. Instead, you can specify the value.
If option is not present, use "search_value_if_ambiguous" attribute of object (default value: undef).
$csv->field_add_link('NAME', 'ID->LOGIN->DISPLAYNAME', 'logins.csv', { value_if_ambiguous => '<ambiguous!>' });
Boolean value. If ignore_ambiguous is true and the searched value is found in more than one record, then, silently fall back on returning the value of the first record. Obviously if ignore_ambiguous is true, then the value of "value_if_ambiguous" is ignored.
ignore_ambiguous
If option is not present, use "search_ignore_ambiguous" attribute of object (default value: 1).
$csv->field_add_link('NAME', 'ID->LOGIN->DISPLAYNAME', 'logins.csv', { ignore_ambiguous => 1 });
Example with multiple options:
$csv->field_add_link('NAME', 'ID->LOGIN->DISPLAYNAME', 'logins.csv', { value_if_not_found => '?', ignore_ambiguous => 1 });
Example of field_add_link usage:
my $nom_compose = 0; my $zip_not_found = 0; Text::AutoCSV->new(in_file => 'pers.csv', walker_hr => \&walk) ->field_add_link('MYCITY', 'ZIP->ZIPCODE->CITY', 'zips.csv')->read(); sub walk { my $hr = shift; $nom_compose++ if $hr->{'NAME'} =~ m/[- ]/; $zip_not_found++ unless defined($hr->{'MYCITY'}); } print("Number of persons with a multi-part name: $nom_compose\n"); print("Number of persons with unknown zipcode: $zip_not_found\n");
$csv->links($prefix, $chain, $linked_file, \%opts);
$prefix is the name to add to joined fields
$prefix
$chain is the JOINCHAIN of the link, that is: 'LOCAL->REMOTE' where:
As opposed to "field_add_link", there is no PICK part, as all fields of target are read.
As opposed to Text::AutoCSV habits of croaking whenever a field name is duplicate, here, the duplicates are resolved by appending _2 to the joined field name if it already exists. If _2 already exists, too, then _3 is appended instead, and so on, until a non-duplicate is found. This mechanism is executed given the difficulty to control all field names when joining CSVs.
$linked_file and \%opts work exactly the same way as for "field_add_link", see "field_add_link" for help.
This function used to be called join but got renamed to avoid clash with perl' builtin join.
join
Text::AutoCSV->new(in_file => 'pers.csv', out_file => 'pers_with_city.csv') ->links('Read from zips.csv:', 'ZIP->ZIPCODE', 'zips.csv')->write();
my $enc = $csv->get_in_encoding();
Return the string of input encoding, for example 'latin2' or 'UTF-8', etc.
my $f = $csv->get_in_file_disp();
Return the printable name of in_file.
my $s = $csv->get_sep_char();
Return the string of the input CSV separator character, for example ',' or ';'.
my $e = $csv->get_escape_char();
Return the string of the input escape character, for example '"' or '\\'.
my $a = $csv->get_is_always_quoted();
Return 1 if all fields of input are always quoted, 0 otherwise.
my @cd = get_coldata();
Return an array that describes each column, from the first one (column 0) to the last.
Each element of the array is itself an array ref that contains 5 elements:
0: Name of the field (as accessed in *_hr functions) 1: Content of the field in the header line (if input has a header line) 2: Column content type, shows some meta-data of fields created with field_add_* functions 3: Datetime format detected, if ever, in the format Strptime 4: Locale of DateTime format detected, if ever 5: Multiline field: '1' if not, 'm' if newlines encountered in the field
my $n = $csv->get_pass_count();
Return the number of input readings done. Useful only if you're interested in Text::AutoCSV internals.
my $m = $csv->get_in_mem_record_count();
Return the number of records currently stored in-memory. Useful only if you're interested in Text::AutoCSV internals.
my $mm = $csv->get_max_in_mem_record_count();
Return the maximum number of records ever stored in-memory. Indeed this number can decrease: certain functions like field_add* member-functions discard in-memory content. Useful only if you're interested in Text::AutoCSV internals.
my @f = $csv->get_fields_names();
Return an array of the internal names of the columns.
my $name = $csv->get_field_name($n);
Return the $n-th column name, the first column being number 0.
$n
# Get the field name of the third column my $col = $csv->get_field_name(2);
my %stats = $csv->get_stats();
Certain callback functions provide a parameter to record event count: "field_add_computed", "read_post_update_hr", "walker_ar" and "walker_hr". By default, these stats are displayed if Text::AutoCSV got created with attribute verbose => 1. get_stats() returns the statistics hash of the object.
As opposed to most functions that trigger input reading automatically (search functions and other get_* functions), get_stats just returns you the stats as it is, regardless of whether some execution already occured.
get_stats
my $nb_rows = $csv->get_nb_rows();
Gives the number of rows of the input. Does not trigger any reading - just provides the number of rows as known at the moment of the call. If unknown, return undef. Typically, the number of rows is known after doing the initial detection of CSV options (escape character, etc.), or, after doing one complete reading.
The header line counts for one row.
As some fields can contain new lines, this number is not necessarily identical to the number of lines.
$csv->set_walker_ar($subref);
Normally one wants to define it at object creation time using "walker_ar" attribute. set_walker_ar allows to assign the attribute walker_ar after object creation.
set_walker_ar
See attribute "walker_ar" for help about the way $subref should work.
# Calculate the total of the two first columns, the first column being money in and the # second one being money out. my ($actif, $passif) = (0, 0); $csv->set_walker_ar(sub { my $ar = $_[0]; $actif += $ar->[0]; $passif += $ar->[1]; })->read(); print("Actif = $actif\n"); print("Passif = $passif\n");
$csv->set_walker_hr($subref);
Normally one wants to define it at object creation time using "walker_hr" attribute. set_walker_hr allows to assign the attribute "walker_hr" after object creation.
set_walker_hr
See attribute "walker_hr" for help about the way $subref should work.
my $csv = Text::AutoCSV->new(in_file => 'directory.csv', verbose => 1); # ... $csv->set_walker_hr( sub { my ($hr, $stat) = @_; $stat{'not capital name'}++, return if $hr->{'NAME'} ne uc($hr->{'NAME'}); $stat{'name is capital letters'}++; } )->read();
$csv->set_out_file($out_file);
Normally one wants to define it at object creation time using "out_file" attribute. set_out_file allows to assign the attribute "out_file" after object creation. It is set to $out_file value.
set_out_file
$out_file
$csv->set_out_file('mycopy.csv')->write();
my @allkeys = $csv->get_keys();
Returns an array of all the record keys of input. A record key is a unique identifier that designates the record.
At the moment it is just an integer being the record number, the first one (that comes after the header line) being of number 0. For example if $csv input is made of one header line and 3 records (that is, a 4-line file typically, if no record contains a line break), $csv->get_keys() returns
(0, 1, 2)
If not yet done, this function causes the input to be read entirely and stored in-memory.
my @allin = $csv->get_hr_all();
Returns an array of all record contents of the input, each record being a hash ref.
my $row_ar = $csv->get_row_ar($record_key);
Returns an array ref of the record designated by $record_key.
$record_key
# Get content (as array ref) of last record my @allkeys = $csv->get_keys(); my $lastk = $allkeys[-1]; my $lastrec_ar = $csv->get_row_ar($lastk);
my $row_hr = $csv->get_row_hr($record_key);
Returns a hash ref of the record designated by $record_key.
# Get content (as hash ref) of first record my @allkeys = $csv->get_keys(); my $firstk = $allkeys[0]; my $firstrec_hr = $csv->get_row_hr($firstk);
my $val = $csv->get_cell($record_key, $field_name);
Return the value of the cell designated by its record key ($record_key) and field name ($field_name).
$field_name
my @allkeys = $csv->get_keys(); my $midk = $allkeys[int($#allkeys / 2)]; my $midname = $csv->get_cell($midk, 'NAME');
Note the above example is equivalent to:
my @allkeys = $csv->get_keys(); my $midk = $allkeys[int($#allkeys / 2)]; my $midrec_hr = $csv->get_row_hr($midk); my $midname = $midrec_hr->{'NAME'};
my @vals = $csv->get_values($field_name, $opt_filter_subref);
Return an array made of the values of the given field name ($field_name), for every records, in the order of the records.
$opt_filter_subref is an optional subref. If defined, it is called with every values in turn (one call per value) and only values for which $opt_filter_subref returned True are included in the returned array. Call to $opt_filter_subref is done with $_ to pass the value.
$opt_filter_subref
my @logins = $csv->get_values('LOGIN");
This is equivalent to:
my @allkeys = $csv->get_keys(); my @logins; push @logins, $csv->get_cell($_, 'LOGIN') for (@allkeys);
Example bis
# @badlogins is the list of logins that contain non alphanumeric characters my @badlogins = Text::AutoCSV->new(in_file => 'logins.csv') ->get_values('LOGIN', sub { m/[^a-z0-9]/ });
# @badlogins is the list of logins that contain non alphanumeric characters # This method leads to carrying all values of a given field across function calls... my @badlogins = grep { m/[^a-z0-9]/ } ( Text::AutoCSV->new(in_file => 'logins.csv')->get_values('LOGIN') );
my $r = $csv->get_recnum();
Returns the current record identifier, if a reading is in progress. If no read is in progress, return undef.
read_update_after is an alias of in_map.
read_update_after
in_map
$csv->in_map($field, $subref);
After reading a record from input, update $field by calling $subref. The value is put in $_. Then the field value is set to the return value of $subref.
$_
This feature is originally meant to manage DateTime fields: the input and output CSVs carry text content, and in-between, the values dealt with are DateTime objects.
See "out_map" for an example.
write_update_before is an alias of out_map.
write_update_before
out_map
$csv->out_map($field, $subref);
Before writing $field field content into the output file, pass it through out_map. The value is put in $_. Then the return value of $subref is written in the output.
Suppose you have a CSV file with the convention that a number surrounded by parenthesis is negative. You can register corresponding "in_map" and "out_map" functions. During the processing of data, the field content will be just a number (positive or negative), while in input and in output, it'll follow the "negative under parenthesis" convention.
In the below example, we rely on convention above and add a new field converted from the original one, that follows the same convention.
sub in_updt { return 0 if !defined($_) or $_ eq ''; my $i; return -$i if ($i) = $_ =~ m/^\((.*)\)$/; $_; } sub out_updt { return '' unless defined($_); return '(' . (-$_) . ')' if $_ < 0; $_; } sub convert { return ; } Text::AutoCSV->new(in_file => 'trans-euros.csv', out_file => 'trans-devises.csv') ->in_map('EUROS', \&in_updt) ->out_map('EUROS', \&out_updt) ->out_map('DEVISE', \&out_updt) ->field_add_copy('DEVISE', 'EUROS', sub { sprintf("%.2f", $_ * 1.141593); } ) ->write();
my $found_ar = $csv->search($field_name, $value, \%opts);
Returns an array ref of all records keys where the field $field_name has the value $value.
$value
\%opts is an optional hash ref of options for the search. See help of "vlookup" about options.
An unsuccessful search returns an empty array ref, that is, [ ]. Thus you cannot check for definedness of search return value to know whether or not the search found something.
search
On the other hand, you can always examine the value search(...)->[0], as search is always an array ref. If the search found nothing, then, search(...)->[0] is not defined.
search(...)->[0]
IMPORTANT bis
my $linux_os_keys_ar = $csv->search('OS', 'linux');
my $found_hr = $csv->search_1hr($field_name, $value, \%opts);
Returns a hash ref of the first record where the field $field_name has the value $value.
Note the options "value_if_not_found" and "value_if_ambiguous" are ignored. If not found, return undef. If the result is ambiguous (more than one record found) and ignore_ambiguous is set to a false value, return undef.
The other options are taken into account as for any search: "ignore_ambiguous", "trim", "case", "ignore_empty".
As opposed to "search", an unsuccessful search_1hr will return undef.
search_1hr
my $hr = $csv->search_1hr('LOGIN', $login); my $full_name = $hr->{'FIRSTNAME'} . ' ' . $hr->{'LASTNAME'};
my $val = $csv->vlookup($searched_field, $value, $target_field, \%opts);
Find the first record where $searched_field contains $value and out of this record, returns the value of $target_field.
$searched_field
$target_field
\%opts is optional. It is a hash of options for vlookup:
vlookup
If true, ignore spaces before and after the values to search.
If option is not present, use "search_trim" attribute of object (default value: 1).
If true, do case sensitive searches.
If true, ignore empty values in the search. The consequence is that you won't be able to find empty values by searching it.
If true, ignore accents in searches. For exampe, if ignore_accents is set, a string like "élémentaire" will match "elementaire".
ignore_accents
If option is not present, use "search_ignore_accents" attribute of object (default value: 1).
This option uses the function "remove_accents" to build its internal hash tables. See "remove_accents" help for more details.
Return value if vlookup finds nothing.
Return value if vlookup finds something.
This option is to just check whether a value exists, regardless of the target value found.
Although the $target_field is ignored when using this option, you must specify it any way.
Return value if vlookup find more than one result. Tune it only if ignore_ambiguous is unset.
If true, then if more than one result is found, silently return the first one.
my $name = $csv->vlookup('LOGIN', $id, 'NAME', { value_if_not_found => '<login not found>' });
my $t = $csv->remove_accents($s);
Take the string $s as argument and return the string without accents. Uses a Unicode decomposition followed by removal of every characters that have the Unicode property Nonspacing_Mark.
$s
Nonspacing_Mark
Only accents are removed. It is not a whatever-encoding -> us-ascii conversion. For example, the French œ character (o followed by e) or the German ß (eszett) are kept as is.
whatever-encoding -> us-ascii
Tested with some latin1 and latin2 characters.
NOTE ter
There is no language-level transformation during accents removal. For example Jürgen is returned as Jurgen, not Juergen.
This function is not exported by default.
use Text::AutoCSV qw(remove_accents); my $s = remove_accents("Français: être élémentaire, Tchèque: služba dům"); die "This script will never die" if $s ne 'Francais: etre elementaire, Tcheque: sluzba dum';
Sébastien Millet <milletseb@laposte.net>
This software is copyright (c) 2016, 2017 by Sébastien Millet.
This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.
To install Text::AutoCSV, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Text::AutoCSV
CPAN shell
perl -MCPAN -e shell install Text::AutoCSV
For more information on module installation, please visit the detailed CPAN module installation guide.