NAME

dt - CLI data transformer

VERSION

This document describes version 0.007 of dt (from Perl distribution App-dt), released on 2020-01-08.

SYNOPSIS

 % dt [OPTIONS] [FILE]...

DESCRIPTION

Roughly similar to utilities like jq, jt, the App::PipeFilter suite, jsonsel, ddsel, yamlsel, this dt utility can be used to transform data structure on the CLI using Perl code. It accepts zero or more files of encoded data structures in several formats (JSON, YAML, Perl), lets you specify Perl code to generate or modify them, and spits them out again in specified output format. The data structures are put into $DATA[0], $DATA[1] and so on. The first data structure is also put in $DATA as well as $_.

Some examples:

• Generate some JSON

   % dt -e '[1..100]'
   % dt -e '[1..100]' -o numbers.json
   % dt -e '$n = 100; $DATA = [1..$n]' -o numbers.yaml

  Input files are specified as arguments. If you do not specify any argument, and dt is not at the right side of a pipeline, then you can generate data in the code. The last expression or $DATA or $_ will be used.

  One or more --output (-o) options can be specified to direct output to files. The first output file will be set to $DATA[0] (or $DATA or $_), the second to $DATA[1] and so on. If no output files are specified, will output to STDOUT.

• Convert JSON to YAML

   % dt books.json -o books.yaml

  Input and output format are guessed from filename extension. Or, you can also use :FORMAT notation for input, e.g.:

   % dt books.dat:json -o books.yml

  These formats are currently supported: json, yaml, perl.

• Convert several JSON files to YAML

   % dt 1.json 2.json 3.json -o 1.yml -o 2.yml -o 3.yml

• Convert YAML to Perl: get the YAML from STDIN and output to a file

   % other-cmd | dt -f yaml -F perl -o datafile

  If no input files are specified but dt is at the right side of a pipeline, data will be retrieved from STDIN.

  Also, when output filename does not give any hint about format, you can set output format using -F.

• Convert YAML to pretty-printed JSON (see pp-json)

   % dt books.yaml | pp-json

  The default output format is JSON.

• Get a branch of data structure

   % dt bookstore.json -e '$DATA->{store}{book}' -o books.json

• Add a field to records, output to STDOUT as JSON

   % dt -e 'my $now = time; for (@$DATA) { $_->{status} = $_->{mtime} >= $now-7*86400 ? "new" : "" } $DATA' < books.json

• Merge several YAML together as array of data structures

   % dt 1.yaml 2.yaml 3.yaml -e '[@DATA]' -o combined.yaml

• Modify several YAML files, save as JSON

   % dt 1.yaml 2.yaml 3.yaml -e 'for (@DATA) { $_->{status} = "Checked" }' -o 1.json -o 2.json -o 3.json

EXIT CODES

0 on success.

255 on I/O error.

99 on command-line options or input data error.

OPTIONS

• -e CODE

  Specify Perl code to run. These variables are set for the code: @DATA to the data structures (in the order of the specified files), also $DATA and $_ to the first data structure.

• --overwrite

  Overwrite existing output files instead of skipping writing to output files when they already exist.

• --default-input-format FORMAT, -f

  Set default output format. Default is json. Can also be yaml, perl, raw.

• --default-output-format FORMAT, -F

  Set default output format. Default is json. Can also be yaml, perl, raw.

• --output FILENAME, -o

  Add an output. Format will be guessed from .

  If not specified, will output all data to STDOUT.

ENVIRONMENT

DEBUG => bool

If set to true, print debugging messages.

HOMEPAGE

Please visit the project's homepage at https://metacpan.org/release/App-dt.

SOURCE

Source repository is at https://github.com/perlancar/perl-App-dt.

BUGS

Please report any bugs or feature requests on the bugtracker website https://rt.cpan.org/Public/Dist/Display.html?Name=App-dt

When submitting a bug or request, please include a test-file or a patch to an existing test-file that illustrates the bug or desired feature.

SEE ALSO

jq

jt, App::PipeFilter, jsonsel, ddsel, yamlsel

AUTHOR

perlancar <perlancar@cpan.org>

COPYRIGHT AND LICENSE

This software is copyright (c) 2020, 2019, 2017 by perlancar@cpan.org.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.

cpanm App::dt

perl -MCPAN -e shell
install App::dt