ETL::Yertl - ETL with a Shell
version 0.028
### On a shell... # Convert file to Yertl's format $ yfrom csv file.csv >work.yml $ yfrom json file.json >work.yml # Mask document $ ymask 'field/inner' work.yml >masked.yml # Convert file to output format $ yto csv work.yml $ yto json work.yml # Parse HTTP logs into documents $ ygrok '%{LOG.HTTP_COMMON}' httpd.log # Read data from a database $ ysql db_name 'SELECT * FROM employee' # Write data to a database $ ysql db_name 'INSERT INTO employee ( id, name ) VALUES ( $.id, $.name )' ### In Perl... use ETL::Yertl; # XXX: To do: Perl API
ETL::Yertl is an ETL (Extract, Transform, Load) for shells. It is designed to accept data from multiple formats (CSV, JSON), manipulate them using simple tools, and then convert them to an output format.
Yertl will have tools for:
The Yertl home page.
Convert incoming data (CSV, JSON) to Yertl documents.
Convert Yertl documents into another format (CSV, JSON).
Parse lines of text into Yertl documents.
Read/write documents from SQL databases.
Filter documents with a mask, letting only matching fields through.
A powerful mini-language for munging and filtering.
Here are some other tools that can be used with Yertl
A set of tools for manipulating JSON (constrast with Yertl's YAML). For interoperability, set the YERTL_FORMAT environment variable to "json".
YERTL_FORMAT
"json"
A filter for JSON documents. The inspiration for yq. For interoperability, set the YERTL_FORMAT environment variable to "json".
JSON Transformer. Allows multiple ways of manipulating JSON, including JSONPath. For interoperability, set the YERTL_FORMAT environment variable to "json".
Doug Bell <preaction@cpan.org>
Doug Bell <doug@preaction.me>
This software is copyright (c) 2016 by Doug Bell.
This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.
To install ETL::Yertl, copy and paste the appropriate command in to your terminal.
cpanm
cpanm ETL::Yertl
CPAN shell
perl -MCPAN -e shell install ETL::Yertl
For more information on module installation, please visit the detailed CPAN module installation guide.