NAME
MySQL::Slurp - Use PIPEs to import a file into MySQL table.
CAVEAT
MySQL::Slurp only works on systems that support FIFOs and
does not support Windows ... yet.SYNOPSIS
use MySQL::Slurp;
# Object method
my $slurper= MySQL::Slurp->new( database => 'test', table => 'table_1' );
$slurper->open;
$slurper->slurp(); # slurp from <STDIN>
print $slurper "Fred\tFlinstone\n"; # Print directly to table
print $slurper "Barney\tRubble\n";
$slurper->close;DESCRIPTION
The command-line tool, mysqlimport, is the fastest way to import data into MySQL especially using --use-threads. It is faster than c<LOAD DATA INFILE> especially when use the --use-threads option. Unfortunately, mysqlimport does not read from <STDIN>. IN fact, mysqlimport only reads from files that have the same name as the target table. This is often inconvenient.
MySQL::Slurp has the speed of mysqlimport but permits loading from <STDIN> or provides a GlobRef file handle for writing directly to a MySQL table. This is very handy for large ETL jobs.
This module simply wraps and mysqlimport and creates the necessary FIFO. As such, catching (data) errors is relegated to mysqlimport. Unike using DBI for trapping errors, catching errors with mysqlimport can be troublesome with inconsitent data. It is recommended that you check you data before writing to the MySQL::Slurp handle or use a suitable DBI method. inconsistent.
METHODS
new
Creates a new MySQL::Slurp object.
- database
-
name of database (required)
- table
-
Name of table to import (required)
- tmp
-
Name of temporary directory (optional)
- args
-
Options to pass to mysqlimport. args is an array ref and should appear exactly as it does in the command line invocation of mysqlimport
open
Opens a connection to the MySQL table through a temporary FIFO. Returns a GlobRef that can be directly written to.
close
Closes and removes the pipe and temporary table.
slurp
Write <STDIN> to the database table.
EXPORT
None.
TODO
- use MooseX::Attribute::Defaults::GNU for object attributes
- remove reliance on installation of mysqlimport, by XS wrapping the C libraries.
- Better error catching than mysqlimport
- create a version to run on windows with named pipes(?)
- alias attribute tmp as temp.
- allow options for slurp to change the MySQL::Slurp object's attributes
SEE ALSO
MySQL::Slurp relies on the Moose metaobject package.
mysqlimport at http://mysql.com, currently http://dev.mysql.com/doc/refman/5.1/en/mysqlimport.html
AUTHOR
Christopher Brown, <ctbrown@cpan.org<gt>
COPYRIGHT AND LICENSE
Copyright (C) 2008 by Open Data
This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.8.8 or, at your option, any later version of Perl 5 you may have available.