The csv_loader plugin provides functionality to load delimited files into a database table. It relies on a control file to define the fields in the incoming data file. The following parameters must be set in the config section:
table_name. The name table that the loader inserts the file data into.
controlfile_dir. The directory that contains the control file.
control_file. The name of the control file
The item parameters are:
name. The name of this item.
ignore_field_count. whether to continue if the number fields in the data (CSV) file don't match the number of fields in the control file.
skip. The number of rows to skip in the file before loading begins. This allows header records to be ignored.
localize. A boolean setting that instructs the loader to localize the end-of-line markers for the current file system. Defaults to 0.
csv_options. A section containing additional options for processing. See http://search.cpan.org/dist/Text-CSV/lib/Text/CSV.pm#new_%28\%attr%29.
on_error. Override the setting for the job.
The csv_loader plugin requires the table that stores the data have a column called file_id of type integer. This is used to link the file to its audit record.
<item> name = load customer file type = csv_loader # Skip one line skip = 1 <csv_options> allow_whitespace = 1 sep_char = | # use pipes not commas for delimiting data </csv_options> </item>