BTRIEVE::SAVE - Perl extension to manipulate BTRIEVE SAVE records.
use BTRIEVE::SAVE my $btr = BTRIEVE::SAVE->new('cc057.std','cc057.dar'); $btr->parse_file(); my $recs = $btr->{'array'}; for (@$recs) { my($rhfixed,$rfixed,$rvar)=@{$_->{values}}; print $rhfixed->{'Title'}."\n"; } # Often the first record is some kind of header. Generally one # treats header records differently from those that follow. E.g., # they often have counts of the following records that must be # adjusted if we are gonna kill or add records. Here we leave it # alone. my $output = ""; $header=shift @$recs; my $data = $header->fixed.$header->var; $output .= $header->counted_rec($data); foreach my $rec (@$recs) { my($rhfixed,$rfixed,$rvar)=@{$rec->{values}}; $rhfixed->{'Title'} =~s/^\s*the/The/; $output .=$rec->counted_rec_hash(); } $output .="\cZ"; # now $output is a legal Btrieve save record. # For large records, one may want to do everything incrementally. open OUT,">>cc057.das" or die "Could not open cc057.das for append: $!\n"; binmode OUT; my $incbtr = BTRIEVE->new('cc057.std','cc057.dar'); my $header = $incbtr->next_rec(); my $data = $header->fixed.$header->var; print OUT $header->counted_rec($data); while (defined( my $rec=$incbtr->next_rec) ) { my($rhfixed,$rfixed,$rvar)=@{$rec->{values}}; $rhfixed->{'Title'} =~s/^\s*the/The/; print OUT $rec->counted_rec_hash(); } print OUT "\cZ"; close OUT or die "Could not close cc057.das: $!\n";
BTRIEVE::SAVE is a Perl 5 module for reading in, manipulating, and outputting Pervasive's save file format for its Btrieve products.
BTRIEVE::SAVE uses BTRIEVE::SAVE::REC which abstracts an individual record in the entire file.
You must have a config file for your save file: this allows BTRIEVE::SAVE::REC to analyse the fixed parts and find the variable parts of each BTRIEVE::SAVE record.
Download the latest version from http://www.cpan.org/modules/by-module/BTRIEVE. The module is provided in standard CPAN distribution format. It will extract into a directory BTRIEVE-version with any necessary subdirectories. Change into the BTRIEVE top directory.
perl Makefile.PL make make test make install
perl Makefile.PL perl test.pl perl install.pl
Once you have installed BTRIEVE::SAVE, you can check if Perl can find it. Change to some other directory and execute from the command line:
perl -e "use BTRIEVE::SAVE"
If you do not get any response that means everything is OK! If you get an error like Can't locate method "use" via package BTRIEVE::SAVE then Perl is not able to find BTRIEVE/SAVE.pm--double check that the file copied it into the right place during the install.
Support for the other 12 documented btrieve data types.
Help for adding the names column in the config file.
More detailed warnings/sanity checks on the config file and save file.
Please let us know if you run into any difficulties using BTRIEVE::SAVE--we'd be happy to try to help. Also, please contact us if you notice any bugs, or if you would like to suggest an improvement/enhancement. Email addresses are listed at the bottom of this page. Lane is probably the most interested in making this work, so may be your best initial bet.
A save file record on disk looks like:
__________________________________________ | Fixed, | Fixed, | | Count[, ] | indexed. | left-over| Variable |\r\n |___________________|__________|___________|
The Count is the number of "boxed bytes" (excluding the count itself, the following comma or space, and the final two bytes at the end). Count is an unpadded ascii integer, like "524".
Count is followed by either a comma or a space. BTRIEVE::SAVE::REC writes a comma, but reads either comma or space.
The boxes are binary data. BTRIEVE::SAVE::REC will not interpret the (fixed, left-over) or variable boxes, and will parse the (fixed, indexed) data into a hash based on a template and names determined from the config file.
The last two boxes may be empty; if the sum of the lengths is equal to the fixed_length defined in the config file the (fixed, left-over) box will have no bytes. Variable will be non-empty if and only if Count > fixed_length.
The number of bytes covered by the first two fields is equal to fixed_length. (In theory the (fixed, indexed) and (fixed, left-over) could be intermixed, with possible overlaps. Has anyone seen this? Docs saying that Pervasive won't support this?)
The final two bytes are unix return and newline bytes.
A save file is a bunch of save file records concatenated with no intervening bytes followed by "\cZ".
Each BTRIEVE::SAVE::REC has admin information in its {opt} key and data in its {values} key.
{opt} has keys: {fixed_defs} keys a ref to an array of hashes with keys: {len} - length in bytes of the field {name} - the user-defined name of that field, found from an extra column in the config file. The config file cannot use "ZZ" as a field name; one of the {name}s is always set to "ZZ" to handle fixed length information which is unaccounted for by btrieve's keys. {type} - Btrieve's idea of the type of that field. The array of hashes is in the order that the defining lines occur in the config file. {template} - pack template used for extracting values {names} - ref to array of names as in fixed_defs. This is not strictly necessary. {len} - total length of the fixed part of the record. This must be at least equal to the sum of the {len}'s in the {fixed_defs}. {values} is a ref to an array [$rhfixed,$rfixed,$rvar] where $rhfixed is a ref to a hash with keys from the {opt}{names} and values the unpacked version of the btrieve-defined fixed portion of the record. "ZZ" will always be the last element in {names}; $rhfixed->{'ZZ'} will include all the bytes of the btrieve fixed part of the record that are not accounted for by the config file. (This means it can be empty.) $rfixed is a reference to the btrieve-defined fixed portion of the record and $rvar is a reference to the btrieve-defined variable portion of the record. Parsing and definitions of the {fixed_defs} appears to assume that all keys are consecutive and that any fixed-length information not accounted for by key defs occurs at the end. If you come across non-consecutive keys in a -stat file, make up extra key fields to cover the missing parts. This will make BTRIEVE::SAVE::REC happy. If folk actually see these kinds of keys, please alert us. We can document this as a limitation. (We also take patches....). We are also interested if folk find examples of overlapping key defs.
Each BTRIEVE::SAVE record has keys:
{opt} with keys: {file} - a file name that BTRIEVE::SAVE will read from. {handle} - a stored filehandle that BTRIEVE::SAVE will use, based on the {file} {increment} - how many records to attempt to read at a time in parse_file(). {proto_rec} - A BTRIEVE::SAVE::REC with empty {values}, used for filling in elements of BTRIEVE::SAVE->{array}. {config} - The config file used to define the {proto_rec} and all elements of the {array} {array} which is a ref to an array (0-based) of BTRIEVE::SAVE::RECs with structure determined by {opt}{proto_rec}.
Here is a list of the methods in BTRIEVE::SAVE and ::REC that are available to you for reading in, manipulating and outputting BTRIEVE::SAVE data.
Creates a new BTRIEVE::SAVE object; also used for ::REC. $x = BTRIEVE::SAVE->new('cc057.std'); $x = BTRIEVE::SAVE->new('cc057.std','cc057.dat'); $rec = BTRIEVE::SAVE::REC->new($ranames,$rtemplate, $packed_length,$rhfixed_defs); $rec = BTRIEVE::SAVE::REC->newconfig('cc057.std');
BTRIEVE::SAVE has an optional config file first parameter and an optional file parameter to create and populate the object with data from a file. If a file is specified it will read in the entire file. If you wish to read in only portions of the file see openbtr(), nextbtr(), and closebtr() below or use next_rec() and BTRIEVE::SAVE::RECs directly.
BTRIEVE::SAVE::REC allows creation with defined state with new() and also from a config file with newconfig(). See the last EXAMPLE below for a definition of the format of the config file.
Installs a prototypical BTRIEVE::SAVE::REC in the BTRIEVE::SAVE object based on the contents of the config file.
$x= BTRIEVE::SAVE->new(); $x->config('cc057.std');
Openbtr sets up incremental reading for a BTRIEVE SAVE file. It takes a hashref with key 'file' (name of the btrieve file). Increment defines how many records to read in and is taken from the object or defaulted to 1.
$x = BTRIEVE::SAVE->new('cc057.std'); $x->openbtr({file=>"cc057.dat",increment=>"2"}); $x->openbtr({file=>"cc057.dat"});
Once a file is open nextbtr() can be used to read in the next group of records. The increment can be passed to change the amount of records read in if necessary. An increment of -1 will read in the rest of the file.
$x->nextbtr(); $x->nextbtr(10); $x->nextbtr(-1);
nextbtr() will return the amount of records read in.
$y=$x->nextbtr(); print "$y more records read in!";
If you are finished reading in records from a file you should close it immediately.
$x->closebtr();
Parse_file reads in a file, possibly incrementally. It APPENDS the record to the BTRIEVE::SAVE {array} field. It returns the number of records read.
$y=$x->parse_file(); print "$y records read!\n";
Next_rec returns a new BTRIEVE::SAVE::REC record with the same structure as {proto_rec} and {values} based on the bits in the next record.
my $rec=$x->next_rec;
Next_recbits returns the data in the next record on-disk. It side-effects the position of the file pointer implied by the handle.
my $string_rec = $x->next_recbits
Output dumps all records in {array} to a file (if passed one) in Btrieve's save file format. It returns a string version of this if there is no file passed.
$x->output(">cc057.das"); my $btr_save_string = $x->output();
As_string returns a string version in Btrieve's save file format of {array}.
my $btr_save_string = $x->as_string();
Save_to_rdb takes an rdb filename, save filename, error file name, and config file name. Also takes the field name for unindexed fixed info and var info, and strings to translate into from tab and newline characters. Writes an rdb file using the save file and config information; warns and writes to the rdb formatted error file if there are problems in the data.
$btr->save_to_rdb('f.rdb','f.sav','ferr.sav', 'ZZ','VAR','<TAB>','<RET>');
Rdb_to_save takes an rdb filename, save filename, error file name, and config file name. Also takes the field name for unindexed fixed info and var info, and strings to translate into tab and newline characters. Writes an rdb file using the rdb file and config information; warns and writes to the save formatted error file if there are problems in the data.
$btr->rdb_to_save('f.rdb','f.sav','ferr.rdb', 'ZZ','VAR','<TAB>','<RET>');
Copy_struct takes a record and returns a new BTRIEVE::SAVE::REC with empty {values} and the same {opt}s.
my $new_rec=$rec->copy_struct();
Parse_string takes a string representation of a btrieve record (typically from next_recbits) and returns a new record with the same {opt}s and appropriate {values}. Parse_string makes sure that all optional fields (ZZ and the var field) are initialised to empty strings if undefined.
my $curr_rec=$rec->parse-string($string_rec);
Counted_rec takes a string version of a record and returns a string that can represent this on-disk in Btrieve's save file format.
my $save_rec = $rec->counted_rec($string_rec);
Counted_rec_hash will take the hash and variable information in {values} and return a legal string in Btrieve's save file format.
my $save_rec = $rec->counted_rec_hash();
Fixed will return a string representation of the fixed string information in {values}.
my $fix_string = $rec->fixed;
Var will return a string representation of the var string information in {values}.
my $var_string = $rec->var;
Data will return a string representation of a record using the hash and variable information in {values}.
my $data = $rec->data();
Fix_hash_to_string returns the fixed part of a record based on its hash {values}. If you want to get access to the raw fixed part, use fixed().
my $fixed = $rec->fix_hash_to_string();
Here are a few examples to fire your imagination.
Input. This example will read in a tab-delimited file and output a Btrieve save file. Last field is variable.
#!/usr/bin/perl use BTRIEVE::SAVE; # ::REC comes along for the ride. my $proto_rec = BTRIEVE::SAVE::REC->newconfig("config.std"); open F,"import.tab" or die "Could not open import.tab for read: $!\n"; open OUT,">output.dat" or die "Could not open output.dat for write: $!\n"; binmode OUT; while (<F>) { chomp; my @fields= split(/\t/); my $var = pop @fields; my $data = (join "",@fields).$var; print OUT $proto_rec->counted_rec($data); } print OUT "\cZ"; close F; close OUT;
Feeling paranoid? I know I am.
Let's say somebody comes to you with a large tab delimited file, a BTRIEVE config file (forced on her by external software), and a burning desire that her data become one with BTRIEVE.
And she added data to the file by hand over the last 3 years. And she is not quite sure that she got the right lengths for the indexes.
Fortunately her tab-delimited file is in /rdb format so at least the field names are available. She tells you that the last, very important, field is variable length. The order of names in the rdb file is different than that in the save file spec, and you won't write a filter to fix that (go figure).
/Rdb files look like:
Author Title Opinion_of_AACR2 ------ ------ ---------------- Fred Fred's holidays Better than chocolate James James' elbows Larger than several trucks See http://www2.linuxjournal.com/lj-issues/issue67/3294.html for details on /rdb which is a cool idea. $ grep '<VAR>\|<RET>' file.rdb |wc -l 0 $ cat rdb_btr.pl #!/usr/bin/perl print "Usage: rdb_btr.pl <rdbfile><conf><savefile><error rdbfile>" unless scalar @ARGV ==4; my ($rdb,$config,$save,$err) = @ARGV; my $btr = BTRIEVE::SAVE->new($config); $btr->rdb_to_save($rdb,$save,$err, 'ZZ','Opinion_of_AACR2','<TAB>','<RET>'); $ rdb_btr.pl file.rdb file.std file.sav file.err Paranoia 2: Lengths do not match for Title at line 6. Paranoia 1: Number of fields do not match rdb spec at line 300. $ wc -l file.err 4 $ etbl file.err (Time passes as you edit the relevant lines' problems.) $ rdb_btr.pl file.err file.std file2.sav file2.err $ wc -l file2.err 2 $ perl -ne 'print unless /^\cZ$/' file.sav | cat - file2.sav > clean.sav *Whew*
Someone hands you a BTRIEVE file. Having paid the tax to Pervasive you have a command line utility "butil" with many options, including -load, -save, -stat and -clone.
You want to get the file out into /rdb tab-delimited form so you only need -save and -stat.
$ butil -save file.btr file.dat $ butil -stat file.btr > file.std (Edit file.std to add an extra column thusly:) (Leave what's above here alone...) Record Length = 171 Record Length = <whatever you want the fixed length to be.> (...leave the next alone until the Key defs below. Some columns elided for space.) Key Pos.. Type Null V.. ACS Segment Len.. Flags Uniq.. 0 1 1 4 Zstring - .. 8 -- 1 1 5 4 Integer MD - .. 8 -- (..and what's after alone.) Add an extra column: Key Pos.. Type Null V.. ACS Segment Len.. Flags Uniq.. Langname 0 1 1 4 Zstring --.. 8 -- Author 1 1 5 4 Integer MD --.. 8 -- dbcn $ cat btr_rdb.pl #!/usr/bin/perl print "Usage: btr_rdb.pl <rdbfile><conf><savefile><error savefile>" unless scalar @ARGV ==4; my ($rdb,$config,$save,$err) = @ARGV; my $btr = BTRIEVE::SAVE->new($config); $btr->save_to_rdb($rdb,$save,$err, 'ZZ','VAR','<TAB>','<RET>'); $ rdb_btr.pl file.rdb file.std file.sav file.err $ wc file.err 0 1 1 file.err $
Chuck Bearden cbearden@rice.edu
Bill Birthisel wcbirthisel@alum.mit.edu
Derek Lane dereklane@pobox.com
Charles McFadden chuck@vims.edu
Ed Summers esummers@odu.edu
perl(1), www.pervasive.com, "Btrieve Complete" by Jim Kyle (Addison-Wesley 1995).
Copyright (C) 2000, Bearden, Birthisel, Lane, McFadden, Summers. All rights reserved. Copyright (C) 2000, Duke University, Lane. All rights reserved.
This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself. Feb 15 2000.
To install BTRIEVE::SAVE, copy and paste the appropriate command in to your terminal.
cpanm
cpanm BTRIEVE::SAVE
CPAN shell
perl -MCPAN -e shell install BTRIEVE::SAVE
For more information on module installation, please visit the detailed CPAN module installation guide.