Derek Lane

NAME

    BTRIEVE::SAVE - Perl extension to manipulate BTRIEVE SAVE records.

SYNOPSIS

    use BTRIEVE::SAVE
    my $btr = BTRIEVE::SAVE->new('cc057.std','cc057.dar');
    $btr->parse_file();
    my $recs = $btr->{'array'};
    for (@$recs) {
        my($rhfixed,$rfixed,$rvar)=@{$_->{values}};
        print $rhfixed->{'Title'}."\n";
    }
    

    
    # Often the first record is some kind of header.  Generally one
    # treats header records differently from those that follow. E.g.,
    # they often have counts of the following records that must be
    # adjusted if we are gonna kill or add records.  Here we leave it
    # alone.

    my $output = "";
    $header=shift @$recs;

    my $data = $header->fixed.$header->var;
    $output .= $header->counted_rec($data);
    
    foreach my $rec (@$recs) {
        my($rhfixed,$rfixed,$rvar)=@{$rec->{values}};
        $rhfixed->{'Title'} =~s/^\s*the/The/;
        $output .=$rec->counted_rec_hash();
    }
    $output .="\cZ"; # now $output is a legal Btrieve save record.
    # For large records, one may want to do everything incrementally.
    
    open OUT,">>cc057.das"
              or die "Could not open cc057.das for append: $!\n";
    binmode OUT;
    my $incbtr = BTRIEVE->new('cc057.std','cc057.dar');
    my $header = $incbtr->next_rec();
    my $data = $header->fixed.$header->var;
    print OUT $header->counted_rec($data);
    
    while (defined( my $rec=$incbtr->next_rec) ) {
        my($rhfixed,$rfixed,$rvar)=@{$rec->{values}};
        $rhfixed->{'Title'} =~s/^\s*the/The/;
        print OUT $rec->counted_rec_hash();
    }
    print OUT "\cZ";
    close OUT or die "Could not close cc057.das: $!\n";

DESCRIPTION

BTRIEVE::SAVE is a Perl 5 module for reading in, manipulating, and outputting Pervasive's save file format for its Btrieve products.

BTRIEVE::SAVE uses BTRIEVE::SAVE::REC which abstracts an individual record in the entire file.

You must have a config file for your save file: this allows BTRIEVE::SAVE::REC to analyse the fixed parts and find the variable parts of each BTRIEVE::SAVE record.

Downloading and Installing

Download

Download the latest version from http://www.cpan.org/modules/by-module/BTRIEVE. The module is provided in standard CPAN distribution format. It will extract into a directory BTRIEVE-version with any necessary subdirectories. Change into the BTRIEVE top directory.

Unix Install
    perl Makefile.PL
    make
    make test
    make install
Win9x/WinNT/Win2000 Install
    perl Makefile.PL
    perl test.pl
    perl install.pl
Test

Once you have installed BTRIEVE::SAVE, you can check if Perl can find it. Change to some other directory and execute from the command line:

    perl -e "use BTRIEVE::SAVE"

If you do not get any response that means everything is OK! If you get an error like Can't locate method "use" via package BTRIEVE::SAVE then Perl is not able to find BTRIEVE/SAVE.pm--double check that the file copied it into the right place during the install.

Todo

  • Support for the other 12 documented btrieve data types.

  • Help for adding the names column in the config file.

  • More detailed warnings/sanity checks on the config file and save file.

Notes

Please let us know if you run into any difficulties using BTRIEVE::SAVE--we'd be happy to try to help. Also, please contact us if you notice any bugs, or if you would like to suggest an improvement/enhancement. Email addresses are listed at the bottom of this page. Lane is probably the most interested in making this work, so may be your best initial bet.

BTRIEVE::SAVE::REC structure.

A save file record on disk looks like:

          __________________________________________
          |   Fixed,          | Fixed,   |           |
Count[, ] |   indexed.        | left-over|  Variable |\r\n
          |___________________|__________|___________|

The Count is the number of "boxed bytes" (excluding the count itself, the following comma or space, and the final two bytes at the end). Count is an unpadded ascii integer, like "524".

Count is followed by either a comma or a space. BTRIEVE::SAVE::REC writes a comma, but reads either comma or space.

The boxes are binary data. BTRIEVE::SAVE::REC will not interpret the (fixed, left-over) or variable boxes, and will parse the (fixed, indexed) data into a hash based on a template and names determined from the config file.

The last two boxes may be empty; if the sum of the lengths is equal to the fixed_length defined in the config file the (fixed, left-over) box will have no bytes. Variable will be non-empty if and only if Count > fixed_length.

The number of bytes covered by the first two fields is equal to fixed_length. (In theory the (fixed, indexed) and (fixed, left-over) could be intermixed, with possible overlaps. Has anyone seen this? Docs saying that Pervasive won't support this?)

The final two bytes are unix return and newline bytes.

A save file is a bunch of save file records concatenated with no intervening bytes followed by "\cZ".

Each BTRIEVE::SAVE::REC has admin information in its {opt} key and data in its {values} key.

  {opt} has keys:
    {fixed_defs} keys a ref to an array of hashes with keys:
      {len}      - length in bytes of the field
      {name}     - the user-defined name of that field, found from an
                   extra column in the config file. The config file
                   cannot use "ZZ" as a field name; one of the {name}s
                   is always set to "ZZ" to handle fixed length
                   information which is unaccounted for by btrieve's
                   keys.
      {type}     - Btrieve's idea of the type of that field.

     The array of hashes is in the order that the defining lines occur
     in the config file.

    {template} - pack template used for extracting values
    {names}    - ref to array of names as in fixed_defs. 
                 This is not strictly necessary.
    
    {len}      - total length of the fixed part of the record.  This
                 must be at least equal to the sum of the {len}'s in
                 the {fixed_defs}.

  {values} is a ref to an array
      [$rhfixed,$rfixed,$rvar]
  
      where $rhfixed is a ref to a hash with keys from the {opt}{names}
      and values the unpacked version of the btrieve-defined
      fixed portion of the record.
  
      "ZZ" will always be the last element in {names}; 

      $rhfixed->{'ZZ'} will include all the bytes of the btrieve fixed
      part of the record that are not accounted for by the config
      file. (This means it can be empty.)  $rfixed is a reference to
      the btrieve-defined fixed portion of the record and $rvar is a
      reference to the btrieve-defined variable portion of the record.

    Parsing and definitions of the {fixed_defs} appears to assume that
    all keys are consecutive and that any fixed-length information not
    accounted for by key defs occurs at the end. 

    If you come across non-consecutive keys in a -stat file, make up
    extra key fields to cover the missing parts. This will make
    BTRIEVE::SAVE::REC happy.

    If folk actually see these kinds of keys, please alert us.  We can
    document this as a limitation. (We also take patches....). We are
    also interested if folk find examples of overlapping key defs.

BTRIEVE::SAVE structure.

Each BTRIEVE::SAVE record has keys:

  {opt} with keys:
       {file}      - a file name that BTRIEVE::SAVE will read from.
       {handle}    - a stored filehandle that BTRIEVE::SAVE will use, 
                     based on the {file}
       {increment} - how many records to attempt to read at a time 
                     in parse_file().
       {proto_rec} - A BTRIEVE::SAVE::REC with empty {values}, used 
                     for filling in elements of BTRIEVE::SAVE->{array}.
       {config}    - The config file used to define the {proto_rec}
                     and all elements of the {array}

   {array} which is a ref to an array (0-based) of BTRIEVE::SAVE::RECs
           with structure determined by {opt}{proto_rec}.

METHODS

Here is a list of the methods in BTRIEVE::SAVE and ::REC that are available to you for reading in, manipulating and outputting BTRIEVE::SAVE data.

new(),newconfig()

    Creates a new BTRIEVE::SAVE object; also used for ::REC.

    $x = BTRIEVE::SAVE->new('cc057.std');
    $x = BTRIEVE::SAVE->new('cc057.std','cc057.dat');

    $rec = BTRIEVE::SAVE::REC->new($ranames,$rtemplate,
                                   $packed_length,$rhfixed_defs);
    $rec = BTRIEVE::SAVE::REC->newconfig('cc057.std');

BTRIEVE::SAVE has an optional config file first parameter and an optional file parameter to create and populate the object with data from a file. If a file is specified it will read in the entire file. If you wish to read in only portions of the file see openbtr(), nextbtr(), and closebtr() below or use next_rec() and BTRIEVE::SAVE::RECs directly.

BTRIEVE::SAVE::REC allows creation with defined state with new() and also from a config file with newconfig(). See the last EXAMPLE below for a definition of the format of the config file.

config()

Installs a prototypical BTRIEVE::SAVE::REC in the BTRIEVE::SAVE object based on the contents of the config file.

     $x= BTRIEVE::SAVE->new();
     $x->config('cc057.std');

openbtr()

Openbtr sets up incremental reading for a BTRIEVE SAVE file. It takes a hashref with key 'file' (name of the btrieve file). Increment defines how many records to read in and is taken from the object or defaulted to 1.

    $x = BTRIEVE::SAVE->new('cc057.std');
    $x->openbtr({file=>"cc057.dat",increment=>"2"});
    $x->openbtr({file=>"cc057.dat"});

nextbtr()

Once a file is open nextbtr() can be used to read in the next group of records. The increment can be passed to change the amount of records read in if necessary. An increment of -1 will read in the rest of the file.

    $x->nextbtr();
    $x->nextbtr(10);
    $x->nextbtr(-1);

nextbtr() will return the amount of records read in.

    $y=$x->nextbtr();
    print "$y more records read in!";

closebtr()

If you are finished reading in records from a file you should close it immediately.

    $x->closebtr();

parse_file()

Parse_file reads in a file, possibly incrementally. It APPENDS the record to the BTRIEVE::SAVE {array} field. It returns the number of records read.

    $y=$x->parse_file();
    print "$y records read!\n";

next_rec()

Next_rec returns a new BTRIEVE::SAVE::REC record with the same structure as {proto_rec} and {values} based on the bits in the next record.

   my $rec=$x->next_rec;

next_recbits()

Next_recbits returns the data in the next record on-disk. It side-effects the position of the file pointer implied by the handle.

   my $string_rec = $x->next_recbits

output()

Output dumps all records in {array} to a file (if passed one) in Btrieve's save file format. It returns a string version of this if there is no file passed.

   $x->output(">cc057.das");
   my $btr_save_string = $x->output();

as_string()

As_string returns a string version in Btrieve's save file format of {array}.

   my $btr_save_string = $x->as_string();         

save_to_rdb()

Save_to_rdb takes an rdb filename, save filename, error file name, and config file name. Also takes the field name for unindexed fixed info and var info, and strings to translate into from tab and newline characters. Writes an rdb file using the save file and config information; warns and writes to the rdb formatted error file if there are problems in the data.

    $btr->save_to_rdb('f.rdb','f.sav','ferr.sav',
                           'ZZ','VAR','<TAB>','<RET>');

rdb_to_save()

Rdb_to_save takes an rdb filename, save filename, error file name, and config file name. Also takes the field name for unindexed fixed info and var info, and strings to translate into tab and newline characters. Writes an rdb file using the rdb file and config information; warns and writes to the save formatted error file if there are problems in the data.

    $btr->rdb_to_save('f.rdb','f.sav','ferr.rdb',
                           'ZZ','VAR','<TAB>','<RET>');

BTRIEVE::SAVE::REC METHODS

copy_struct()

Copy_struct takes a record and returns a new BTRIEVE::SAVE::REC with empty {values} and the same {opt}s.

    my $new_rec=$rec->copy_struct();

parse_string()

Parse_string takes a string representation of a btrieve record (typically from next_recbits) and returns a new record with the same {opt}s and appropriate {values}. Parse_string makes sure that all optional fields (ZZ and the var field) are initialised to empty strings if undefined.

    my $curr_rec=$rec->parse-string($string_rec);

counted_rec()

Counted_rec takes a string version of a record and returns a string that can represent this on-disk in Btrieve's save file format.

    my $save_rec = $rec->counted_rec($string_rec);

counted_rec_hash()

Counted_rec_hash will take the hash and variable information in {values} and return a legal string in Btrieve's save file format.

    my $save_rec = $rec->counted_rec_hash();

fixed()

Fixed will return a string representation of the fixed string information in {values}.

   my $fix_string = $rec->fixed;

var()

Var will return a string representation of the var string information in {values}.

   my $var_string = $rec->var;

data()

Data will return a string representation of a record using the hash and variable information in {values}.

   my $data = $rec->data();

fix_hash_to_string()

Fix_hash_to_string returns the fixed part of a record based on its hash {values}. If you want to get access to the raw fixed part, use fixed().

   my $fixed = $rec->fix_hash_to_string();

EXAMPLES

Here are a few examples to fire your imagination.

  • Input. This example will read in a tab-delimited file and output a Btrieve save file. Last field is variable.

        #!/usr/bin/perl
        use BTRIEVE::SAVE; # ::REC comes along for the ride.
        my $proto_rec = BTRIEVE::SAVE::REC->newconfig("config.std"); 
    
        open F,"import.tab"    
                   or die "Could not open import.tab for read: $!\n";
        open OUT,">output.dat" 
                   or die "Could not open output.dat for write: $!\n";
        binmode OUT;
        while (<F>) {
              chomp;
              my @fields= split(/\t/);
              my $var = pop @fields;
              my $data = (join "",@fields).$var;
              print OUT $proto_rec->counted_rec($data);
         }
         print OUT "\cZ";
    
         close F;
         close OUT;
  • Feeling paranoid? I know I am.

    Let's say somebody comes to you with a large tab delimited file, a BTRIEVE config file (forced on her by external software), and a burning desire that her data become one with BTRIEVE.

    And she added data to the file by hand over the last 3 years. And she is not quite sure that she got the right lengths for the indexes.

    Fortunately her tab-delimited file is in /rdb format so at least the field names are available. She tells you that the last, very important, field is variable length. The order of names in the rdb file is different than that in the save file spec, and you won't write a filter to fix that (go figure).

    /Rdb files look like:

         Author        Title                Opinion_of_AACR2
         ------        ------               ----------------
         Fred          Fred's holidays      Better than chocolate
         James         James' elbows        Larger than several trucks
    
      See 
      http://www2.linuxjournal.com/lj-issues/issue67/3294.html 
      for details on /rdb which is a cool idea.
    
        $ grep '<VAR>\|<RET>' file.rdb |wc -l 
          0
        $ cat rdb_btr.pl
        #!/usr/bin/perl
        print "Usage: rdb_btr.pl <rdbfile><conf><savefile><error rdbfile>" 
                unless scalar @ARGV ==4;
        my ($rdb,$config,$save,$err) = @ARGV;
        my $btr = BTRIEVE::SAVE->new($config);
        $btr->rdb_to_save($rdb,$save,$err,
                               'ZZ','Opinion_of_AACR2','<TAB>','<RET>');
        
        $ rdb_btr.pl file.rdb file.std file.sav file.err
          Paranoia 2: Lengths do not match for Title at line 6.
          Paranoia 1: Number of fields do not match rdb spec at line 300.
        $ wc -l file.err
          4
        $ etbl file.err
        (Time passes as you edit the relevant lines' problems.)
        $ rdb_btr.pl file.err file.std file2.sav file2.err
        $ wc -l file2.err
          2
        $ perl -ne 'print unless /^\cZ$/' file.sav | cat - file2.sav > clean.sav
    
        *Whew*
  • Someone hands you a BTRIEVE file. Having paid the tax to Pervasive you have a command line utility "butil" with many options, including -load, -save, -stat and -clone.

    You want to get the file out into /rdb tab-delimited form so you only need -save and -stat.

        $ butil -save file.btr file.dat    
        $ butil -stat file.btr > file.std
        
        (Edit file.std to add an extra column thusly:)
        
        (Leave what's above here alone...)
        Record Length = 171 
        Record Length = <whatever you want the fixed length to be.>
    
        (...leave the next alone until the Key defs below.
        Some columns elided for space.)
        
        Key         Pos..     Type        Null V..    ACS
            Segment      Len..       Flags       Uniq..     
          0    1      1    4  Zstring         - .. 8   --
          1    1      5    4  Integer  MD     - .. 8   --
        
        (..and what's after alone.)
        
        
        Add an extra column:
        
        Key         Pos..    Type         Null V..    ACS
            Segment      Len..        Flags      Uniq..    Langname
          0    1      1    4  Zstring         --.. 8  --   Author
          1    1      5    4  Integer  MD     --.. 8  --   dbcn
    
        $ cat btr_rdb.pl
        #!/usr/bin/perl
        print "Usage: btr_rdb.pl <rdbfile><conf><savefile><error savefile>" 
                unless scalar @ARGV ==4;
        my ($rdb,$config,$save,$err) = @ARGV;
        my $btr = BTRIEVE::SAVE->new($config);
        $btr->save_to_rdb($rdb,$save,$err,
                               'ZZ','VAR','<TAB>','<RET>');
    
        $ rdb_btr.pl file.rdb file.std file.sav file.err
        $ wc file.err
          0       1       1 file.err
        $

AUTHORS

Chuck Bearden cbearden@rice.edu

Bill Birthisel wcbirthisel@alum.mit.edu

Derek Lane dereklane@pobox.com

Charles McFadden chuck@vims.edu

Ed Summers esummers@odu.edu

SEE ALSO

perl(1), www.pervasive.com, "Btrieve Complete" by Jim Kyle (Addison-Wesley 1995).

COPYRIGHT

Copyright (C) 2000, Bearden, Birthisel, Lane, McFadden, Summers. All rights reserved. Copyright (C) 2000, Duke University, Lane. All rights reserved.

This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself. Feb 15 2000.