NAME

File::BufferedInput - Large and giant text file performance buffered reader.

SYNOPSIS

        use File::BufferedInput;

        # create new object with default options, buffer size 20MB and ANSI text file
        # file argument can be file name or file handle
        my $fileobj = File::BufferedInput->new(file=>$filename);
        
        # or create new object with custom options, set buffer size 50MB and UTF-8 text file
        my $fileobj = File::BufferedInput->new(file=>$filename, buffer=>50_000_000, utf8=>1);

        # or create new object then set the file name/handle
        my $fileobj = File::BufferedInput->new();
        $fileobj->file($filename_or_handle);

        # now loop through all the text file lines straight forward
        
        my $counter = 0;
        # loop through the file lines sequentially
        while (!$fileobj->eof()) {
                 $line=$fileobj->line();
                $counter++;
                # print "$counter)- $line\n";
        }

        # start again from the begaining of the file
        #$fileobj->rewind()

        $fileobj->close(); # close the file and frees the memory used for the block
        print "$counter lines found\n";
        
        

DESCRIPTION

This module solves the problem with reading large and huge text files in Perl. It is designed to read only block by block as needed. It does not load the whole file into memory, it only reads one block at a time and once the last sequential line reached, it reads the next block from the file and frees the previous block from memory, so at all times only one block of the file is kept in menory.

For example if you are reading a 2GB file, once you start reading lines from the file, the module reads the first block from the file on disk, while you loop through the lines, when you reach the line at the end of the read block, the module delete this block from memory and read the next block from the file on disk and parses it to lines and so on.

SEE ALSO

AUTHOR

Ahmed Amin Elsheshtawy, <support@mewsoft.com> Website: http://www.mewsoft.com

COPYRIGHT AND LICENSE

Copyright (C) 2014 by Ahmed Amin Elsheshtawy support@mewsoft.com http://www.mewsoft.com

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.