NAME

File::BufferedInput - Large and giant text file performance buffered reader.

SYNOPSIS

use File::BufferedInput;

# create new object with default options, buffer size 20MB and ANSI text file
# file argument can be file name or file handle
my $fileobj = File::BufferedInput->new(file=>$filename);

# or create new object with custom options, set buffer size 50MB and UTF-8 text file
my $fileobj = File::BufferedInput->new(file=>$filename, buffer=>50_000_000, utf8=>1);

# or create new object then set the file name/handle
my $fileobj = File::BufferedInput->new();
$fileobj->file($filename_or_handle);

# now loop through all the text file lines straight forward

my $counter = 0;
# loop through the file lines sequentially
while (!$fileobj->eof()) {
	 $line=$fileobj->line();
	$counter++;
	# print "$counter)- $line\n";
}

# start again from the begaining of the file
#$fileobj->rewind()

$fileobj->close(); # close the file and frees the memory used for the block
print "$counter lines found\n";

DESCRIPTION

This module solves the problem with reading large and huge text files in Perl. It is designed to read only block by block as needed. It does not load the whole file into memory, it only reads one block at a time and once the last sequential line reached, it reads the next block from the file and frees the previous block from memory, so at all times only one block of the file is kept in menory.

For example if you are reading a 2GB file, once you start reading lines from the file, the module reads the first block from the file on disk, while you loop through the lines, when you reach the line at the end of the read block, the module delete this block from memory and read the next block from the file on disk and parses it to lines and so on.

SEE ALSO

AUTHOR

Ahmed Amin Elsheshtawy, <support@mewsoft.com> Website: http://www.mewsoft.com

COPYRIGHT AND LICENSE

Copyright (C) 2014 by Ahmed Amin Elsheshtawy support@mewsoft.com http://www.mewsoft.com

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.