-
-
03 Sep 2008 20:47:49 UTC
- Distribution: IO-Compress-Bzip2
- Source (raw)
- Browse (raw)
- Changes
- How to Contribute
- Issues (0)
- Testers (488 / 57 / 0)
- Kwalitee
- License: perl_5
- Activity
24 month- Tools
- Download (73.62KB)
- MetaCPAN Explorer
- Permissions
- Subscribe to distribution
- Permalinks
- This version
- Dependencies
- Compress::Raw::Bzip2
- IO::Compress::Base
- IO::Uncompress::Base
- and possibly others
- Reverse dependencies
- CPAN Testers List
- Dependency graph
NAME
IO::Compress::Bzip2::FAQ -- Frequently Asked Questions about IO::Compress::Bzip2
DESCRIPTION
Common questions answered.
Compatibility with Unix compress/uncompress.
This module is not compatible with Unix
compress
.If you have the
uncompress
program available, you can use this to read compressed filesopen F, "uncompress -c $filename |"; while (<F>) { ...
Alternatively, if you have the
gunzip
program available, you can use this to read compressed filesopen F, "gunzip -c $filename |"; while (<F>) { ...
and this to write compress files, if you have the
compress
program availableopen F, "| compress -c $filename "; print F "data"; ... close F ;
Accessing .tar.Z files
See previous FAQ item.
If the
Archive::Tar
module is installed and either theuncompress
orgunzip
programs are available, you can use one of these workarounds to read.tar.Z
files.Firstly with
uncompress
use strict; use warnings; use Archive::Tar; open F, "uncompress -c $filename |"; my $tar = Archive::Tar->new(*F); ...
and this with
gunzip
use strict; use warnings; use Archive::Tar; open F, "gunzip -c $filename |"; my $tar = Archive::Tar->new(*F); ...
Similarly, if the
compress
program is available, you can use this to write a.tar.Z
fileuse strict; use warnings; use Archive::Tar; use IO::File; my $fh = new IO::File "| compress -c >$filename"; my $tar = Archive::Tar->new(); ... $tar->write($fh); $fh->close ;
Accessing Zip Files
This module does not support reading/writing zip files.
Support for reading/writing zip files is included with the
IO::Compress::Zip
andIO::Uncompress::Unzip
modules.The primary focus of the
IO::Compress::Zip
andIO::Uncompress::Unzip
modules is to provide anIO::File
compatible streaming read/write interface to zip files/buffers. They are not fully flegged archivers. If you are looking for an archiver check out theArchive::Zip
module. You can find it on CPAN athttp://www.cpan.org/modules/by-module/Archive/Archive-Zip-*.tar.gz
Compressed files and Net::FTP
The
Net::FTP
module provides two low-level methods calledstor
andretr
that both return filehandles. These filehandles can used with theIO::Compress/Uncompress
modules to compress or uncompress files read from or written to an FTP Server on the fly, without having to create a temporary file.Firstly, here is code that uses
retr
to uncompressed a file as it is read from the FTP Server.use Net::FTP; use IO::Uncompress::Bunzip2 qw(:all); my $ftp = new Net::FTP ... my $retr_fh = $ftp->retr($compressed_filename); bunzip2 $retr_fh => $outFilename, AutoClose => 1 or die "Cannot uncompress '$compressed_file': $Bunzip2Error\n";
and this to compress a file as it is written to the FTP Server
use Net::FTP; use IO::Compress::Bzip2 qw(:all); my $stor_fh = $ftp->stor($filename); bzip2 "filename" => $stor_fh, AutoClose => 1 or die "Cannot compress '$filename': $Bzip2Error\n";
How do I recompress using a different compression?
This is easier that you might expect if you realise that all the
IO::Compress::*
objects are derived fromIO::File
and that all theIO::Uncompress::*
modules can read from anIO::File
filehandle.So, for example, say you have a file compressed with gzip that you want to recompress with bzip2. Here is all that is needed to carry out the recompression.
use IO::Uncompress::Gunzip ':all'; use IO::Compress::Bzip2 ':all'; my $gzipFile = "somefile.gz"; my $bzipFile = "somefile.bz2"; my $gunzip = new IO::Uncompress::Gunzip $gzipFile or die "Cannot gunzip $gzipFile: $GunzipError\n" ; bzip2 $gunzip => $bzipFile or die "Cannot bzip2 to $bzipFile: $Bzip2Error\n" ;
Note, there is a limitation of this technique. Some compression file formats store extra information along with the compressed data payload. For example, gzip can optionally store the original filename and Zip stores a lot of information about the original file. If the original compressed file contains any of this extra information, it will not be transferred to the new compressed file usign the technique above.
Using
InputLength
to uncompress data embedded in a larger file/buffer.A fairly common use-case is where compressed data is embedded in a larger file/buffer and you want to read both.
As an example consider the structure of a zip file. This is a well-defined file format that mixes both compressed and uncompressed sections of data in a single file.
For the purposes of this discussion you can think of a zip file as sequence of compressed data streams, each of which is prefixed by an uncompressed local header. The local header contains information about the compressed data stream, including the name of the compressed file and, in particular, the length of the compressed data stream.
To illustrate how to use
InputLength
here is a script that walks a zip file and prints out how many lines are in each compressed file (if you intend write code to walking through a zip file for real see "Walking through a zip file" in IO::Uncompress::Unzip )use strict; use warnings; use IO::File; use IO::Uncompress::RawInflate qw(:all); use constant ZIP_LOCAL_HDR_SIG => 0x04034b50; use constant ZIP_LOCAL_HDR_LENGTH => 30; my $file = $ARGV[0] ; my $fh = new IO::File "<$file" or die "Cannot open '$file': $!\n"; while (1) { my $sig; my $buffer; my $x ; ($x = $fh->read($buffer, ZIP_LOCAL_HDR_LENGTH)) == ZIP_LOCAL_HDR_LENGTH or die "Truncated file: $!\n"; my $signature = unpack ("V", substr($buffer, 0, 4)); last unless $signature == ZIP_LOCAL_HDR_SIG; # Read Local Header my $gpFlag = unpack ("v", substr($buffer, 6, 2)); my $compressedMethod = unpack ("v", substr($buffer, 8, 2)); my $compressedLength = unpack ("V", substr($buffer, 18, 4)); my $uncompressedLength = unpack ("V", substr($buffer, 22, 4)); my $filename_length = unpack ("v", substr($buffer, 26, 2)); my $extra_length = unpack ("v", substr($buffer, 28, 2)); my $filename ; $fh->read($filename, $filename_length) == $filename_length or die "Truncated file\n"; $fh->read($buffer, $extra_length) == $extra_length or die "Truncated file\n"; if ($compressedMethod != 8 && $compressedMethod != 0) { warn "Skipping file '$filename' - not deflated $compressedMethod\n"; $fh->read($buffer, $compressedLength) == $compressedLength or die "Truncated file\n"; next; } if ($compressedMethod == 0 && $gpFlag & 8 == 8) { die "Streamed Stored not supported for '$filename'\n"; } next if $compressedLength == 0; # Done reading the Local Header my $inf = new IO::Uncompress::RawInflate $fh, Transparent => 1, InputLength => $compressedLength or die "Cannot uncompress $file [$filename]: $RawInflateError\n" ; my $line_count = 0; while (<$inf>) { ++ $line_count; } print "$filename: $line_count\n"; }
The majority of the code above is concerned with reading the zip local header data. The code that I want to focus on is at the bottom.
while (1) { # read local zip header data # get $filename # get $compressedLength my $inf = new IO::Uncompress::RawInflate $fh, Transparent => 1, InputLength => $compressedLength or die "Cannot uncompress $file [$filename]: $RawInflateError\n" ; my $line_count = 0; while (<$inf>) { ++ $line_count; } print "$filename: $line_count\n"; }
The call to
IO::Uncompress::RawInflate
creates a new filehandle$inf
that can be used to read from the parent filehandle$fh
, uncompressing it as it goes. The use of theInputLength
option will guarantee that at most$compressedLength
bytes of compressed data will be read from the$fh
filehandle (The only exception is for an error case like a truncated file or a corrupt data stream).This means that once RawInflate is finished
$fh
will be left at the byte directly after the compressed data stream.Now consider what the code looks like without
InputLength
while (1) { # read local zip header data # get $filename # get $compressedLength # read all the compressed data into $data read($fh, $data, $compressedLength); my $inf = new IO::Uncompress::RawInflate \$data, Transparent => 1, or die "Cannot uncompress $file [$filename]: $RawInflateError\n" ; my $line_count = 0; while (<$inf>) { ++ $line_count; } print "$filename: $line_count\n"; }
The difference here is the addition of the temporary variable
$data
. This is used to store a copy of the compressed data while it is being uncompressed.If you know that
$compressedLength
isn't that big then using temporary storage won't be a problem. But if$compressedLength
is very large or you are writing an application that other people will use, and so have no idea how big$compressedLength
will be, it could be an issue.Using
InputLength
avoids the use of temporary storage and means the application can cope with large compressed data streams.One final point -- obviously
InputLength
can only be used whenever you know the length of the compressed data beforehand, like here with a zip file.SEE ALSO
Compress::Zlib, IO::Compress::Gzip, IO::Uncompress::Gunzip, IO::Compress::Deflate, IO::Uncompress::Inflate, IO::Compress::RawDeflate, IO::Uncompress::RawInflate, IO::Compress::Bzip2, IO::Uncompress::Bunzip2, IO::Compress::Lzop, IO::Uncompress::UnLzop, IO::Compress::Lzf, IO::Uncompress::UnLzf, IO::Uncompress::AnyInflate, IO::Uncompress::AnyUncompress
File::GlobMapper, Archive::Zip, Archive::Tar, IO::Zlib
AUTHOR
This module was written by Paul Marquess, pmqs@cpan.org.
MODIFICATION HISTORY
See the Changes file.
COPYRIGHT AND LICENSE
Copyright (c) 2005-2008 Paul Marquess. All rights reserved.
This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
Module Install Instructions
To install IO::Compress::Bzip2, copy and paste the appropriate command in to your terminal.
cpanm IO::Compress::Bzip2
perl -MCPAN -e shell install IO::Compress::Bzip2
For more information on module installation, please visit the detailed CPAN module installation guide.