Archive::BagIt - The main module to handle bags.
version 0.095
Achive::BagIt - The main module to handle Bags
The original development version was on github at http://github.com/rjeschmi/Archive-BagIt and may be cloned from there.
The actual development version is available at https://git.fsfe.org/art1pirat/Archive-BagIt
The module should fulfill the RFC requirements, with following limitations:
At the moment only filepaths in linux-style are supported.
To get an more detailled overview, see the testsuite under t/verify_bag.t and corresponding test bags from the BagIt conformance testsuite of Library of Congress under bagit_conformance_suite/.
See https://datatracker.ietf.org/doc/rfc8493/?include_text=1 for details.
Try this:
foreach my $algorithm ( keys %{ $self->manifests }) { my $entries_ref = $self->manifests->{$algorithm}->manifest_entries(); # $entries_ref returns a hashref like: # { # data/hello.txt "e7c22b994c59d9cf2b48e549b1e24666636045930d3da7c1acb299d1c3b7f931f94aae41edda2c2b207a36e10f8bcb8d45223e54878f5b316e7ce3b6bc019629" # } }
Similar for tagmanifests
I have made great efforts to optimize Archive::BagIt for high throughput. There are two limiting factors:
On my system with 8cores, SSD and a large 9GB bag with 568 payload files the results for verify_bag() are:
verify_bag()
processing time run time throughput Version user time system time total time total MB/s v0.71 38.31s 1.60s 39.938s 100% 230 v0.81 25.48s 1.68s 27.1s 67% 340 v0.82 48.85s 3.89s 6.84s 17% 1346
It depends. On my system with 8cores, SSD and a 38MB bag with 48 payload files the results for verify_bag() are:
Rate Base Fast Base 3.01/s -- -21% Fast 3.80/s 26% --
s/iter Base Fast Base 74.6 -- -9% Fast 68.3 9% --
But you should measure which variant is best for you. In general the default Archive::BagIt is fast enough.
You could try this:
use Archive::BagIt; my $bag=Archive::BagIt->new( $my_old_bag_filepath ); $bag->load(); $bag->store();
For versions < Windows10: I have no idea and suggestions for a portable solution are very welcome! For Windows 10: Thanks to https://superuser.com/questions/1033088/is-it-possible-to-set-locale-of-a-windows-application-to-utf-8/1451686#1451686 you have to enable UTF-8 support via 'System Administration' -> 'Region' -> 'Administrative' -> 'Region Settings' -> Flag 'Use Unicode UTF-8 for worldwide language support'
Hint: The better way is to use only portable filenames. See perlport for details.
There are problems related to Parallel::parallel_map and IO::AIO under MS Windows. The tests are skipped there. Use the parallel feature or the Archive::BagIt::Fast at your own risks on a MS Window System. If you are a MS Windows developer, feel free to send me patches or hints to fix the issues.
Thanks to Rob Schmidt <rjeschmi@gmail.com> for the trustful handover of the project and thanks for your initial work! I would also like to thank Patrick Hochstenbach and Rusell McOrmond for their valuable and especially detailed advice! And without the helpful, sometimes rude help of the IRC channel #perl I would have been stuck in a lot of problems. Without the support of my colleagues at SLUB Dresden, the project would never have made it this far.
This modules will hopefully help with the basic commands needed to create and verify a bag. This part supports BagIt 1.0 according to RFC 8493 ([https://tools.ietf.org/html/rfc8493](https://tools.ietf.org/html/rfc8493)).
You only need to know the following methods first:
use Archive::BagIt; #read in an existing bag: my $bag_dir = "/path/to/bag"; my $bag = Archive::BagIt->new($bag_dir);
use Archive::BagIt; my $bag2 = Archive::BagIt->make_bag($bag_dir);
use Archive::BagIt; # Validate a BagIt archive against its manifest my $bag3 = Archive::BagIt->new($bag_dir); my $is_valid1 = $bag3->verify_bag(); # Validate a BagIt archive against its manifest, report all errors my $bag4 = Archive::BagIt->new($bag_dir); my $is_valid2 = $bag4->verify_bag( {report_all_errors => 1} );
Because all methods operate lazy, you should ensure to parse parts of the bag *BEFORE* you modify it. Otherwise it will be overwritten!
use Archive::BagIt; my $bag5 = Archive::BagIt->new($bag_dir); # lazy, nothing happened $bag5->load(); # this updates the object representation by parsing the given $bag_dir $bag5->store(); # this writes the bag new
The constructor sub, will create a bag with a single argument,
or use hashreferences
use Archive::BagIt; #read in an existing bag: my $bag_dir = "/path/to/bag"; my $bag = Archive::BagIt->new( bag_path => $bag_dir, );
The arguments are:
bag_path
force_utf8
use_async
use_parallel
use_plugins
Archive::BagIt::Plugin::Manifest::SHA256
The bag object will use $bag_dir, BUT an existing $bag_dir is not read. If you use store() an existing bag will be overwritten!
store()
See load() if you want to parse/modify an existing bag.
load()
if set it uses parallel digest processing, default: false
if set it uses async IO, default: false
to check if force_utf8() was set.
If set it ignores warnings about potential filepath problems.
Getter/setter for bag path
Getter for metadata path
Getter for payload path
Getter for registered Checksums
Getter for bag version
Getter for bag encoding.
HINT: the current version of Archive::BagIt only supports UTF-8, but the method could return other values depending on given Bags.
Getter/Setter for bag info. Expects/returns an array of HashRefs implementing simple key-value pairs.
HINT: RFC8493 does not allow *reordering* of entries!
returns true if bag info exists.
Getter to return collected errors after a verify_bag() call with Option report_all_errors
report_all_errors
Getter to return collected warnings after a verify_bag() call
This method could be reimplemented by derived classes to handle fixity checks in own way. The getter returns an anonymous function with following interface:
my $digest = $self->digest_callback; &$digest( $digestobject, $filename);
This anonymous function MUST use the get_hash_string() function of the Archive::BagIt::Role::Algorithm role, which is implemented by each Archive::BagIt::Plugin::Algorithm::XXXX module.
get_hash_string()
See Archive::BagIt::Fast for details.
Returns all values which match $searchkey, undef otherwise
returns true if key is reserved and should be uniq
returns true if key is reserved
checks baginfo-keys, returns true if all fine, otherwise returns undef and the message is pushed to errors(). Warnings pushed to warnings()
errors()
warnings()
deletes an entry of given $searchkey if exists. If multiple entries with $searchkey exists, only the last one is deleted.
returns true if a given $searchkey exists
Appends a key value pair to bag_info.
HINT: check return code if append was successful, because some keys needs to be uniq.
It replaces the first entry with $newvalue if $searchkey exists, otherwise it appends.
Getter to return the forced fixity algorithm depending on BagIt version
Getter to find all manifest-files
Getter to find all tagmanifest-files
Getter to find all payload-files
Getter to find all non payload-files
Getter/setter to algorithm plugins
Getter/Setter to all manifests (objects)
Getter/Setter to all registered Algorithms
As default SHA512 and MD5 will be loaded and therefore used. If you want to create a bag only with one or a specific checksum-algorithm, you could use this method to (re-)register it. It expects list of strings with namespace of type: Archive::BagIt::Plugin::Algorithm::XXX where XXX is your chosen fixity algorithm.
Triggers loading of an existing bag
A method to verify a bag deeply. If $opts is set with {return_all_errors} all fixity errors are reported. The default ist to croak with error message if any error is detected.
$opts
{return_all_errors}
HINT: You might also want to check Archive::BagIt::Fast to see a more direct way of accessing files (and thus faster).
returns an array with octets and streamcount of payload-dir
returns a string with human readable size of paylod
creates a bagit.txt file
creates a bag-info.txt file
Hint: the entries 'Bagging-Date', 'Bag-Software-Agent', 'Payload-Oxum' and 'Bag-Size' will be automagically set, existing values in internal bag-info representation will be overwritten!
store a bagit-obj if bagit directory-structure was already constructed.
A constructor that will just create the metadata directory
This won't make a bag, but it will create the conditions to do that eventually
A constructor that will make and return a bag from a directory,
It expects a preliminary bagit-dir exists. If there a data directory exists, assume it is already a bag (no checking for invalid files in root)
The latest version of this module is available from the Comprehensive Perl Archive Network (CPAN). Visit http://www.perl.com/CPAN/ to find a CPAN site near you, or see https://metacpan.org/module/Archive::BagIt/.
You can make new bug reports, and view existing ones, through the web interface at http://rt.cpan.org.
Andreas Romeyke <cpan@andreas.romeyke.de>
This software is copyright (c) 2022 by Rob Schmidt <rjeschmi@gmail.com>, William Wueppelmann and Andreas Romeyke.
This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.
To install Archive::BagIt, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Archive::BagIt
CPAN shell
perl -MCPAN -e shell install Archive::BagIt
For more information on module installation, please visit the detailed CPAN module installation guide.