NAME
Data::Resolver - resolve keys to data
VERSION
This document describes Data::Resolver version 0.003.
SYNOPSIS
# generate is a single entry point useful to instantiate from
# metadata/configurations
use Data::Resolver 'generate';
my $spec = { -factory => 'resolver_from_tar',
path => '/to/archive.tar' };
my $dr_tar = generate($spec);
my $dr_dir = generate({ -factory => 'resolver_from_dir',
path => '.' });
# functions can be imported and used directly though
use Data::Resolver qw< resolver_from_tar resolver_from_dir >;
my $dr_tar = resolver_from_tar(path => 'to/somewhere.tar');
# getting stuff is easy, this is how to get the default
# representation
my ($thing, $meta) = $dr_something->($key);
my $type = $meta->{type};
if ($type eq 'file') { say 'got a file' }
elsif ($type eq 'filehandle') { say 'got a filehandle' }
elsif ($type eq 'data') ) { say 'got data' }
elsif ($type eq 'error') { ... }
# you can specify the type and just get the value
my $data = $dr_something->($key, 'data');
my $path = $dr_something->($key, 'file');
my $fh = $dr_something->($key, 'filehandle');
# it's possible to get a list of available keys
my $list_aref = $dr_something->(undef, 'list');
DESCRIPTION
While coding, two problems often arise:
Using several modules, there can be a variety of ways on how they get access to data. Many times they support reading from a file, but often times they expect to receive data (e.g. JSON::PP). Other times modules an be OK with both, and even accept filehandles.
Deciding on where to store data and what to use as a source can be limiting, especially when multiple things might be needed. What is best at that point? A directory? An archive? A few URLs?
This module aims at providing a way forward to cope for both problems, by providing a unified interface that can get three types of data types (i.e. data
, file
, or filehandle
) while at the same time providing a very basic interface that can be backed by several different fetching approaches, like reading from a directory, taking items from an archive, or download stuff on the fly from a URL.
The Resolver Interface
A valid resolver is a function that supports at least the following interface:
The resolver has the following signature:
my $resolver = sub ($key, [$type]) { ... }
where
$key
is a mandatory parameter providing the key that we want to resolve to a data representation, and$type
is an optional parameter that specifies what representation is needed.When called in list context, two items are provided back, i.e. the value and the metadata:
my ($value, $meta) = $resolver->(@args);
The
$meta
is a hash reference that contains at least one keytype
, indicating the type of the$value
. Allowed types are at least the following:data
-
The
$value
is directly data. It might be provided either as a plain scalar, or as a reference to a plain scalar (ref()
will help disambiguate the two cases). error
-
The
$value
should be ignored because an error occurred during retrieval. When the resolver is set for throwing an exception, this is never returned. file
-
The
$value
represents a file path in the filesystem. filehandle
-
The
$value
is a filehandle, suitable for reading. Characteristics of thi filehandle may vary, although it SHOULD support seeking.
When called in scalar context, only the
$value
is provided back:my $value = $resolver->(@args);
In this case it is usually better to also provide a type as the second argument, unless the default return type for the resolver is already known in advance.
The following invocation provides back a list of all supported keys:
my $list = $resolver->(undef, 'list');
Examples:
# get list of supported keys, as an array ref
my $list = $resolver->(undef, 'list');
# get value associated to key 'foo', as raw data
my $data = $resolver->(foo => 'data');
# get value and metadata, decide later how to use them
my ($value, $meta) = $resolver->('foo');
if ($meta->{type} eq 'file') { ... }
Stock Factories
The module comes with a few stock factory functions to generate resolvers in a few cases:
A directory in the filesystem, via "resolver_from_dir".
A TAR archive, via "resolver_from_tar".
A list of resolvers, via "resolver_from_alternatives". This allows e.g. looking for a resolution of the key from multiple sources (possibly of different kinds).
INTERFACE
The interface provided by this module can be broadly divided into three areas:
factories to generate resolvers;
transformers to ease turning a data representation into another (e.g. turning data into a file or a filehandle)
utilities for coding new resolvers/resolver factories.
Factories
These functions generate resolvers.
generate
my $resolver = generate($specification_hash);
Generate a new resolver based on a hash containing a specification. The following meta-keys are supported:
-factory
-
The name of the factory function (e.g. "resolver_from_tar").
-package
-
The package where the factory function above is located. By default this is
Data::Resolver
. --recursed-args
-
A sub-hash where values are array references, holding sub-specifications that are generated recursively via
generate
itself. The key and the resulting array are then inserted as new keys/value pairs in the hash.
All the rest of the hash is passed to the factory function as key/value pairs.
Example:
my $spec = { -factory => 'resolver_from_tar',
path => '/to/archive.tar' };
my $dr_tar = generate($spec);
my $spec = {
-factory => 'resolver_from_alternatives',
-recursed => {
alternatives => [
{ -factory => resolver_from_tar => archive => $tar },
{ -factory => resolver_from_dir => root => $dir },
],
}
};
my $dr_multi = generate($spec);
resolver_from_alternatives
my $dr = resolver_from_alternatives(%args);
Generate a resolver that wraps other resolvers and tries them in sequence, until the first supporing the input key.
It cares about two keys in %args
:
alternatives
-
Accepts a reference to an array of sub-resolvers (i.e.
CODE
references) or sub-resolver specifications (which will be instantiated via "generate"). throw
-
If set to a true value, raise an exception in case of errors.
The list
type is supported for the undef
key only. It replicates the call over all alternatives, aggregating the result in the order they appear while filtering out duplicates (it does not try to normalize keys in any way, so this might give practical duplicates out).
The search for a key is performed in the same order as the sub-resolvers appear in alternatives
; when a result is found, it is returned. Exceptions from sub-resolvers are trapped.
If throw
is set, errors will raise exceptions thrown as hashes, via "resolved_error"/"resolved". This happens in two cases:
the call for type
list
does not provide anundef
key. In this case, the error code is set to400
.the call for any other type does not provide a result back. In this case, the error code is set to
404
.
resolver_from_dir
my $dr = resolver_from_dir(%args);
Generate a resolver that serves files from a directory in the local filesystem. It supports the following keys in %args
:
path
root
-
The path to the directory containing the files.
root
takes precedence overpath
. throw
-
If set to a true value, raise an exception in case of errors.
Errors are handled via "resolved_error"/"resolved", including raising exceptions.
The call to type list
with an undef
key will generate a lit of all files in the subtree starting from path
/root
. As an extension, it's also possible to pass the name of a sub-directory and get only that subtree back; this is prone to errors if the sub-directory does not exist (error code 404
) or is a file instead of a directory (error code 400
).
The call for other types will resolve the key and provide back the requested type, defaulting to type file
for effort minimization. The code tries to restrict looking for the file only inside the sub-tree but you should check by yourself if this is really critical (patches welcome!).
If the key cannot be found, error code 404
is set; if the key refers to a directory, error code 400
is set; if the type
cannot be handled via "transform", error code 400
is set.
resolver_from_tar
my $dr = resolver_from_tar(%args);
Generate a resolver that serves file from a TAR file in the local filesystem. It supports the following keys in %args
:
archive
path
-
The path to the TAR file.
archive
takes precedence overpath
. throw
-
If set to a true value, raise an exception in case of errors.
Errors are handled via "resolved_error"/"resolved", including raising exceptions.
The call for type list
only supports key undef
and will lead to error code 400
otherwise.
The call for any other type will look for the key and return the requested type, defaulting to type data
. Two keys are actually searched, to cater for the equivalence of path/to/file
and ./path/to/file
; this means that it's possible to ask for somefile
and get back the contents of ./somefile
, or vice-versa.
If a file for a key cannot be found, error code 404
is returned. This also applies if the key is present, but represents a directory item.
Type transformation is performed via "transform"; unsupported types will lead to error code 400
after the search and extraction of data from the archive (i.e. there is no attempt to pre-validate the type and this is by design).
resolver_from_passthrough
my $dr = resolver_from_passthrough(%args);
Generate a minimal, fake-like resolver that always returns what is provided. Arguments %args
are added to the return value via "resolved", so key throw
might lead to exceptions.
In the generated resolver, the type defaults to undef
. No attempt at validation is done, by design.
Transformers
data_to_fh
my $data = 'Some thing';
my $fh1 = data_to_fh($data); # first way
my $fh2 = data_to_fh(\$data); # second way
Gets a scalar or a scalar reference and provides a filehandle back, suitable for reading/seeking.
data_to_file
my $data = 'Some thing';
my $path1 = data_to_file($data); # plain scalar in
my $path2 = data_to_file(\$data); # scalar reference in
my $persistent_path = data_to_file($data, 1);
Gets a scalar or a scalar reference and saves it to a temporary file in the filesystem. A second parameter, when true, makes it possible to persist the file after the process exists (the file would be removed otherwise).
fh_to_data
open my $fh, '<:raw', $some_path or die '...';
my $data = fh_to_data($fh);
Slurps data from a filehandle. The filehandle is not changed otherwise, so it's up to the caller to set the right Perl IO layers if needed.
fh_to_file
open my $fh, '<:raw', $some_path or die '...';
my $path = fh_to_file($fh);
my $persistent_path = fh_to_file($fh, 1);
Slurps data from a filehandle and saves it into a temporary file (or persistent, if the second parameter is present and true).
file_to_data
my $data = file_to_data($path);
Slurps data from a file in the filesystem. Data are read in raw mode.
file_to_fh
my $fh = file_to_fh($path);
Opens a file in the filesystem in read raw mode.
transform
my $ref_to_that = transform($this, $this_type, $that_type);
Treat input $this
as having $this_type
and return it as a reference to a $that_type
.
Input and output types can be:
fh
filehandle
-
the input is a file handle
data
-
the input is raw data
file
path
-
the input is a path in the filesystem.
NOTE the return value is a reference to the target data form, to avoid transferring too much data around.
Utilities For New Resolvers
This module comes with two non-trivial resolvers, one for wrapping a directory and another one for tar archives. There can be other possible resolvers, e.g. using different archive formats (like ZIP), leveraging any file format that supports carrying metadata (like PDF, or many image formats), or wrapping remote resources (plain HTTP or some fancy API).
These functions help complying with the output API of a resolver, i.e.:
throw an exception when errors occur and the resolver was created with
throw
parameter set;return just the content in scalar context;
return the content and additional metadata in list context.
A typical way of using these function is like this:
sub resolver_for_whatever (%args) {
my $OK = resolved_factory($args{throw});
my $KO = resolved_error_factory($args{throw});
return sub ($key, $type = 'xxx') {
return $KO->(400 => 'Wrong inputs!') if $some_error;
return $OK->($data, type => 'data');
};
}
resolved
return resolved($throw, $value, $meta_as_href);
return resolved($throw, $value, %meta);
Throw an exception if $throw
is true and metadata have type
set to error
.
Otherwise, return $value
if called in scalar context.
Otherwise, return a list with $value
and a hash reference with the metadata.
resolved_error
return resolved_error($throw, $code, $message, $meta);
return resolved_error($throw, $code, $message, %meta);
If an error has to be returned, this is a shorthand to integrate the optional metadata with a code and a message. If $throw
is set, an exception is thrown.
resolved_error_factory
my $error_return = resolved_error_factory($throw);
Wrap "resolved_error" with the specific value for $throw
. This can be useful because whether a resolver should throw exceptions or not is usually set at resolver creation time, so it makes sense to wrap this characteristic.
resolved_factory
my $return = resolved_factory($throw);
Wrap "resolved" with the specific value for $throw
. This can be useful because whether a resolver should throw exceptions or not is usually set at resolver creation time, so it makes sense to wrap this characteristic.
BUGS AND LIMITATIONS
Minimum perl version 5.24 because reasons (it's been around since 2016 and signatures just make sense).
Report bugs through Codeberg (patches welcome) at https://codeberg.org/polettix/Data-Resolver/issues.
AUTHOR
Flavio Poletti <flavio@polettix.it>
COPYRIGHT AND LICENSE
Copyright 2023 by Flavio Poletti <flavio@polettix.it>
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.