The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.
 / 
Ref-Store-0.02_0
River stage zero No dependents
2 ++
/ Ref::Store

NAME

Ref::Store - Store objects, index by object, tag by objects - all without leaking.

SYNOPSIS

        my $table = Ref::Store->new();

Store a value under a simple string key, maintain the value as a weak reference. The string key will be deleted when the value is destroyed:

        $table->store("key", $object);

Store $object under a second index ($fh), which is a globref; $fh will automatically be garbage collected when $object is destroyed.

        {
                open my $fh, ">", "/foo/bar";
                $table->store($fh, $object, StrongKey => 1);
        }
        # $fh still exists with a sole reference remaining in the table

Register an attribute type (foo_files), and tag $fh as being one of $foo_files, $fh is still dependent on $object

        # assume $fh is still in scope
        
        $table->register_kt("foo_files");
        $table->store_a(1, "foo_files", $fh);

Store another foo_file

        open my $fh2, ">", "/foo/baz"
        $table->store_a(1, "foo_files", $fh);
        # $fh2 will automatically be deleted from the table when it goes out of scope
        # because we did not specify StrongKey

Get all foo_files

        my @foo_files = $table->fetch_a(1, "foo_files");
        
        # @foo_files contains ($fh, $fh2);

Get rid of $object. This can be done in one of the following ways:

        # Implicit garbage collection
        undef $object;
        
        # Delete by value
        $table->purge($object);
        
        # Delete by key ($fh is still stored under the foo_keys attribute)
        $table->purgeby($fh);

        # remove each key for the $object value
        $table->unlink("key");
        $table->unlink($fh); #fh still exists under "foo" files

Get rid of foo_file entries

        # delete, by attribute
        $table->purgeby_a(1, "foo_files");
        
        # delete a single attribute from all entries
        $table->unlink_a(1, "foo_files");
        
        # dissociate the 'foo_files' attribtue from each entry
        $table->dissoc_a(1, "foo_files", $fh);
        $table->dissoc_a(1, "foo_files", $fh2);
        
        # implicit garbage collection:
        undef $fh;
        undef $fh2;

For a more detailed walkthrough, see Ref::Store::Walkthrough

DESCRIPTION

Ref::Store provides an efficient and worry-free way to index objects by arbitrary data - possibly other objects, simple scalars, or whatever.

It relies on magic and such to ensure that objects you put in the lookup table are not maintained there unless you want them to be. In other words, you can store objects in the table, and delete them without having to worry about what other possible indices/references may be holding down the object.

If you are looking for something to store Data, then direct your attention to KiokuDB, Tangram or Pixie - these modules will store your data.

However, if you are specifically wanting to maintain garbage-collected and reference counted perl objects, then this module is for you. continue reading.

USAGE APPLICATIONS AND BENEFITS

This module caters to the common, but very narrow scope of opaque perl references. It cares nothing about what kind of objects you are using as keys or values. It will never dereference your object or call any methods on it, thus the only requirement for an object is that it be a perl reference.

Using this module, it is possible to create arbitrarily complex, but completely leak free dependency and relationship graphs between objects.

Sometimes, expressing object lifetime dependencies in terms of encapsulation is not desirable. Circular references can happen, and the hierarchy can become increasingly complex and deep - not just logically, but also syntactically.

This becomes even more unwieldy when the same object has various sets of dependants.

A good example would be a socket server proxy, which accepts requests from clients, opens a second connection to a third-party server to process the client request, forwards the request to the client's intended origin server, and then finally relays the response back to the client.

At a more basic level, this module is good for general simple and safe by-object indexing and object tagging. It is also a good replacement for Hash::Util::FieldHash support for perls which do not support tied hash uvar magic; so you can use Ref::Store for inside out objects with no limitations, on any perl >= 5.8

For most simple applications there is no true need to have multiple dynamically associated and deleted object entries. The benefits of this module become apparent in design and ease of use when larger and more complex, event-oriented systems are in use.

In shorter terms, this module allows you to reliably use a Single Source Of Truth for your object lookups. There is no need to synchronize multiple lookup tables to ensure that there are no dangling references to an object you should have deleted

SYNOPSIS

FEATURES

One-To-Many association

It is possible, given a value, to retrieve all its keys, and vice versa. It is also possible to establish many-to-many relationships by using the same object as both a key and a value for different entries.

Key Types

This table accepts a key of any type, be it a simple string or an object reference. Keys are internally stored as object references and encapsulate the original key.

Garbage Collection (or not)

Both key and value types can be automatically selected for garbage collection, and strength relationships established between them. Thus, it is possible for a value to be automatically deleted if all its keys are deleted, and for all keys to be deleted once a value is deleted. Since both keys and values can be object references, this provides a lot of flexibility

API

LOOKUP TYPES

There are three common lookup types by which values can be indexed and mapped to.

A Lookup Type is just an identifier by which one can fetch and store a value. The uniqueness of identifiers is dependent on the lookup type. Performance for various lookup types varies.

Each lookup type has a small tag by which API functions pertaining to it can be identified

Value-specific operations

These functions take a value as their argument, and work regardless of the lookup type

purge($value)

Remove $value from the database. For all lookup types which are linked to $value, they will be removed from the database as well if they do not link to any other values

vexists($value)

Returns true if $value is stored in the database

Simple Key (SK)

This is the quickest and simplest key type. It can use either string or object keys. It support. The functions it supports are

store($key, $value, %options)

Store $value under lookup <$key>. Key can be an object reference or string.

A single value can be stored under multiple keys, but a single key can only be linked to a single value.

Options are two possible hash options:

StrongKey

If the key is an object reference, by default it will be weakened in the databse, and when the last reference outside the database is destroyed, an implicit "unlink" will be called on it. Setting StrongKey to true will disable this behavior and not weaken the key object.

A strong key is still deleted if its underlying value gets deleted

StrongValue

By default the value is weakened before it is inserted into the database, and when the last external reference is destroyed, an implicit "purge" is performed. Setting this to true will disable this behavior and not weaken the value object.

It is important to note the various rules and behaviors with key and value storage options.

There are two conditions under which an entry (key and value) may be deleted from the table. The first condition is when a key or value is a reference type, and its referrent goes out of scope; the second is when either a key or a value is explicitly deleted from the table.

It is helpful to think of entries as a miniature version of implicit reference counting. Each key represents an inherent increment in the value's reference count, and each key has a reference count of one, represented by the amount of values it actually stores.

Based on that principle, when either a key or a value is forced to leave the table (either explicitly, or because its referrant has gone out of scope), its dependent objects decrease in their table-based implicit references.

Consider the simple case of implicit deletion:

        {
                my $key = "string":
                my $value = \my $foo
                $table->store($key, $foo);
        }

In which case, the string "string" is deleted from the table as $foo goes out of scope.

The following is slightly more complex

        my $value = \my $foo;
        {
                my $key = \my $baz;
                $table->store($key, $value, StrongValue => 1);
        }

In this case, $value is removed from the table, because its key object's referrant ($baz) has gone out of scope. Even though StrongValue was specified, the value is not deleted because its own referrant ($foo) has been destroyed, but rather because its table-implicit reference count has gone down to 0 with the destruction of $baz

The following represents an inverse of the previous block

        my $key = \my $baz;
        {
                my $value = \my $foo;
                $table->store($key, $value, StrongKey => 1);
        }

Here $value is removed from the table because naturally, its referrant, $foo has been destroyed. StrongKey only maintains an extra perl reference to $baz.

However, by specifying both StrongKey and StrongValue, we are able to completely disable garbage collection, and nothing gets deleted

        {
                my $key = \my $baz;
                my $value = \my $foo;
                $table->store($key, $value, StrongKey => 1, StrongValue => 1);
        }

This method is also available as store_sk.

It is an error to call this method twice on the same lookup <-> value specification.

fetch($key)

Returns the value object indexed under $key, if any. Also available under fetch_sk

lexists($key)

Returns true if $key exists in the database. Also available as lexists_sk

unlink($key)

Removes $key from the database. If $key is linked to a value, and that value has no other keys linked to it, then the value will also be deleted from the databse. Also available as unlink_sk

        $table->store("key1", $foo);
        $table->store("key2", $foo);
        $table->store("key3", $bar);
        
        $table->unlink("key1"); # $foo is not deleted because it exists under "key2"
        $table->unlink("key3"); # $bar is deleted because it has no remaining lookups
purgeby($key)

If $key is linked to a value, then that value is removed from the database via "purge". Also available as purgeby_sk.

These two blocks are equivalent:

        # 1
        my $v = $table->fetch($k);
        $table->purge($v);
        
        # 2
        $table->purgeby($k);
Typed Keys

Typed keys are like simple keys, but with more flexibility. Whereas a simple key can only store associate any string with a specific value, typed keys allow for associating the same string key with different values, so long as the type is different. A scenario when this is useful is associating IDs received from different libraries, which may be identical, to different values.

For instance:

        use Library1;
        use Library1;
        
        my $hash = Ref::Store->new();
        $hash->register_kt('l1_key');
        $hash->register_kt('l2_key');
        
        #later on..
        my $l1_value = Library1->get_handle();
        my $l2_value = Library2->get_handle();
        
        #assume that this is possible:
        
        $l1_value->ID == $l2_value->ID();
        
        $hash->store_kt($l1_value->ID(), 'l1_key', $l1_value);
        $hash->store_kt($l2_value->ID(), 'l2_key', $l2_value);

Note that this will only actually work for string keys. Object keys can still only be unique to a single value at a time.

All functions described for "Simple Keys" are identical to those available for typed keys, except that the $key argument is transformed into two arguments;

thus:

        store_kt($key, $type, $value);
        fetch_kt($key, $type);

and so on.

In addition, there is a function which must be used to register key types:

register_kt($ktype, $id)

Register a keytype. $ktype is a constant string which is the type, and $id is a unique identifier-prefix (which defaults to $ktype itself)

Attributes

Whereas keys map value objects according to their identities, attributes map objects according to arbitrary properties or user defined tags. Hence an attribute allows for a one-to-many relationship between a lookup index and its corresponding value.

The common lookup API still applies. Attributes must be typed, and therefore all attribute functions must have a type as their second argument.

A suffix of _a is appended to all API functions. In addition, the following differences in behavior and options exist

store_a($attr, $type, $value, %options)

Like "store", but option hash takes a StrongAttr option instead of a StrongKey option, which is the same. Attributes will be weakened for all associated values if StrongAttr was not specified during any insertion operation.

fetch_a($attr, $type)

Fetch function returns an array of values, and not a single value.

thus:

        my $value = $hash->fetch($key);
        #but
        my @values = $hash->fetch_a($attr,$type);

However, storing an attribute is done only one value at a time.

dissoc_a($attr, $type, $value)

Dissociates an attribute lookup from a single value. This function is special for attributes, where a single attribute can be tied to more than a single value.

Removes the attribtue from the database. Since multiple values can be tied to the same attribute, this can potentially remove many values from the DB. Be sure to use this function with caution

It is possible to use attributes as tags for boolean values or flags, though the process right now is somewhat tedious (eventually this API will be extended to allow less boilerplate)

        use constant ATTR_FREE => "attr_free";
        use constant ATTR_BUSY => "attr_busy";
        
        $hash->register_kt(ATTR_FREE);
        $hash->register_kt(ATTR_BUSY);
        
        $hash->store_a(1, ATTR_FREE, $value); #Value is now tagged as 'free';
        
        #to mark the value as busy, be sure to inclusively mark the busy tag first,
        #and then remove the 'free' mark. otherwise the value will be seen as destroyed
        #and associated references removed:
        
        $hash->store_a(1, ATTR_BUSY, $value);
        $hash->dissoc_a(1, ATTR_FREE, $value);
        
        #mark as free again:
        
        $hash->store_a(1, ATTR_FREE, $value);
        $hash->dissoc_a(1, ATTR_BUSY, $value);

The complexities come from dealing with a triadic value for a tag. A tag for a value can either be true, false, or unset. so 0, ATTR_FREE is valid as well.

CONSTRUCTION

new(%options)

Creates a new Ref::Store object. It takes a hash of options:

keyfunc

This function is responsible for converting a key to something 'unique'. The default implementation checks to see whether the key is a reference, and if so uses its address, otherwise it uses the stringified value. It takes the user key as its argument

Ref::Store will try and select the best implementation (Ref::Store::XS and Ref::Store::PP, in that order). You can override this by seting $Ref::Store::SelectedImpl to a package of your choosing (which must be loaded).

DEBUGGING

Often it is helpful to know what the table is holding and indexing, possibly because there is a bug or because you have forgotten to delete something.

The following functions are available for debugging

vexists($value)

Returns true if $value exists in the database. The database internally maintains a hash of values. When functioning properly, a value should never exist without a key lookup, but this is still alpha software

vlookups($value)

Returns an array of stringified lookups for which this value is registered

lexists(K)

Returns true if the lookup K exists. See the "API" section for lookup-specific parameters for K

is_empty

Returns true if there are no lookups and no values in the database

dump

Prints a tree-like representation of the database. This will recurse the entire database and print information about all values and all lookup types. In addition, for object references, it will print the reference address in decimal and hexadecimal, the actual SV address of the reference, and whether the reference is a weak reference.

AUTHOR

Copyright (C) 2011 by M. Nunberg

You may use and distribute this program under the same terms as perl itself

SEE ALSO

Hash::Util::FieldHash

Ref::Store implements a superset of Hash::Util::FieldHash, but the latter is most likely quicker. However, it will only work with perls newer than 5.10

Tie::RefHash::Weak
Variable::Magic

Perl API for magic interface, used by the PP backend

KiokuDB, Tangram, Pixie