Frost - Feasible Referential Object STorage
package My::BaseFoo; # use Moose; use Frost; has my_base_attr => ( ... ); no Frost; __PACKAGE__->meta->make_immutable(); # mutable is VERBOTEN ################### package My::Foo; use Moose; extends 'My::BaseFoo'; use Frost::Util; # provides true() and false() aka 1 and 0 has my_attr => ( ... ); has my_index_attr => ( index => true, ... ); has my_unique_attr => ( index => 'unique', ... ); has my_transient_attr => ( transient => true, ... ); has my_virtual_attr => ( virtual => true, ... ); has my_derived_attr => ( derived => true, ... ); sub _build_my_derived_attr { ... } no Moose; __PACKAGE__->meta->make_immutable(); # mutable is VERBOTEN ################### package My::Bar::AutoId; use Moose; extends 'My::Foo'; use Frost::Util; # provides true() and false() aka 1 and 0 has id => ( auto_id => true ); # UUID no Moose; __PACKAGE__->meta->make_immutable(); ################### package My::Bar::AutoInc; use Moose; extends 'My::Foo'; use Frost::Util; # provides true() and false() aka 1 and 0 has id => ( auto_inc => true ); # 1, 2, 3, 4, .... no Moose; __PACKAGE__->meta->make_immutable(); ################### package My::Loop; use Moose; extends 'My::BaseFoo'; has next => ( isa => 'My::Loop', ... ) # weak ref is VERBOTEN has last => ( isa => 'My::Loop', ... ) # weak ref is VERBOTEN no Moose; __PACKAGE__->meta->make_immutable(); ################### # application.cgi # use Frost::Asylum; use My::Foo; use My::Bar::AutoId; use My::Bar::AutoInc; my $asylum = Frost::Asylum->new ( data_root => '/existing/path/for/my/data' ); my ( $id_auto_id, $id_auto_inc ); # Create { my $foo = My::Foo->new ( id => 42, asylum => $asylum, my_attr => ... ); my $bar_id = My::Bar::AutoId->new ( asylum => $asylum, my_attr => ... ); my $bar_inc = My::Bar::AutoInc->new ( asylum => $asylum, my_attr => ... ); $id_auto_id = $bar_id->id; # something like ECDAEFD4-9247-11DE-9343-7794CBAD412D $id_auto_inc = $bar_inc->id; # if empty 1, otherwise last used id + 1 } # Circular references { my $loop1 = My::Loop->new ( id => 'L1', asylum => $asylum ); my $loop2 = My::Loop->new ( id => 'L2', asylum => $asylum, last => $loop1 ); my $loop3 = My::Loop->new ( id => 'L3', asylum => $asylum, last => $loop2 ); $loop1->last ( $loop3 ); $loop1->next ( $loop2 ); $loop2->next ( $loop3 ); $loop3->next ( $loop1 ); } $asylum->close; # and auto-save # Evoke { my $foo = My::Foo->new ( id => 42, asylum => $asylum ); my $bar_id = My::Bar::AutoId->new ( id => $id_auto_id, asylum => $asylum ); my $bar_inc = My::Bar::AutoInc->new ( id => $id_auto_inc, asylum => $asylum ); my $loop2 = My::Loop->new ( id => 'L2', asylum => $asylum ); ( $loop2->last->id eq 'L1' ) or die; ( $loop2->next->id eq 'L3' ) or die; my $loop1 = $loop2->next->next; }
Frost is an extension of Moose providing persistence even to circular structures.
Once upon a time, when there were still real winters with frosty air, an old moose named Elly Elk walked through the snow-white forest and murmured over and over "How can I become persistent?", "I want to live forever!" and more and more of that agonizing stuff.
Suddenly she ran into Mona Moose, her old friend. Elly noticed, that Mona was looking younger than ever, fresh and clean, although Mona was as old as Elly. "How comes, that you're looking so fine?" Elly said and Mona replied, that she would share her secret with Elly: "Come on, join me at the Asylum."
Said and done. The Asylum was a warm and cozy place and the warden was a friendly man, although - with his peaked hat - he looked more like a wizard. The room was dimly lit. Elly recognized many of her old friends hanging out in the Twilight.
"Yes, you can live forever as all the other moose, just sign here with blood." the warden said.
Elly Elk shivered but Mona Moose encouraged her: "This is a mere formality."
What the warden didn't tell, that he was a Necromancer and that behind the Asylum was a big Cemetery, where his assistants - the Morticians - did their clandestine work. But he knew, that - although he was practicing only white magic - the truth would have frighten his guests.
"Trust me - I know what I'm doing!" he said and continued: "Look at Mona and all your friends."
So Elly Elk signed the registration form and the Necromancer blessed her with an exclusive mantra and said: "With this mantra you'll be unique in the world. As long as it's a known value, you'll live for ever. But now take a sleep - you're looking tired."
The Necromancer conducted Elly to his studio, where she was silenced. And as she passed away, he removed her spirit and put it in the Twilight zone.
As he came back, Mona wanted to see Elly again. "That's easy!" answered the Necromancer and with a little hex he evoked Elly and turns her from a spirit to - no, not to the original Elly - but to a confusingly similar Locum.
"Welcome back, Elly!" Mona said, "You're looking so good now." And really, Elly was feeling young and strong. Mona, Elly and all their friends had much fun all day.
In the evening the warden started to silence everybody - um, every Locum - with the magic words "We've done a full day's work and it's time for bed."
When the Asylum was closed, the Necromancer absolved all spirits in the Twilight and tasked the Morticians to extract the essence of every spirit and bury it in the Cemetery while collecting every spirit's mantra in a huge book called Illuminator.
The next morning, when the Asylum was opened again. the Morticians were instructed by the Necromancer to grub all essences from the Cemetery. His secret assistents did as told: After consulting the Illuminator, they aspirated every mantra to conjure up the spirit from the essence. Then the Necromancer evoked every spirit and so the lounge filled up with Locums again.
So it went day in, day out. Elly and Mona and their friends were persistent now and they all lived happily ever after.
As Matt Sergeant pointed out in his article at http://www.perl.com/pub/a/2002/08/07/proxyobject.html using referential structures is problematic, because it leads often to memory leakage.
Weak references for object relations like Tree->parent and Tree->children or Loop->next and Loop->last are no solution, because at time the root object is leaving the scope, all references are lost also.
Using a proxy object as described in the article is bad too, because deploying a tree constructed with a weak proxy will fail, if you hold onto a sub-tree while letting the root of the tree go out of scope. All of a sudden you'll find your data structure has become untangled, like so much wool from a snagged jumper. (Matt Sergeant)
An AUTOLOAD solution would foil Moose' fast immutable mode. Moreover you always have to be sure, which attribute has to be weak or not - an endless source of bugs.
But we need referential structures - imagine a repertoire management system for a theatre:
We have productions (plays) bound to events (stage performances), while any event points to a production and furthermore to a location, each connected in turn to many productions. Moreover we have authors, directors, technicians, actors etc. attached to events, locations and productions and so on.
Loops, loops, loops...
The Moose example in t/020_attributes/028_no_slot_access.t gave us the right direction.
But just backending all attributes of a complete object - as in the example - wouldn't provide us with a solution for circular references. So why not deliver always a small proxy object (the Locum), which holds no real attributes in memory except 'id' (the mantra) and 'asylum' (the storage object Asylum) for speed. Every other attribute is kept in a cache (the Twilight) as a unique hash structure (the spirit), which in turn is tied to a Berkeley DB backend (the Cemetery) managed by the Necromancer and his assistants, the Morticians.
Frost's meta voodoo provides us with a lightweight proxy mechanism without AUTOLOAD tricks: Any other object is not stored in the owner object's attribute as a real reference, but only as a special pointer to the other already cached hash (aka spirit). Every time, the referenced object is accessed, the factory (Asylum) delivers a new proxy object (Locum). This way we'll never create hard circular references, which can't be garbage collected.
The program:
# ===> next # ---> last # # +---------+ +---------+ # | |====>| | # | |<----| | # | Loop(1) | | Loop(2) | # | |---->| | # | |<====| | # +---------+ +---------+ { package Loop; use Frost; use Frost::Util; # provides true() and false() aka 1 and 0 has content => ( is => 'rw', isa => 'Str' ); has 'last' => ( is => 'rw', isa => 'Loop', weak_ref => false, # weak refs are VERBOTEN ); has 'next' => ( is => 'rw', isa => 'Loop', weak_ref => false, # weak refs are VERBOTEN ); sub add_next { my ( $self, $next ) = @_; $next->last ( $self ); $self->next ( $next ); } no Frost; __PACKAGE__->meta->make_immutable(); } use Frost::Asylum; my $asylum = Frost::Asylum->new ( data_root => '/existing/path/for/my/data' ); my $loop1 = Loop->new ( asylum => $asylum, id => 1, content => 'This is Loop 1' ); my $loop2 = Loop->new ( asylum => $asylum, id => 2, content => 'This is Loop 2' ); $loop1->add_next ( $loop2 ); $loop2->add_next ( $loop1 );
gives us the structures (memory addresses simplified):
Loop=HASH(100) ( = $loop1 ) { id => 1, asylum => Asylum=HASH(666), _status => "missing", } Loop=HASH(200) ( = $loop2 ) { id => 2, asylum => Asylum=HASH(666), _status => "missing", } Asylum=HASH(666) { twilight => Twilight=HASH(42) { 'Loop|1' => { id => 1, _dirty => false, content => 'This is Loop 1', next => { type => 'Loop', ref => 2 }, last => { type => 'Loop', ref => 2 } }, 'Loop|2' => { id => 2, _dirty => true, content => 'This is Loop 2', next => { type => 'Loop', ref => 1 }, last => { type => 'Loop', ref => 1 } }, } }
Loop 1 in the Twilight is NOT dirty as one may expect, because it was saved automatically during silencing due to the referential structure.
Somewhere else in the program:
{ my $loop1n2 = $loop1->next(); my $loop1l2 = $loop1->last(); my $loop2n1 = $loop2->next(); my $loop2l1 = $loop2->last(); }
gives us the structures:
Loop=HASH(300) ( = $loop1n2 ) { id => 2, asylum => Asylum=HASH(666), _status => "exists", } Loop=HASH(400) ( = $loop1l2 ) { id => 2, asylum => Asylum=HASH(666), _status => "exists", } Loop=HASH(500) ( = $loop2n1 ) { id => 1, asylum => Asylum=HASH(666), _status => "exists", } Loop=HASH(600) ( = $loop2l1 ) { id => 1, asylum => Asylum=HASH(666), _status => "exists", }
The Twilight has not changed:
Asylum=HASH(666) { twilight => Twilight=HASH(42) { 'Loop|1' => { id => 1, _dirty => false, content => 'This is Loop 1', next => { type => 'Loop', ref => 2 }, last => { type => 'Loop', ref => 2 } }, 'Loop|2' => { id => 2, _dirty => true, content => 'This is Loop 2', next => { type => 'Loop', ref => 1 }, last => { type => 'Loop', ref => 1 } }, } }
But we got new instances:
Loop=HASH(300) != Loop=HASH(200) Loop=HASH(400) != Loop=HASH(200) Loop=HASH(500) != Loop=HASH(100) Loop=HASH(600) != Loop=HASH(100)
Continuing the program:
my $loop3 = Loop->new ( asylum => $asylum, id => 3, content => 'This is Loop 3' ); $loop1->add_next ( $loop3 ); $loop3->add_next ( $loop2 );
Loop=HASH(333) ( = $loop3 ) { id => 3, asylum => Asylum=HASH(666), _status => "missing", } Asylum=HASH(666) { twilight => Twilight=HASH(42) { 'Loop|1' => { id => 1, _dirty => true, content => 'This is Loop 1', next => { type => 'Loop', ref => 3 }, last => { type => 'Loop', ref => 2 } }, 'Loop|2' => { id => 2, _dirty => false, content => 'This is Loop 2', next => { type => 'Loop', ref => 1 }, last => { type => 'Loop', ref => 3 } }, 'Loop|3' => { id => 3, _dirty => true, content => 'This is Loop 3', next => { type => 'Loop', ref => 2 }, last => { type => 'Loop', ref => 1 } }, } }
Now Loop 2 in the Twilight is NOT dirty, but the others. Which Locum was automatically saved depends on the order of entering the referential structure - adding to $loop3 first i.e.:
$loop3->add_next ( $loop2 ); $loop1->add_next ( $loop3 );
would yield:
Asylum=HASH(666) { twilight => Twilight=HASH(42) { 'Loop|1' => { id => 1, _dirty => true, content => 'This is Loop 1', next => { type => 'Loop', ref => 3 }, last => { type => 'Loop', ref => 2 } }, 'Loop|2' => { id => 2, _dirty => false, content => 'This is Loop 2', next => { type => 'Loop', ref => 1 }, last => { type => 'Loop', ref => 3 } }, 'Loop|3' => { id => 3, _dirty => false, content => 'This is Loop 3', next => { type => 'Loop', ref => 2 }, last => { type => 'Loop', ref => 1 } }, } }
But this doesn't matter, as long as you always close the Asylum. This will auto-save all remaining dirty Loops:
$asylum->close();
What the user wanted:
# ===> next # ---> last # # +---------+ +---------+ +---------+ # | |====>| |====>| | # | |<----| |<----| | # | Loop(1) | | Loop(3) | | Loop(2) | # | | | | | | # | | | | | | # | | | | | | # | | | | | | # | | +---------+ | | # | |-------------------->| | # | |<====================| | # +---------+ +---------+
could easily be reloaded - in another script the next day just say:
my $loop3 = Loop->new ( id => 3, asylum => $asylum ); print $loop3->content; # 'This is Loop 3' print $loop3->last->content; # 'This is Loop 1' print $loop3->next->content; # 'This is Loop 2' print $loop3->next->next->content; # 'This is Loop 1' print $loop3->last->next->content; # 'This is Loop 3'
Frost comes with six new features extending Moose::Meta::Attribute.
They provide some sugar to deal with the location of an attribute, to save some typing and to create indices.
So migrating from a SQL database to an object-oriented storage is possible without loosing one of the main features of a relational database: indexing.
has foo => ( transient => true, ... );
A transient attribute lives at run-time and is "local" :
transient
It becomes undef, when the Locum object goes out of scope, and it's not saved.
has bar => ( virtual => true, ... );
A virtual attribute lives at run-time and is "global" :
virtual
It's still present, when the Locum object goes out of scope, but it's not saved.
has dat => ( derived => true, ... );
A derived attribute lives at run-time and is "global" :
derived
# The definition: has dat => ( derived => true, isa => 'Str' ); # is a shortcut for: has dat => ( virtual => true, isa => 'Str', is => 'ro', init_arg => undef, lazy_build => true ); # which becomes: has dat => ( virtual => true, # from derived is => 'ro', # init_arg => undef, # isa => 'Str', lazy => true, # from lazy_build builder => '_build_dat', # clearer => 'clear_dat', # predicate => 'has_dat', # ); # and requires: sub _build_dat { ... }
package Foo; ... has foo => ( index => true, ... ); has bar => ( index => 'unique', ... );
This creates a multiple or unique index (Frost::Illuminator) of the attribute values. Numeric sorting is automatically determined from the attribute's isa.
isa
Thus, if you create objects with following attributes:
{ id => 'id1', foo => 'foo1', bar => 'bar1' } { id => 'id2', foo => 'foo2', bar => 'bar2' } { id => 'id3', foo => 'foo3', bar => 'bar3' } { id => 'id4', foo => 'foo3', bar => 'bar2' } { id => 'id5', foo => 'foo1', bar => 'bar3' }
you'll get two indices:
foo1 -> id1 foo1 -> id5 foo2 -> id2 foo3 -> id3 foo3 -> id4 bar1 -> id1 id2 silently eaten... id3 silently eaten... bar2 -> id4 bar3 -> id5
Later you can search in the unique index:
my $id = $asylum->find ( 'Foo', 'bar3', 'bar' ); # class, key, attribute_name # $id is 'id5' my $foo = Foo->new ( id => $id, asylum => $asylum );
or iterate over the multiple index:
my $id = $asylum->first ( 'Foo', 'foo1', 'foo' ); # class, key, attribute_name while ( $id ) # 1st loop: $id is 'id1' # 2nd loop: $id is 'id5' # 3rd loop: $id is '' { my $foo = Foo->new ( id => $id, asylum => $asylum ); $id = $asylum->next ( 'Foo', 'foo1', 'foo' ); }
and much more - see Frost::Asylum.
has id => ( auto_id => true, ... ); has +id => ( auto_id => true, ... );
If the basic attribute id is sweetened with auto_id, the instance is automatically assigned a UUID - see Data::UUID.
auto_id
The resulting id is something like ECDAEFD4-9247-11DE-9343-7794CBAD412D.
ECDAEFD4-9247-11DE-9343-7794CBAD412D
has id => ( auto_inc => true, ... ); has +id => ( auto_inc => true, ... );
If the basic attribute id is sweetened with auto_inc, the instance is automatically assigned a sequential number: highest used id + 1, where highest used id = 0 in an empty cemetery.
auto_inc
See Frost::Meta::Attribute
Frost can only store Moose objects, more precisely: objects inheriting from Frost::Locum.
This does not mean, that each of your old classes must now use Frost instead of Moose. Migrating is easily possible by creating or changing a base class from use Moose to use Frost - i.e.
use Moose
use Frost
package My::BaseFoo; # use Moose; use Frost; no Frost; __PACKAGE__->meta->make_immutable(); # mutable is VERBOTEN package My::Foo; use Moose; extends 'My::BaseFoo'; has my_attr => ( ... ); no Moose; __PACKAGE__->meta->make_immutable(); # mutable is VERBOTEN
Thus My::Foo and all inheriting classes can be defined the usual Moose way. The class My::BaseFoo inherits from Frost::Locum, pulling in all its persistence stuff.
While applying Moose' tests for Frost it appeared, that reblessing an object - i.e. because of applying a role at run-time - creates mutable objects blessed in an anonymous class. This destroys the silence/evoke mechanism, which depends on the real classname.
So reblessing as well as make_mutable is VERBOTEN.
Due to Frost's proxy algorithm there is no need for weak references. See "MOTIVATION".
To create a new frostable object you must always say:
my $foo = Foo->new ( id => 'a_unique_id', asylum => $asylum, an_attr => ..., another_attr => ... ); $remembered_id = $foo->id;
And to reload a frosted object just say:
my $foo = Foo->new ( id => $remembered_id, asylum => $asylum ); # other params ignored
Or use the Asylum API - a matter of taste:
my $remembered_id = 'a_unique_id'; my $foo = $asylum->silence ( 'Foo', $remembered_id, an_attr => ..., another_attr => ... ); # later... my $foo = $asylum->evoke ( 'Foo', $remembered_id ); # other params ignored
If the id is defined with auto_id or auto_inc, Asylum is creating a mantra for you - i.e.:
id
package Bar; ... has id ( auto_id => 1 ); ... # Create and silence without id: # my $bar = Bar->new ( asylum => $asylum, an_attr => ..., another_attr => ... ); my $bar_id = $bar->id; # something like ECDAEFD4-9247-11DE-9343-7794CBAD412D # Evoke with id: # my $bar = Bar->new ( id => $bar_id, asylum => $asylum );
Docs, docs, docs...
I'm reading the Moose mailing list frequently, so please ask your questions there.
The mailing list is moose@perl.org. You must be subscribed to send a message. To subscribe, send an empty message to moose-subscribe@perl.org
This module requires these other modules and libraries:
BerkeleyDB Carp Class::MOP Data::Dumper Data::UUID Exporter Fcntl File::Path IO::File Log::Log4perl Moose Scalar::Util Storable Tie::Cache::LRU Time::HiRes
For testing these modules are needed:
DateTime::Format::MySQL List::Util List::MoreUtils Test::Deep Test::Differences Test::Exception Test::More Test::Requires
Optional modules to run some tests:
Devel::Size Sys::MemInfo Declare::Constraints::Simple Regexp::Common Locale::US
Please see Makefile.PL for required versions.
All complex software has bugs lurking in it, and this module is no exception.
Please report any bugs to me or the mailing list.
Ernesto ernesto@dienstleistung-kultur.de
Copyright (C) 2010 by Dienstleistung Kultur Ltd. & Co. KG
http://dienstleistung-kultur.de/frost/
This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
To install Frost, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Frost
CPAN shell
perl -MCPAN -e shell install Frost
For more information on module installation, please visit the detailed CPAN module installation guide.