NAME

Frost - Feasible Referential Object STorage

SYNOPSIS

   package My::BaseFoo;
   # use Moose;
   use Frost;

   has my_base_attr => ( ... );

   no Frost;
   __PACKAGE__->meta->make_immutable();   #  mutable is VERBOTEN

   ###################

   package My::Foo;
   use Moose;
   extends 'My::BaseFoo';

   use Frost::Util;   # provides true() and false() aka 1 and 0

   has my_attr           => ( ... );
   has my_index_attr     => ( index     => true, ... );
   has my_unique_attr    => ( index     => 'unique', ... );
   has my_transient_attr => ( transient => true, ... );
   has my_virtual_attr   => ( virtual   => true, ... );
   has my_derived_attr   => ( derived   => true, ... );

   sub _build_my_derived_attr { ... }

   no Moose;
   __PACKAGE__->meta->make_immutable();   #  mutable is VERBOTEN

   ###################

   package My::Bar::AutoId;
   use Moose;
   extends 'My::Foo';

   use Frost::Util;   # provides true() and false() aka 1 and 0

   has id => ( auto_id => true );   # UUID

   no Moose;
   __PACKAGE__->meta->make_immutable();

   ###################

   package My::Bar::AutoInc;
   use Moose;
   extends 'My::Foo';

   use Frost::Util;   # provides true() and false() aka 1 and 0

   has id => ( auto_inc => true );  # 1, 2, 3, 4, ....

   no Moose;
   __PACKAGE__->meta->make_immutable();

   ###################

   package My::Loop;
   use Moose;
   extends 'My::BaseFoo';

   has next => ( isa => 'My::Loop', ... )   #  weak ref is VERBOTEN
   has last => ( isa => 'My::Loop', ... )   #  weak ref is VERBOTEN

   no Moose;
   __PACKAGE__->meta->make_immutable();

   ###################

   #  application.cgi
   #
   use Frost::Asylum;
   use My::Foo;
   use My::Bar::AutoId;
   use My::Bar::AutoInc;

   my $asylum   = Frost::Asylum->new ( data_root => '/existing/path/for/my/data' );

   my ( $id_auto_id, $id_auto_inc );

   #   Create
   {
      my $foo      = My::Foo->new          ( id => 42, asylum => $asylum, my_attr => ... );
      my $bar_id   = My::Bar::AutoId->new  (           asylum => $asylum, my_attr => ... );
      my $bar_inc  = My::Bar::AutoInc->new (           asylum => $asylum, my_attr => ... );

      $id_auto_id  = $bar_id->id;    # something like ECDAEFD4-9247-11DE-9343-7794CBAD412D
      $id_auto_inc = $bar_inc->id;   # if empty 1, otherwise last used id + 1
   }

   #   Circular references
   {
      my $loop1 = My::Loop->new ( id => 'L1', asylum => $asylum );
      my $loop2 = My::Loop->new ( id => 'L2', asylum => $asylum, last => $loop1 );
      my $loop3 = My::Loop->new ( id => 'L3', asylum => $asylum, last => $loop2  );

      $loop1->last ( $loop3 );

      $loop1->next ( $loop2 );
      $loop2->next ( $loop3 );
      $loop3->next ( $loop1 );
   }

   $asylum->close;   # and auto-save

   #   Evoke

   {
      my $foo      = My::Foo->new          ( id => 42,           asylum => $asylum );
      my $bar_id   = My::Bar::AutoId->new  ( id => $id_auto_id,  asylum => $asylum );
      my $bar_inc  = My::Bar::AutoInc->new ( id => $id_auto_inc, asylum => $asylum );

      my $loop2    = My::Loop->new ( id => 'L2', asylum => $asylum );

      ( $loop2->last->id eq 'L1' ) or die;
      ( $loop2->next->id eq 'L3' ) or die;

      my $loop1 = $loop2->next->next;
   }

ABSTRACT

Frost is an extension of Moose providing persistence even to circular structures.

DESCRIPTION

Once upon a time, when there were still real winters with frosty air, an old moose named Elly Elk walked through the snow-white forest and murmured over and over "How can I become persistent?", "I want to live forever!" and more and more of that agonizing stuff.

Suddenly she ran into Mona Moose, her old friend. Elly noticed, that Mona was looking younger than ever, fresh and clean, although Mona was as old as Elly. "How comes, that you're looking so fine?" Elly said and Mona replied, that she would share her secret with Elly: "Come on, join me at the Asylum."

Said and done. The Asylum was a warm and cozy place and the warden was a friendly man, although - with his peaked hat - he looked more like a wizard. The room was dimly lit. Elly recognized many of her old friends hanging out in the Twilight.

"Yes, you can live forever as all the other moose, just sign here with blood." the warden said.

Elly Elk shivered but Mona Moose encouraged her: "This is a mere formality."

What the warden didn't tell, that he was a Necromancer and that behind the Asylum was a big Cemetery, where his assistants - the Morticians - did their clandestine work. But he knew, that - although he was practicing only white magic - the truth would have frighten his guests.

"Trust me - I know what I'm doing!" he said and continued: "Look at Mona and all your friends."

So Elly Elk signed the registration form and the Necromancer blessed her with an exclusive mantra and said: "With this mantra you'll be unique in the world. As long as it's a known value, you'll live for ever. But now take a sleep - you're looking tired."

The Necromancer conducted Elly to his studio, where she was silenced. And as she passed away, he removed her spirit and put it in the Twilight zone.

As he came back, Mona wanted to see Elly again. "That's easy!" answered the Necromancer and with a little hex he evoked Elly and turns her from a spirit to - no, not to the original Elly - but to a confusingly similar Locum.

"Welcome back, Elly!" Mona said, "You're looking so good now." And really, Elly was feeling young and strong. Mona, Elly and all their friends had much fun all day.

In the evening the warden started to silence everybody - um, every Locum - with the magic words "We've done a full day's work and it's time for bed."

When the Asylum was closed, the Necromancer absolved all spirits in the Twilight and tasked the Morticians to extract the essence of every spirit and bury it in the Cemetery while collecting every spirit's mantra in a huge book called Illuminator.

The next morning, when the Asylum was opened again. the Morticians were instructed by the Necromancer to grub all essences from the Cemetery. His secret assistents did as told: After consulting the Illuminator, they aspirated every mantra to conjure up the spirit from the essence. Then the Necromancer evoked every spirit and so the lounge filled up with Locums again.

So it went day in, day out. Elly and Mona and their friends were persistent now and they all lived happily ever after.

MOTIVATION

Persistent storage even for circular structures like trees or loops is feasible

As Matt Sergeant pointed out in his article at http://www.perl.com/pub/a/2002/08/07/proxyobject.html using referential structures is problematic, because it leads often to memory leakage.

Weak references for object relations like Tree->parent and Tree->children or Loop->next and Loop->last are no solution, because at time the root object is leaving the scope, all references are lost also.

Using a proxy object as described in the article is bad too, because deploying a tree constructed with a weak proxy will fail, if you hold onto a sub-tree while letting the root of the tree go out of scope. All of a sudden you'll find your data structure has become untangled, like so much wool from a snagged jumper. (Matt Sergeant)

An AUTOLOAD solution would foil Moose' fast immutable mode. Moreover you always have to be sure, which attribute has to be weak or not - an endless source of bugs.

But we need referential structures - imagine a repertoire management system for a theatre:

We have productions (plays) bound to events (stage performances), while any event points to a production and furthermore to a location, each connected in turn to many productions. Moreover we have authors, directors, technicians, actors etc. attached to events, locations and productions and so on.

Loops, loops, loops...

The Moose example in t/020_attributes/028_no_slot_access.t gave us the right direction.

But just backending all attributes of a complete object - as in the example - wouldn't provide us with a solution for circular references. So why not deliver always a small proxy object (the Locum), which holds no real attributes in memory except 'id' (the mantra) and 'asylum' (the storage object Asylum) for speed. Every other attribute is kept in a cache (the Twilight) as a unique hash structure (the spirit), which in turn is tied to a Berkeley DB backend (the Cemetery) managed by the Necromancer and his assistants, the Morticians.

Frost's meta voodoo provides us with a lightweight proxy mechanism without AUTOLOAD tricks: Any other object is not stored in the owner object's attribute as a real reference, but only as a special pointer to the other already cached hash (aka spirit). Every time, the referenced object is accessed, the factory (Asylum) delivers a new proxy object (Locum). This way we'll never create hard circular references, which can't be garbage collected.

IMPLEMENTATION

The underlying magic disclosed

The program:

   #   ===>   next
   #   --->   last
   #
   #   +---------+     +---------+
   #   |         |====>|         |
   #   |         |<----|         |
   #   | Loop(1) |     | Loop(2) |
   #   |         |---->|         |
   #   |         |<====|         |
   #   +---------+     +---------+

   {
      package Loop;
      use Frost;
      use Frost::Util;   # provides true() and false() aka 1 and 0

      has content => ( is => 'rw', isa => 'Str' );

      has 'last' =>
      (
         is       => 'rw',
         isa      => 'Loop',
         weak_ref => false,      #   weak refs are VERBOTEN
      );

      has 'next' =>
      (
         is       => 'rw',
         isa      => 'Loop',
         weak_ref => false,      #   weak refs are VERBOTEN
      );

      sub add_next
      {
         my ( $self, $next )  = @_;

         $next->last ( $self );
         $self->next ( $next );
      }

      no Frost;

      __PACKAGE__->meta->make_immutable();
   }

   use Frost::Asylum;

   my $asylum  = Frost::Asylum->new ( data_root => '/existing/path/for/my/data' );

   my $loop1   = Loop->new ( asylum => $asylum, id => 1, content => 'This is Loop 1' );
   my $loop2   = Loop->new ( asylum => $asylum, id => 2, content => 'This is Loop 2' );

   $loop1->add_next ( $loop2 );
   $loop2->add_next ( $loop1 );

gives us the structures (memory addresses simplified):

   Loop=HASH(100)         ( = $loop1 )
   {
      id       => 1,
      asylum   => Asylum=HASH(666),
      _status  => "missing",
   }

   Loop=HASH(200)         ( = $loop2 )
   {
      id       => 2,
      asylum   => Asylum=HASH(666),
      _status  => "missing",
   }

   Asylum=HASH(666)
   {
      twilight => Twilight=HASH(42)
      {
         'Loop|1' => { id => 1, _dirty => false,
                       content => 'This is Loop 1',
                       next => { type => 'Loop', ref => 2 }, last => { type => 'Loop', ref => 2 } },
         'Loop|2' => { id => 2, _dirty => true,
                       content => 'This is Loop 2',
                       next => { type => 'Loop', ref => 1 }, last => { type => 'Loop', ref => 1 } },
      }
   }

Loop 1 in the Twilight is NOT dirty as one may expect, because it was saved automatically during silencing due to the referential structure.

Somewhere else in the program:

   {
      my $loop1n2 = $loop1->next();
      my $loop1l2 = $loop1->last();

      my $loop2n1 = $loop2->next();
      my $loop2l1 = $loop2->last();
   }

gives us the structures:

   Loop=HASH(300)         ( = $loop1n2 )
   {
      id       => 2,
      asylum   => Asylum=HASH(666),
      _status  => "exists",
   }

   Loop=HASH(400)         ( = $loop1l2 )
   {
      id       => 2,
      asylum   => Asylum=HASH(666),
      _status  => "exists",
   }

   Loop=HASH(500)         ( = $loop2n1 )
   {
      id       => 1,
      asylum   => Asylum=HASH(666),
      _status  => "exists",
   }

   Loop=HASH(600)         ( = $loop2l1 )
   {
      id       => 1,
      asylum   => Asylum=HASH(666),
      _status  => "exists",
   }

The Twilight has not changed:

   Asylum=HASH(666)
   {
      twilight => Twilight=HASH(42)
      {
         'Loop|1' => { id => 1, _dirty => false,
                       content => 'This is Loop 1',
                       next => { type => 'Loop', ref => 2 }, last => { type => 'Loop', ref => 2 } },
         'Loop|2' => { id => 2, _dirty => true,
                       content => 'This is Loop 2',
                       next => { type => 'Loop', ref => 1 }, last => { type => 'Loop', ref => 1 } },
      }
   }

But we got new instances:

   Loop=HASH(300) != Loop=HASH(200)
   Loop=HASH(400) != Loop=HASH(200)
   Loop=HASH(500) != Loop=HASH(100)
   Loop=HASH(600) != Loop=HASH(100)

Continuing the program:

   my $loop3   = Loop->new ( asylum => $asylum, id => 3, content => 'This is Loop 3' );

   $loop1->add_next ( $loop3 );
   $loop3->add_next ( $loop2 );

gives us the structures:

   Loop=HASH(333)         ( = $loop3 )
   {
      id       => 3,
      asylum   => Asylum=HASH(666),
      _status  => "missing",
   }

   Asylum=HASH(666)
   {
      twilight => Twilight=HASH(42)
      {
         'Loop|1' => { id => 1, _dirty => true,
                       content => 'This is Loop 1',
                       next => { type => 'Loop', ref => 3 }, last => { type => 'Loop', ref => 2 } },
         'Loop|2' => { id => 2, _dirty => false,
                       content => 'This is Loop 2',
                       next => { type => 'Loop', ref => 1 }, last => { type => 'Loop', ref => 3 } },
         'Loop|3' => { id => 3, _dirty => true,
                       content => 'This is Loop 3',
                       next => { type => 'Loop', ref => 2 }, last => { type => 'Loop', ref => 1 } },
      }
   }

Now Loop 2 in the Twilight is NOT dirty, but the others. Which Locum was automatically saved depends on the order of entering the referential structure - adding to $loop3 first i.e.:

   $loop3->add_next ( $loop2 );
   $loop1->add_next ( $loop3 );

would yield:

   Asylum=HASH(666)
   {
      twilight => Twilight=HASH(42)
      {
         'Loop|1' => { id => 1, _dirty => true,
                       content => 'This is Loop 1',
                       next => { type => 'Loop', ref => 3 }, last => { type => 'Loop', ref => 2 } },
         'Loop|2' => { id => 2, _dirty => false,
                       content => 'This is Loop 2',
                       next => { type => 'Loop', ref => 1 }, last => { type => 'Loop', ref => 3 } },
         'Loop|3' => { id => 3, _dirty => false,
                       content => 'This is Loop 3',
                       next => { type => 'Loop', ref => 2 }, last => { type => 'Loop', ref => 1 } },
      }
   }

But this doesn't matter, as long as you always close the Asylum. This will auto-save all remaining dirty Loops:

   $asylum->close();

What the user wanted:

   #   ===>   next
   #   --->   last
   #
   #   +---------+     +---------+     +---------+
   #   |         |====>|         |====>|         |
   #   |         |<----|         |<----|         |
   #   | Loop(1) |     | Loop(3) |     | Loop(2) |
   #   |         |     |         |     |         |
   #   |         |     |         |     |         |
   #   |         |     |         |     |         |
   #   |         |     |         |     |         |
   #   |         |     +---------+     |         |
   #   |         |-------------------->|         |
   #   |         |<====================|         |
   #   +---------+                     +---------+

could easily be reloaded - in another script the next day just say:

   my $loop3 = Loop->new ( id => 3, asylum => $asylum );

   print $loop3->content;               #   'This is Loop 3'

   print $loop3->last->content;         #   'This is Loop 1'
   print $loop3->next->content;         #   'This is Loop 2'
   print $loop3->next->next->content;   #   'This is Loop 1'
   print $loop3->last->next->content;   #   'This is Loop 3'

My friend Jack eats sugar lumps - more fun with attributes

Frost comes with six new features extending Moose::Meta::Attribute.

They provide some sugar to deal with the location of an attribute, to save some typing and to create indices.

So migrating from a SQL database to an object-oriented storage is possible without loosing one of the main features of a relational database: indexing.

transient
   has foo => ( transient => true, ... );

A transient attribute lives at run-time and is "local" :

It becomes undef, when the Locum object goes out of scope, and it's not saved.

virtual
   has bar => ( virtual => true, ... );

A virtual attribute lives at run-time and is "global" :

It's still present, when the Locum object goes out of scope, but it's not saved.

derived
   has dat => ( derived => true, ... );

A derived attribute lives at run-time and is "global" :

It's still present, when the Locum object goes out of scope, but it's not saved.

   # The definition:

   has dat => ( derived => true, isa => 'Str' );

   # is a shortcut for:

   has dat => ( virtual => true, isa => 'Str', is => 'ro', init_arg => undef, lazy_build => true );

   # which becomes:

   has dat =>
   (
      virtual   => true,          # from derived
      is        => 'ro',          #
      init_arg  => undef,         #
      isa       => 'Str',
      lazy      => true,          # from lazy_build
      builder   => '_build_dat',  #
      clearer   => 'clear_dat',   #
      predicate => 'has_dat',     #
   );

   # and requires:

   sub _build_dat { ... }
index
   package Foo;
   ...
   has foo => ( index => true, ... );
   has bar => ( index => 'unique', ... );

This creates a multiple or unique index (Frost::Illuminator) of the attribute values. Numeric sorting is automatically determined from the attribute's isa.

Thus, if you create objects with following attributes:

   { id => 'id1', foo => 'foo1', bar => 'bar1' }
   { id => 'id2', foo => 'foo2', bar => 'bar2' }
   { id => 'id3', foo => 'foo3', bar => 'bar3' }
   { id => 'id4', foo => 'foo3', bar => 'bar2' }
   { id => 'id5', foo => 'foo1', bar => 'bar3' }

you'll get two indices:

   foo1 -> id1
   foo1 -> id5
   foo2 -> id2
   foo3 -> id3
   foo3 -> id4

   bar1 -> id1
           id2 silently eaten...
           id3 silently eaten...
   bar2 -> id4
   bar3 -> id5

Later you can search in the unique index:

   my $id  = $asylum->find ( 'Foo', 'bar3', 'bar' );   # class, key, attribute_name

   #  $id is 'id5'

   my $foo = Foo->new ( id => $id, asylum => $asylum );

or iterate over the multiple index:

   my $id  = $asylum->first ( 'Foo', 'foo1', 'foo' );  # class, key, attribute_name

   while ( $id )
   #  1st loop: $id is 'id1'
   #  2nd loop: $id is 'id5'
   #  3rd loop: $id is ''
   {
      my $foo = Foo->new ( id => $id, asylum => $asylum );

      $id   = $asylum->next ( 'Foo', 'foo1', 'foo' );
   }

and much more - see Frost::Asylum.

auto_id
   has id  => ( auto_id => true, ... );
   has +id => ( auto_id => true, ... );

If the basic attribute id is sweetened with auto_id, the instance is automatically assigned a UUID - see Data::UUID.

The resulting id is something like ECDAEFD4-9247-11DE-9343-7794CBAD412D.

auto_inc
   has id  => ( auto_inc => true, ... );
   has +id => ( auto_inc => true, ... );

If the basic attribute id is sweetened with auto_inc, the instance is automatically assigned a sequential number: highest used id + 1, where highest used id = 0 in an empty cemetery.

See Frost::Meta::Attribute

The Four Commandments of Frost

Thou shalt absolve no other Moose before Locum

Frost can only store Moose objects, more precisely: objects inheriting from Frost::Locum.

This does not mean, that each of your old classes must now use Frost instead of Moose. Migrating is easily possible by creating or changing a base class from use Moose to use Frost - i.e.

   package My::BaseFoo;
   # use Moose;
   use Frost;

   no Frost;
   __PACKAGE__->meta->make_immutable();   #  mutable is VERBOTEN

   package My::Foo;
   use Moose;
   extends 'My::BaseFoo';

   has my_attr      => ( ... );

   no Moose;
   __PACKAGE__->meta->make_immutable();   #  mutable is VERBOTEN

Thus My::Foo and all inheriting classes can be defined the usual Moose way. The class My::BaseFoo inherits from Frost::Locum, pulling in all its persistence stuff.

Thou shalt honor immutability

While applying Moose' tests for Frost it appeared, that reblessing an object - i.e. because of applying a role at run-time - creates mutable objects blessed in an anonymous class. This destroys the silence/evoke mechanism, which depends on the real classname.

So reblessing as well as make_mutable is VERBOTEN.

Thou shalt not weaken thy reference

Due to Frost's proxy algorithm there is no need for weak references. See "MOTIVATION".

Thou shalt not create or load objects without an id and thy asylum

To create a new frostable object you must always say:

   my $foo = Foo->new ( id => 'a_unique_id', asylum => $asylum, an_attr => ..., another_attr => ... );

   $remembered_id   = $foo->id;

And to reload a frosted object just say:

   my $foo = Foo->new ( id => $remembered_id, asylum => $asylum );   #   other params ignored

Or use the Asylum API - a matter of taste:

   my $remembered_id   = 'a_unique_id';

   my $foo = $asylum->silence ( 'Foo', $remembered_id, an_attr => ..., another_attr => ... );

   # later...

   my $foo = $asylum->evoke ( 'Foo', $remembered_id );   #   other params ignored

If the id is defined with auto_id or auto_inc, Asylum is creating a mantra for you - i.e.:

   package Bar;
   ...
   has id ( auto_id => 1 );
   ...

   # Create and silence without id:
   #
   my $bar    = Bar->new ( asylum => $asylum, an_attr => ..., another_attr => ... );

   my $bar_id = $bar->id;   # something like ECDAEFD4-9247-11DE-9343-7794CBAD412D

   # Evoke with id:
   #
   my $bar    = Bar->new ( id => $bar_id, asylum => $asylum );

TODO

Docs, docs, docs...

GETTING HELP

I'm reading the Moose mailing list frequently, so please ask your questions there.

The mailing list is moose@perl.org. You must be subscribed to send a message. To subscribe, send an empty message to moose-subscribe@perl.org

DEPENDENCIES

This module requires these other modules and libraries:

        BerkeleyDB
        Carp
        Class::MOP
        Data::Dumper
        Data::UUID
        Exporter
        Fcntl
        File::Path
        IO::File
        Log::Log4perl
        Moose
        Scalar::Util
        Storable
        Tie::Cache::LRU
        Time::HiRes

For testing these modules are needed:

        DateTime::Format::MySQL
        List::Util
        List::MoreUtils
        Test::Deep
        Test::Differences
        Test::Exception
        Test::More
        Test::Requires

Optional modules to run some tests:

        Devel::Size
        Sys::MemInfo
        Declare::Constraints::Simple
        Regexp::Common
        Locale::US

Please see Makefile.PL for required versions.

BUGS

All complex software has bugs lurking in it, and this module is no exception.

Please report any bugs to me or the mailing list.

AUTHOR

Ernesto ernesto@dienstleistung-kultur.de

COPYRIGHT AND LICENSE

Copyright (C) 2010 by Dienstleistung Kultur Ltd. & Co. KG

http://dienstleistung-kultur.de/frost/

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.