The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.
 / 
EntityModel-0.102
River stage one • 3 direct dependents • 7 total dependents
++
/ EntityModel

NAME

EntityModel - manage entity model definitions

VERSION

version 0.102

SYNOPSIS

 use EntityModel;
 # Define model
 my $model = EntityModel->new->load_from(
        JSON => { entity : [
                { name : 'article', field : [
                        { name : 'idarticle', type : 'bigserial' },
                        { name : 'title', type : 'varchar' },
                        { name : 'content', type : 'text' }
                ], primary => { field : [ 'idarticle' ], separator : ':' }
         ] }
 );
 # Apply PostgreSQL schema (optional, only needed if the model changes)
 $model->apply('PostgreSQL' => { schema => 'datamodel', host => 'localhost', user => 'testuser' });
 # Create Perl classes
 $model->apply('Perl' => { namespace => 'Entity', baseclass => 'EntityModel::EntityBase' });

 my $article = Entity::Article->create(
        title => 'Test article',
        content => 'Article content'
 )->done(sub {
        my $article = shift;
        say "ID was " . $article->id;
 })->fail(sub {
        die 'Failed to create new article';
 });
 Entity::Article->find(
        title => 'Test article'
 )->first(sub {
        my $match = shift;
        $match->title('Revised title');
        die "Instances of the same object should always be linked, consistent and up-to-date"
                unless $article->title eq $match->title;
 });

DESCRIPTION

This module provides a data storage abstraction system (in the form of an Object Relational Model) for accessing backend storage from Perl and other languages. The intent is to take a model definition and generate or update database tables, caching layer and the corresponding code (Perl/C++/JS) for accessing data.

A brief comparison and list of alternatives is in the "MOTIVATION" and "SEE ALSO" sections, please check there before investing any time into using this module.

Eventually a full set of documentation will be added to http://entitymodel.com/documentation.html but for now see the examples further down in this document.

METHODS

new

Constructor. Given a set of options, will load any plugins specified (and/or the defaults), applying other config options via the appropriate plugins.

Typically run without options:

 my $model = EntityModel->new;

The exciting things happen elsewhere. See:

load_from

Read in a model definition from the given EntityModel::Definition-based source.

Parameters:

  • Type - must be a valid EntityModel::Definition subclass, such as 'Perl', 'JSON' or 'XML'.

  • Definition - dependent on the subclass, typically the filename or raw string data.

Common usage includes reading from inline Perl:

 $model->load_from(
  Perl => {
   name => 'kvstore',
   entity => [
    name => 'object',
    primary => 'iditem',
    field => [
     { name => 'iditem', type => 'bigserial' },
     { name => 'key', type => 'varchar' },
     { name => 'value', type => 'varchar' },
    ],
   ],
  }
 );

or the equivalent from JSON:

 $model->load_from(
  JSON => \q{
   "name" : "kvstore",
   "entity" : [
    "name" : "object",
    "primary" : "iditem",
    "field" : [
     { "name" : "iditem", "type" : "bigserial" },
     { "name" : "key", "type" : "varchar" },
     { "name" : "value", "type" : "varchar" }
    ]
   ]
  }
 );

save_to

Saves the current model definition to a definition.

Parameters:

  • Type - must be a valid EntityModel::Definition subclass, such as 'Perl', 'JSON' or 'XML'.

  • Definition - dependent on the subclass, typically the filename or scalarref to hold raw string data.

You might use something like this to store the current model to a file in JSON format:

 $model->save_to(
  JSON => 'model.json'
 );

or this to copy everything from a source model to a target model (wiping everything in the target in the process):

 my $target = EntityModel->new;
 $source->save_to(
  model => $target
 );

load_component

Brings in the given component if it hasn't already been loaded.

Typically used by internal methods only.

add_support

Bring in a new EntityModel::Support class for this EntityModel::Model.

Example:

 $model->add_support(Perl => { namespace => 'Entity' });

add_storage

Add backend storage provided by an EntityModel::Storage subclass.

Example:

 $model->add_storage(PostgreSQL => { service => ... });

backend_ready

Returns true if all storage and cache backends are ready, false otherwise.

wait_for_backend

Requests an event to run after all backends signal readiness.

add_cache

Add backend cache provided by an EntityModel::Storage subclass.

Example:

 $model->add_cache(PostgreSQL => { service => ... });

add_plugin

Adds a plugin. Currently the definition of a 'plugin' is somewhat nebulous, but EntityModel::Web is one example.

transaction

Run the coderef in a transaction.

Notifies all the attached EntityModel::Storage instances that we want a transaction, runs the code, then signals end-of-transaction.

load_plugin

Used internally, see "add_plugin". Will disappear in the future.

handler_for

Returns the handler for a given entry in the EntityModel::Definition.

DESTROY

Unload all plugins on exit.

ENTITY MODELS

A model contains metadata and zero or more entities.

Each entity typically represents something that is able to be instantiated as an object, such as a row in a table. Since this module is heavily biased towards SQL-style applications, most of the entity definition is similar to SQL-92, with some additional features suitable for ORM-style access via other languages (Perl, JS, C++).

An entity definition primarily contains the following information - see EntityModel::Entity for more details:

  • name - the name of the entity, must be unique in the model

  • type - typically 'table'

  • description - a more detailed description of the entity and purpose

  • primary - information about the primary key

Each entity may have zero or more fields:

  • name - the unique name for this field

  • type - standard SQL type for the field

  • null - true if this can be null

  • reference - foreign key information

Additional metadata can be defined for entities and fields. Indexes apply at an entity level. They are used in table construction and updates to ensure that common queries can be optimised, and are also checked in query validation to highlight potential performance issues.

  • name - unique name for the index

  • type - index type, typically one of 'gin', 'gist', or 'btree'

  • fields - list of fields that are indexed

The fields in an index can be defined as functions.

Constraints include attributes such as unique column values.

Models can also contain additional information as defined by plugins - see EntityModel::Plugin for more details on this.

USAGE

An entity model can be loaded from several sources. If you have a database definition:

 create table test ( id int, name varchar(255), url text );

then loading the SQL plugin with the database name will create a single entity holding two fields.

If you also load EntityModel::Plugin::Apply::Perl, you can access this table as follows:

 my $tbl = Entity::Test->create({ name => 'Test', url => '/there' })->commit;
 my ($entity) = Entity::Test->find({ name => 'Test' });
 is($orig->id, $entity->id, 'found same id');

IMPLEMENTATION

Nearly all classes use EntityModel::Class to provide basic structure including accessors and helper functions and methods. This also enables strict, warnings and Perl 5.10 features.

Logging is handled through EntityModel::Log, which imports functions such as logDebug.

Arrays and hashes are typically wrapped using EntityModel::Array and EntityModel::Hash respectively, similar in concept to autobox.

For error handling, an EntityModel::Error object is returned - this allows chained method calling without having to wrap in eval or check the result of each step when you don't care about failure. The last method in the chain will return false in boolean context.

OVERVIEW

The current EntityModel::Model can be read from a number of sources:

  • EntityModel::Definition::XML - XML structured definition holding the entities, fields and any additional plugin-specific data. All information is held in the content - no attributes are used, allowing this format to be interchangeable with JSON and internal Perl datastructures.

  • EntityModel::Definition::JSON - standard Javascript Object Notation format.

  • EntityModel::Definition::Perl - nested Perl datastructures, with the top level being a hashref. Note that this is distinct from the Perl class/entity structure described later.

Aside from entities, models can also contain plugin-specific information such as site definition or database schema. It is also possible - but not recommended - to store credentials such as database user and password.

Once a model definition has been loaded, it can be applied to one or more of the following:

The SQL handling is provided as a generic DBI-compatible layer with additional support in subclasses for specific databases. Again, Note that the EntityModel::Support::SQL is intended for applying the model to the database schema, rather than accessing the backend storage. The EntityModel::Support classes apply the model to the API, so in the case of the database this involves creating and updating tables. For Perl, this dynamically creates a class structure in memory, and for C++ or JS this will export the required support code for inclusion in other projects.

In terms of accessing backend storage, each of the language-specific support options provides an API which can communicate with one or more backend storage implementations, rather than being tightly coupled to a data storage method. Typically the Perl backend would interact directly with the database, and C++/JS would use a REST API against a Perl server.

BACKEND STORAGE

Backend storage services are provided by subclasses of EntityModel::Storage.

CACHING

Cache layers are handled by EntityModel::Cache subclasses.

USAGE EXAMPLE

Given a simple JSON model definition:

 { entity : [
        { name : 'article', field : [
                { name : 'idarticle', type : 'bigserial' },
                { name : 'title', type : 'varchar' },
                { name : 'content', type : 'text' }
        ], primary => { field : [ 'idarticle' ], separator : ':' }
 ] }

this would create or alter the article table to meet this definition:

 create table "article" (
        idarticle bigserial,
        title varchar,
        content text,
        primary key (idarticle)
 )

Enabling the Perl plugin would grant access via Perl code:

 my $article = Entity::Article->create(title => 'Test article', content => 'Article content')
 say "ID was " . $article->id;
 my ($match) = Entity::Article->find(title => 'Test article');
 $match->title('Revised title');
 die "Instances of the same object should always be linked, consistent and up-to-date"
        unless $article->title eq $match->title;

with the equivalent through Javascript being:

 var article = Entity.Article.create({ title : 'Test article', 'content' : 'Article content' });
 alert("ID was " + article.id());
 var match = Entity.Article.find({ title : 'Test article' })[0];
 match.title('Revised title');
 if(article.title() != match.title())
        alert("Instances of the same object should always be linked, consistent and up-to-date");

or in C++:

 Entity::Article article = new Entity::Article().title('Test article').content('Article content');
 std::cout << "ID was a.id() << std::endl;
 Entity::Article *match = Entity::Article::find().title('Test article').begin();
 match->title('Revised title');
 if(article->title() != match->title())
         throw new std::string("Instances of the same object should always be linked, consistent and up-to-date");

The actual backend implementation may vary between these, but the intention is to maintain a recognisable, autogenerated API across all supported languages. The C++ implementation may inherit from a class that writes directly to the database, for example, and the Javascript code could be designed to run in a web browser accessing the resources through HTTP or as a node.js implementation linked directly to the database, but the top-level code should not need to care which underlying storage method is being used.

ASYNCHRONOUS MODEL ACCESS

Since backend storage response times can vary, it may help to use an asynchronous API for accessing entities. Given the 'article' example from earlier, the Perl code is now:

 my $article = Entity::Article->create(
        title => 'Test article',
        content => 'Article content'
 )->done(sub {
        my $a = shift;
        say "ID was " . $a->id;
 })->fail(sub {
        die 'Failed to create the requested article :(';
 });
 Entity::Article->find(title => 'Test article')->each(sub {
        my $match = shift;
        $match->title('Revised title');
        die "Instances of the same object should always be linked, consistent and up-to-date"
                unless $article->title eq $match->title;
 })->none(sub {
        die 'Nothing found';
 });

EXPORTING MODEL DEFINITIONS

Although it is possible to reverse-engineer the model in some cases, such as SQL, normally this is not advised. This may be useful however for a one-off database structure import, by writing the results to a model config file in JSON or XML format:

 my $model = EntityModel::Plugin::Apply::SQL->loadFrom(
        db => $db,
        schema => $schema
 );
 $model->export(xml => 'model.xml');

Once the model has been exported any further updates should be done in the model definition file rather than directly to the database if possible, since this would allow the generation of suitable upgrade/downgrade scripts.

Currently there is support for SQL and Perl model export, but not for Javascript or C++.

AUDITING

Audit tables are generated by default in the _audit schema, following the same naming convention as the audited tables with the addition of the following columns:

  • audit_action - one of:

    • Insert - regular insert or mass import (such as PostgreSQL COPY statement)

    • Update - updates directly through database or through the EntityModel API

    • Delete - manual or API removal

    and indicates the action that generated this audit entry.

  • audit_date - timestamp of this action

  • audit_context - description of the action, typically the command that was called

CLASS STRUCTURE

The primary classes used for interaction with models include:

The following classes provide features that are used throughout the code:

See http://entitymodel.com/documentation.html for more details and diagrams.

MOTIVATION

Some of the primary motivations for this distribution over any of the existing approaches (see next section for some alternatives):

  • Event-based operation - backend storage requests run 'asynchronously' where possible, allowing the application to continue with other tasks while waiting for database queries to complete.

  • Support for languages other than Perl - most projects end up using at least one other language, e.g. web-based projects typically have some Javascript on the frontend talking to the Perl backend.

  • Configurable with a single file - I like to be able to export, backup and diff the entity layout and having this in a single file as JSON or XML is more convenient for me than using multiple Perl packages. This also allows the configuration to be used by non-Perl code, since it's a rare project these days that only involves a single language.

  • Backend abstraction - the conceptual model generally doesn't need to be tied to a particular backend, for example the Perl side of things could either talk directly to a database or use a webservice.

  • Easy editing and visualisation - I like to have the option of using other tools such as diagram editors to add/modify the entity layout, rather than having to create or edit Perl files. This helps to separate actual code changes from configuration / layout issues; the EntityModel distribution can be installed once and for many common applications no further custom code should be required.

  • Flexibility - although too much interdependence is generally a bad thing, being able to include other concepts in the configuration file has been useful for tasks such as building websites with common features.

Clearly none of these features are necessarily unique to EntityModel, and many of the alternative systems described in the next section could be adapted to support the above requirements.

SEE ALSO

There are plenty of other ORM implementations available on CPAN, one of which may be more suited to your needs than this is. These are the ones I've found so far:

Asynchronous ORMs

The list here is sadly lacking:

Synchronous ORMs

If you're happy for the database to tie up your process for an indefinite amount of time, you're in luck - there's a nice long list of modules to choose from here:

  • DBIx::Class - appears to be the most highly regarded and actively developed one out there, and the available features, code quality and general stability are far in advance of this module. Unless you need the EntityModel multi-language or asynchronous support features I would strongly encourage looking at DBIx::Class first

  • Rose::DB::Object - written for speed, appears to cover most of the usual requirements, personally found the API less intuitive than other options but it appears to be widely deployed

  • Fey::ORM - newer than the other options, also appears to be reasonably flexible

  • DBIx::DataModel - UML-based Object-Relational Mapping (ORM) framework

  • Alzabo - another ORM which includes features such as GUI schema editing and SQL diff

  • Class::DBI - generally considered to be superceded by DBIx::Class, which provides a compatibility layer for existing applications

  • Class::DBI::Lite - like Class::DBI but lighter, presumably

  • ORMesque - lightweight class-based ORM using SQL::Abstract

  • Oryx - Object persistence framework, meta-model based with support for both DBM and regular RDBMS backends, uses tied hashes and arrays

  • Tangram - An object persistence layer

  • KiokuDB - described as an "Object Graph storage engine" rather than an ORM

  • DBIx::DataModel - ORM using UML definitions

  • Jifty::DBI - another ORM

  • ORLite - minimal SQLite-based ORM

  • Ormlette - object persistence, "heavily influenced by Adam Kennedy's ORLite". "light and fluffy", apparently!

  • ObjectDB - another lightweight ORM, currently has only DBI as a dependency

  • ORM - looks like it has support for MySQL, PostgreSQL and SQLite

  • fytwORM - described as a "bare minimum ORM used for prototyping / proof of concepts"

  • DBR - Database Repository ORM

  • SweetPea::Application::Orm - specific to the SweetPea web framework

  • Jorge - ORM Made simple

  • Persistence::ORM - looks like a combination between persistent Perl objects and standard ORM

  • Teng - lightweight minimal ORM

  • Class::orMapper - DBI-based "easy O/R Mapper"

  • UR - class framework and object/relational mapper (ORM) for Perl

  • DBIx::NinjaORM - "Flexible Perl ORM for easy transitions from inline SQL to objects"

  • DBIx::Oro - Simple Relational Database Accessor

  • LittleORM - Moose-based ORM

  • Storm - another Moose-based ORM

  • DBIx::Mint - "A mostly class-based ORM for Perl"

Database interaction

Since this is Perl, there are probably many more, if you have something which isn't in the above list (or a better description of any of the existing entries), please raise via RT or email.

Distributions which provide class structure and wrappers around the Perl OO mechanism are likewise covered by several other CPAN modules, with the clear winner here in the forms of Moose, Moo and derivatives.

Eventually I'll try to put up a better set of comparisons on http://entitymodel.com.

INHERITED METHODS

EntityModel::Model

add_entity, add_field_to_table, add_table, apply, apply_fields, commit, commit_pending_add, commit_pending_remove, commit_pending_update, create_entity, create_table, dump, entity_by_name, flush, handle_item, hasPending, load_model, matches, new_entity, pending, pending_entities, provide_handler_for, read_tables, remove_entity, remove_table, resolve_entity_dependencies, rollback, table, update_from, update_table

Mixin::Event::Dispatch

add_handler_for_event, clear_event_handlers, event_handlers, invoke_event, subscribe_to_event, unsubscribe_from_event

EntityModel::BaseClass

clone, sap

AUTHOR

Tom Molesworth <cpan@entitymodel.com>

LICENSE

Copyright Tom Molesworth 2008-2013. Licensed under the same terms as Perl itself.