NAME

Data::Consumer::MySQL2 - Data::Consumer implementation for a mysql database table resource

VERSION

Version 0.17

SYNOPSIS

    use Data::Consumer::MySQL2;

    my $consumer = Data::Consumer::MySQL2->new(
        dbh => $dbh,
        table => 'T',
        id_field= > 'id',
        flag_field => 'done',
        lock_prefix => $worker_name,
        unprocessed => 0,
        working => 1,
        processed => 2,
        failed => 3,
    );

    $consumer->consume( sub {
        my $id = shift;
        print "processed $id\n";
    } );

FUNCTIONS

CLASS->new(%opts)

Constructor for a Data::Consumer::MySQL2 instance.

Options are as follows:

connect => \@connect_args

Will use @connect_args to connect to the database using DBI-connect()>. This argument is mandatory if the dbh argument is not provided.

dbh => $dbh

Use $dbh as the database connection object. If this argument is provided then connect will be ignored.

table => 'some_table_name'

Process records in the specified table.

id_field => 'id'

The column name of the primary key of the table being processed

flag_field => 'process_state'

The column name in the table being processed which shows whether an object is processed or not.

lock_prefix => 'my-lock-name'

The prefix to use for the mysql locks. Defaults to $0-$table.

It is strongly recommended that end-users of this module explicitly specify a lock_prefix in production environments. A multi-process system relying on mutual exclusion will run into problems when consuming from the same source if $0 and $table are not identical between workers. Generally, using the name of the consuming module should suffice (e.g. Your::Data::Consumer::Worker).

unprocessed => 0

The value of the flag_field which indicates that an item is not processed. If not provided defaults to 0.

working => 1

The value of the flag_field which indicates that an item is currently being processed. If not provided defaults to 1.

processed => 2

The value of the flag_field which indicates that an item has been successfully processed. If not provided defaults to 2.

failed => 3

The value of the flag_field which indicates that processing of an item has failed. If not provided defaults to 3.

init_id => 0

The value which the first acquired record's id_field must be greater than. Should be smaller than any legal id in the table. Defaults to 0.

select_sql

select_args

These arguments are optional, and will be synthesized from the other values if not provided.

SQL select query which can be executed to acquire an item to be processed. Should return a single record with a single column contain the id to be processed, at the same time it should ensure that a lock on the id is created.

The query will be executed with the id of the last processed item, followed by the arguments provided by the select_args property.

check_sql

check_args

These arguments are optional, unless you specify select_sql yourself, in which case it is required you also specify check_sql as well.

SQL select query which can be executed to verify that the item to be processed still has the expected flag fields set appropriately.

There is a very annoying and sublte race condition (possibly only in modern MySQL's) which means that is possible that the query used for select_sql might return an id for a record which has already been processed. This query is used to avoid that race condition.

The query should validate any flag fields or constraints specified in select_sql are true, it should return only the id of the record to be processed.

The query will be executed with the id of the item to process, followed by the arguments provided by the check_args property.

update_sql

update_args

These arguments are optional, and will be synthesized from the other values if not provided.

SQL update query which can be used to change the status the record being processed.

Will be executed with the arguments provided in update_args followed the new status, and the id.

release_sql

release_args

These arguments are optional, and will be synthesized from the other values if not provided.

SQL select query which can be used to clear the currently held lock.

Will be called with the arguments provided in release_args, plust the id.

$object->reset()

Reset the state of the object.

$object->acquire()

Aquire an item to be processed.

Returns an identifier to be used to identify the item acquired.

$object->release()

Release any locks on the currently held item.

Normally there is no need to call this directly.

$object->dbh()

returns the database handle the object is using to communicate to the db with.

AUTHOR

Yves Orton, <YVES at cpan.org>

BUGS

Please report any bugs or feature requests to bug-data-consumer at rt.cpan.org, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Data-Consumer.

I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.

ACKNOWLEDGEMENTS

Igor Sutton <IZUT@cpan.org> for ideas, testing and support.

COPYRIGHT & LICENSE

Copyright 2008, 2010, 2011 Yves Orton, all rights reserved.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

cpanm Data::Consumer

perl -MCPAN -e shell
install Data::Consumer