NAME

Redis::CappedCollection - Provides fixed size (determined by 'maxmemory' Redis server setting) collections with FIFO data removal.

VERSION

This documentation refers to Redis::CappedCollection version 1.10

SYNOPSIS

use 5.010;
use strict;
use warnings;

#-- Common
use Redis::CappedCollection qw(
    $DEFAULT_SERVER
    $DEFAULT_PORT
);

my $server = $DEFAULT_SERVER.':'.$DEFAULT_PORT;
my $coll = Redis::CappedCollection->create( redis => { server => $server } );

# Insert new data into collection
my $list_id = $coll->insert( 'Some List_id', 'Some Data_id', 'Some data' );

# Change the element of the list with the ID $list_id
$updated = $coll->update( $list_id, $data_id, 'New data' );

# Get data from a list with the ID $list_id
@data = $coll->receive( $list_id );
# or to obtain the data ordered from the oldest to the newest
while ( my ( $list_id, $data ) = $coll->pop_oldest ) {
    say "List '$list_id' had '$data'";
}

A brief example of the Redis::CappedCollection usage is provided in "An Example" section.

The data structures used by Redis::CappedCollection on Redis server are explained in "CappedCollection data structure" section.

ABSTRACT

Redis::CappedCollection module provides fixed sized collections that have a auto-FIFO age-out feature.

The collection consists of multiple lists containing data items ordered by time. Each list must have an unique ID within the collection and each data item has unique ID within its list.

Automatic data removal (when size limit is reached) may remove the oldest item from any list.

Collection size is determined by 'maxmemory' Redis server setting.

DESCRIPTION

Main features of the package are:

Support creation of capped collection, status monitoring, updating the data set, obtaining consistent data from the collection, automatic data removal, error reporting.
Simple API for inserting and retrieving data and for managing collection.

Capped collections are fixed-size collections that have an auto-FIFO age-out feature based on the time of the inserted data. When collection size reaches memory limit, the oldest data elements are removed automatically to provide space for the new elements.

The lists in capped collection store their data items ordered by item time.

To insert a new data item into the capped collection, provide list ID, data ID, data and optional data time (current time is used if not specified). If there is a list with the given ID, the data is inserted into the existing list, otherwise the new list is created automatically.

You may update the existing data in the collection, providing list ID, data ID and optional data time. If no time is specified, the updated data will keep its existing time.

Once the space is fully utilized, newly added data will replace the oldest data in the collection.

Limits are specified when the collection is created. Collection size is determined by 'maxmemory' redis server settings.

The package includes the utilities to dump and restore the collection: dump_collection, restore_collection .

EXPORT

None by default.

Additional constants are available for import, which can be used to define some type of parameters.

These are the defaults:

$DEFAULT_SERVER

Default Redis local server: 'localhost'.

$DEFAULT_PORT

Default Redis server port: 6379.

$DEFAULT_CONNECTION_TIMEOUT

Default socket timeout for connection, number of seconds: 0.1 .

$DEFAULT_OPERATION_TIMEOUT

Default socket timeout for read and write operations, number of seconds: 1.

$NAMESPACE

Namespace name used keys on the Redis server: 'C'.

$MIN_MEMORY_RESERVE, $MAX_MEMORY_RESERVE

Minimum and maximum memory reserve limits based on 'maxmemory' configuration of the Redis server.

Not used when 'maxmemory' = 0 (not set in the redis.conf).

The following values are used by default:

$MIN_MEMORY_RESERVE = 0.05; # 5%
$MAX_MEMORY_RESERVE = 0.5;  # 50%

$DEFAULT_CLEANUP_ITEMS

Number of additional elements to delete from collection during cleanup procedure when collection size exceeds 'maxmemory'.

Default 100 elements. 0 means no minimal cleanup required, so memory cleanup will be performed only to free up sufficient amount of memory.

$DATA_VERSION

Current data structure version - 3.

Error codes: More details about error codes are provided in "DIAGNOSTICS" section.

Possible error codes:

$E_NO_ERROR

-1000 - No error

$E_MISMATCH_ARG

-1001 - Invalid argument.

Thrown by methods when there is a missing required argument or argument value is invalid.

$E_DATA_TOO_LARGE

-1002 - Data is too large.

$E_NETWORK

-1003 - Error in connection to Redis server.

$E_MAXMEMORY_LIMIT

-1004 - Command not allowed when used memory > 'maxmemory'.

This means that the command is not allowed when used memory > maxmemory in the redis.conf file.

$E_MAXMEMORY_POLICY

-1005 - Redis server have incompatible maxmemory-policy setting.

Thrown when Redis server have incompatible maxmemory-policy setting in redis.conf.

$E_COLLECTION_DELETED

-1006 - Collection elements was removed prior to use.

This means that the system part of the collection was removed prior to use.

$E_REDIS

-1007 - Redis error message.

This means that other Redis error message detected.

$E_DATA_ID_EXISTS

-1008 - Attempt to add data with an existing ID

This means that you are trying to insert data with an ID that is already in the data list.

$E_OLDER_THAN_ALLOWED

-1009 - Attempt to add outdated data

This means that you are trying to insert the data with the time older than the time of the oldest element currently stored in collection.

$E_NONEXISTENT_DATA_ID

-1010 - Attempt to access the elements missing in the collection.

This means that you are trying to update data which does not exist.

$E_INCOMP_DATA_VERSION

-1011 - Attempt to access the collection with incompatible data structure, created by an older or newer version of this module.

$E_REDIS_DID_NOT_RETURN_DATA

-1012 - The Redis server did not return data.

Check the settings in the file redis.conf.

$E_UNKNOWN_ERROR

-1013 - Unknown error.

Possibly you should modify the constructor parameters for more intense automatic memory release.

CONSTRUCTOR

create

create( redis => $server, name => $name, ... )

Create a new collection on the Redis server and return an Redis::CappedCollection object to access it. Must be called as a class method only.

The create creates and returns a Redis::CappedCollection object that is configured to work with the default settings if the corresponding arguments were not given.

redis argument can be either an existing object of Redis class (which is then used for all communication with Redis server) or a hash reference used to create a new internal Redis object. See documentation of Redis module for details.

create takes arguments in key-value pairs.

This example illustrates a create() call with all the valid arguments:

my $coll = Redis::CappedCollection->create(
    redis           => { server => "$server:$port" },   # Redis object
                    # or hash reference to parameters to create a new Redis object.
    name            => 'Some name', # Redis::CappedCollection collection name.
    cleanup_bytes   => 50_000,  # The minimum size, in bytes,
                    # of the data to be released when performing memory cleanup.
    cleanup_items   => 1_000,   # The minimum number of the collection
                    # elements to be realesed when performing memory cleanup.
    max_list_items  => 0, # Maximum list items limit
    max_datasize    => 1_000_000,   # Maximum size, in bytes, of the data.
                    # Default 512MB.
    older_allowed   => 0, # Allow adding an element to collection that's older
                    # than the last element removed from collection.
                    # Default 0.
    check_maxmemory => 1, # Controls if collection should try to find out maximum
                    # available memory from Redis.
                    # In some cases Redis implementation forbids such request,
                    # but setting 'check_maxmemory' to false can be used
                    # as a workaround.
    memory_reserve  => 0.05,    # Reserve coefficient of 'maxmemory'.
                    # Not used when 'maxmemory' == 0 (it is not set in the redis.conf).
                    # When you add or modify the data trying to ensure
                    # reserve of free memory for metadata and bookkeeping.
    reconnect_on_error  => 1,   # Controls ability to force re-connection with Redis on error.
                    # Boolean argument - default is true and conservative_reconnect is true.
    connection_timeout  => $DEFAULT_CONNECTION_TIMEOUT,    # Socket timeout for connection,
                    # number of seconds (can be fractional).
                    # NOTE: Changes external socket configuration.
    operation_timeout   => $DEFAULT_OPERATION_TIMEOUT,     # Socket timeout for read and write operations,
                    # number of seconds (can be fractional).
                    # NOTE: Changes external socket configuration.
);

The redis and name arguments are required. Do not use the symbol ':' in name.

The following examples illustrate other uses of the create method:

my $redis = Redis->new( server => "$server:$port" );
my $coll = Redis::CappedCollection->create( redis => $redis, name => 'Next collection' );
my $next_coll = Redis::CappedCollection->create( redis => $coll, name => 'Some name' );

An error exception is thrown (croak) if an argument is not valid or the collection with same name already exists.

open

open( redis => $server, name => $name, ... )

Example:

my $redis = Redis->new( server => "$server:$port" );
my $coll = Redis::CappedCollection::open( redis => $redis, name => 'Some name' );

Create a Redis::CappedCollection object to work with an existing collection (created by "create"). It must be called as a class method only.

open takes optional arguments. These arguments are in key-value pairs. Arguments description is the same as for "create" method.

redis
name
max_datasize
check_maxmemory
reconnect_on_error
connection_timeout
operation_timeout

The redis and name arguments are mandatory.

The open creates and returns a Redis::CappedCollection object that is configured to work with the default settings if the corresponding arguments are not given.

If redis argument is not a Redis object, a new connection to Redis is established using passed hash reference to create a new Redis object.

An error exception is thrown (croak) if an argument is not valid.

METHODS

An exception is thrown (croak) if any method argument is not valid or if a required argument is missing.

ATTENTION: In the Redis module the synchronous commands throw an exception on receipt of an error reply, or return a non-error reply directly.

name

Get collection name attribute (collection ID). The method returns the current value of the attribute. The name attribute value is used in the constructor.

redis

Existing Redis object or a hash reference with parameters to create a new one.

reconnect_on_error

Controls ability to force re-connection with Redis on error.

connection_timeout

Controls socket timeout for Redis server connection, number of seconds (can be fractional).

NOTE: Changes external socket configuration.

operation_timeout

Controls socket timeout for Redis server read and write operations, number of seconds (can be fractional).

NOTE: Changes external socket configuration.

cleanup_bytes

Accessor for cleanup_bytes attribute - The minimum size, in bytes, of the data to be released when performing memory cleanup. Default 0.

The cleanup_bytes attribute is designed to reduce the release of memory operations with frequent data changes.

The cleanup_bytes attribute value can be provided to "create". The method returns and sets the current value of the attribute.

The cleanup_bytes value must be less than or equal to 'maxmemory'. Otherwise an error exception is thrown (croak).

cleanup_items

The minimum number of the collection elements to be realesed when performing memory cleanup. Default 100.

The cleanup_items attribute is designed to reduce number of times collection cleanup takes place. Setting value too high may result in unwanted delays during operations with Redis.

The cleanup_items attribute value can be used in the constructor. The method returns and sets the current value of the attribute.

max_list_items

Maximum list items limit.

Default 0 means that number of list items not limited.

The max_list_items attribute value can be used in the constructor. The method returns and sets the current value of the attribute.

max_datasize

Accessor for the max_datasize attribute.

The method returns the current value of the attribute if called without arguments.

Non-negative integer value can be used to specify a new value to the maximum size of the data introduced into the collection (methods "insert" and "update").

The max_datasize attribute value is used in the constructor and operations data entry on the Redis server.

The constructor uses the smaller of the values of 512MB and 'maxmemory' limit from a redis.conf file.

older_allowed

Accessor for the older_allowed attribute which controls if adding an element that is older than the last element removed from collection is allowed. Default is 0 (not allowed).

The method returns the current value of the attribute. The older_allowed attribute value is used in the constructor.

memory_reserve

Accessor for the memory_reserve attribute which specifies the amount of additional memory reserved for metadata and bookkeeping. Default 0.05 (5%) of 'maxmemory'. Not used when 'maxmemory' == 0 (it is not set in the redis.conf).

Valid values must be between $MIN_MEMORY_RESERVE and $MAX_MEMORY_RESERVE.

The method returns the current value of the attribute. The memory_reserve attribute value is used in the constructor.

last_errorcode

Get code of the last error.

See the list of supported error codes in "DIAGNOSTICS" section.

insert

insert( $list_id, $data_id, $data, $data_time )

Example:

$list_id = $coll->insert( 'Some List_id', 'Some Data_id', 'Some data' );

$list_id = $coll->insert( 'Another List_id', 'Data ID', 'More data', Time::HiRes::time() );

Insert data into the capped collection on the Redis server.

Arguments:

$list_id

Mandatory, non-empty string: list ID. Must not contain ':'.

The data will be inserted into the list with given ID, and the list is created automatically if it does not exist yet.

$data_id

Mandatory, non-empty string: data ID, unique within the list identified by $list_id argument.

$data

Data value: a string. Data length should not exceed value of "max_datasize" attribute.

$data_time

Optional data time, a non-negative number. If not specified, the current value returned by time() is used instead. Floating values (such as those returned by Time::HiRes module) are supported to have time granularity of less than 1 second and stored with 4 decimal places.

If collection is set to older_allowed == 1 and $data_time less than time of the last removed element (last_removed_time - see collection_info) then last_removed_time is set to 0. The "older_allowed" attribute value is used in the constructor.

The method returns the ID of the data list to which the data was inserted (value of the $list_id argument).

update

update( $list_id, $data_id, $data, $new_data_time )

Example:

if ( $coll->update( $list_id, $data_id, 'New data' ) ) {
    say "Data updated successfully";
} else {
    say "The data is not updated";
}

Updates existing data item.

Arguments:

$list_id: Mandatory, non-empty string: list ID. Must not contain ':'.
$data_id: Mandatory, non-empty string: data ID, unique within the list identified by $list_id argument.
$data: New data value: a string. Data length should not exceed value of "max_datasize" attribute.
$new_data_time: Optional new data time, a non-negative number. If not specified, the existing data time is preserved.

If the collection is set to older_allowed == 1 and $new_data_time less than time of the last removed element (last_removed_time - see "collection_info") then last_removed_time is set to 0. The "older_allowed" attribute value is used in the constructor.

Method returns true if the data is updated or false if the list with the given ID does not exist or is used an invalid data ID.

Throws an exception on other errors.

upsert

upsert( $list_id, $data_id, $data, $data_time )

Example:

$list_id = $coll->upsert( 'Some List_id', 'Some Data_id', 'Some data' );

$list_id = $coll->upsert( 'Another List_id', 'Data ID', 'More data', Time::HiRes::time() );

If the list $list_id does not contain data with $data_id, then it behaves like an "insert", otherwise behaves like an "update".

The method returns the ID of the data list to which the data was inserted (value of the $list_id argument) as the "insert" method.

receive

receive( $list_id, $data_id )

Example:

my @data = $coll->receive( $list_id );
say "List '$list_id' has '$_'" foreach @data;
# or
my $list_len = $coll->receive( $list_id );
say "List '$list_id' has '$list_len' item(s)";
# or
my $data = $coll->receive( $list_id, $data_id );
say "List '$list_id' has '$data_id'" if defined $data;

If the $data_id argument is not specified or is an empty string:

In a list context, the method returns all the data from the list given by the $list_id identifier.

Method returns an empty list if the list with the given ID does not exist.
In a scalar context, the method returns the length of the data list given by the $list_id identifier.

If the $data_id argument is specified:

The method returns the specified element of the data list. If the data with $data_id ID does not exist, undef is returned.

pop_oldest

The method retrieves the oldest data stored in the collection and removes it from the collection.

Returns a list of two elements. The first element contains the identifier of the list from which the data was retrieved. The second element contains the extracted data.

The returned data item is removed from the collection.

Method returns an empty list if the collection does not contain any data.

The following examples illustrate uses of the pop_oldest method:

while ( my ( $list_id, $data ) = $coll->pop_oldest ) {
    say "List '$list_id' had '$data'";
}

redis_config_ok

redis_config_ok( redis => $server )

Example:

say 'Redis server config ', $coll->redis_config_ok ? 'OK' : 'NOT OK';
my $redis = Redis->new( server => "$server:$port" );
say 'Redis server config ',
    Redis::CappedCollection::redis_config_ok( redis => $redis )
        ? 'OK'
        : 'NOT OK'
;

Check whether there is a Redis server config correct, now that the 'maxmemory-policy' setting is 'noeviction'. Returns true if config correct and false otherwise.

It can be called as either the existing Redis::CappedCollection object method or a class function.

If invoked as the object method, redis_config_ok uses the redis attribute from the object as default.

If invoked as the class function, redis_config_ok requires mandatory redis argument.

This argument are in key-value pair as described for "create" method.

An error exception is thrown (croak) if an argument is not valid.

collection_info

collection_info( redis => $server, name => $name )

Example:

my $info = $coll->collection_info;
say 'An existing collection uses ', $info->{cleanup_bytes}, " byte of 'cleanup_bytes', ",
    $info->{items}, ' items are stored in ', $info->{lists}, ' lists';
# or
my $info = Redis::CappedCollection::collection_info(
    redis   => $redis,  # or redis => { server => "$server:$port" }
    name    => 'Collection name',
);

Get collection information and status. It can be called as either an existing Redis::CappedCollection object method or a class function.

collection_info arguments are in key-value pairs. Arguments description match the arguments description for "create" method:

redis
name

If invoked as the object method, collection_info, arguments are optional and use corresponding object attributes as defaults.

If called as a class methods, the arguments are mandatory.

Returns a reference to a hash with the following elements:

lists - Number of lists in a collection.
items - Number of data items stored in the collection.
oldest_time - Time of the oldest data in the collection. undef if the collection does not contain data.
older_allowed - True if it is allowed to put data in collection that is older than the last element removed from collection.
memory_reserve - Memory reserve coefficient.
cleanup_bytes - The minimum size, in bytes, of the data to be released when performing memory cleanup.
cleanup_items - The minimum number of the collection elements to be realesed when performing memory cleanup.
max_list_items - Maximum list items limit.
data_version - Data structure version.
last_removed_time - time of the last removed element from collection or 0 if nothing was removed from collection yet.

An error will cause the program to throw an exception (croak) if an argument is not valid or the collection does not exist.

list_info

list_info( $list_id )

Get data list information and status.

$list_id must be a non-empty string.

Returns a reference to a hash with the following elements:

items - Number of data items stored in the data list.
oldest_time - The time of the oldest data in the list. undef if the data list does not exist.

oldest_time

my $oldest_time = $coll->oldest_time;

Get the time of the oldest data in the collection. Returns undef if the collection does not contain data.

An error exception is thrown (croak) if the collection does not exist.

list_exists

list_exists( $list_id )

Example:

say "The collection has '$list_id' list" if $coll->list_exists( 'Some_id' );

Check whether there is a list in the collection with given ID $list_id.

Returns true if the list exists and false otherwise.

collection_exists

collection_exists( redis => $server, name => $name )

Example:

say 'The collection ', $coll->name, ' exists' if $coll->collection_exists;
my $redis = Redis->new( server => "$server:$port" );
say "The collection 'Some name' exists"
    if Redis::CappedCollection::collection_exists( redis => $redis, name => 'Some name' );

Check whether there is a collection with given name. Returns true if the collection exists and false otherwise.

It can be called as either the existing Redis::CappedCollection object method or a class function.

If invoked as the object method, collection_exists uses redis and name attributes from the object as defaults.

If invoked as the class function, collection_exists requires mandatory redis and name arguments.

These arguments are in key-value pairs as described for "create" method.

An error exception is thrown (croak) if an argument is not valid.

lists

lists( $pattern )

Example:

say "The collection has '$_' list" foreach $coll->lists;

Returns an array of list ID of lists stored in a collection. Returns all list IDs matching $pattern if $pattern is not empty. $patten must be a non-empty string.

Supported glob-style patterns:

h?llo matches hello, hallo and hxllo
h*llo matches hllo and heeeello
h[ae]llo matches hello and hallo, but not hillo

Use '\' to escape special characters if you want to match them verbatim.

Warning: consider lists as a command that should only be used in production environments with extreme care. Its performance is not optimal for large collections. This command is intended for debugging and special operations. Don't use lists in your regular application code.

In addition, it may cause an exception (croak) if the collection contains a very large number of lists ('Error while reading from Redis server').

resize

resize( redis => $server, name => $name, ... )

Example:

$coll->resize( cleanup_bytes => 100_000 );
my $redis = Redis->new( server => "$server:$port" );
Redis::CappedCollection::resize( redis => $redis, name => 'Some name', older_allowed => 1 );

Use the resize to change the values of the parameters of the collection. It can be called as either the existing Redis::CappedCollection object method or a class function.

If invoked as the object method, resize uses redis and name attributes from the object as defaults. If invoked as the class function, resize requires mandatory redis and name arguments.

These arguments are in key-value pairs as described for "create" method.

It is possible to change the following parameters: older_allowed, cleanup_bytes, cleanup_items, memory_reserve. One or more parameters are required.

Returns the number of completed changes.

An error exception is thrown (croak) if an argument is not valid or the collection does not exist.

drop_collection

drop_collection( redis => $server, name => $name )

Example:

$coll->drop_collection;
my $redis = Redis->new( server => "$server:$port" );
Redis::CappedCollection::drop_collection( redis => $redis, name => 'Some name' );

Use the drop_collection to remove the entire collection from the redis server, including all its data and metadata.

Before using this method, make sure that the collection is not being used by other customers.

It can be called as either the existing Redis::CappedCollection object method or a class function. If invoked as the class function, drop_collection requires mandatory redis and name arguments. These arguments are in key-value pairs as described for "create" method.

Warning: consider drop_collection as a command that should only be used in production environments with extreme care. Its performance is not optimal for large collections. This command is intended for debugging and special operations. Avoid using drop_collection in your regular application code.

drop_collection mat throw an exception (croak) if the collection contains a very large number of lists ('Error while reading from Redis server').

An error exception is thrown (croak) if an argument is not valid.

drop_list

drop_list( $list_id )

Use the drop_list method to remove the entire specified list. Method removes all the structures on the Redis server associated with the specified list.

$list_id must be a non-empty string.

Method returns true if the list is removed, or false otherwise.

clear_collection

$coll->clear_collection;

Use the clear_collection to remove the entire collection data from the redis server,

Before using this method, make sure that the collection is not being used by other customers.

Warning: consider clear_collection as a command that should only be used in production environments with extreme care. Its performance is not optimal for large collections. This command is intended for debugging and special operations. Avoid using clear_collection in your regular application code.

clear_collection mat throw an exception (croak) if the collection contains a very large number of lists ('Error while reading from Redis server').

ping

$is_alive = $coll->ping;

This command is used to test if a connection is still alive.

Returns 1 if a connection is still alive or 0 otherwise.

External connections to the server object (eg, C <$redis = Redis->new( ... );>), and the collection object can continue to work after calling ping only if the method returned 1.

If there is no connection to the Redis server (methods return 0), the connection to the server closes. In this case, to continue working with the collection, you must re-create the Redis::CappedCollection object with the "open" method. When using an external connection to the server, to check the connection to the server you can use the $redis->echo( ... ) call. This is useful to avoid closing the connection to the Redis server unintentionally.

quit

$coll->quit;

Close the connection with the redis server.

It does not close the connection to the Redis server if it is an external connection provided to collection constructor as existing Redis object. When using an external connection (eg, $redis = Redis->new (...);), to close the connection to the Redis server, call $redis->quit after calling this method.

DIAGNOSTICS

All recognizable errors in Redis::CappedCollection set corresponding value into the "last_errorcode" and throw an exception (croak). Unidentified errors also throw exceptions but "last_errorcode" is not set.

In addition to errors in the Redis module, detected errors are "$E_MISMATCH_ARG", "$E_DATA_TOO_LARGE", "$E_MAXMEMORY_POLICY", "$E_COLLECTION_DELETED", "$E_DATA_ID_EXISTS", "$E_OLDER_THAN_ALLOWED", "$E_NONEXISTENT_DATA_ID", "$E_INCOMP_DATA_VERSION", "$E_REDIS_DID_NOT_RETURN_DATA", "$E_UNKNOWN_ERROR".

The user has the choice:

Use the module methods and independently analyze the situation without the use of "last_errorcode".
Piece of code wrapped in eval {...}; and analyze "last_errorcode" (look at the "An Example" section).

Debug mode

An error exception is thrown with confess if the package variable $DEBUG set to true.

An Example

An example of error handling.

use 5.010;
use strict;
use warnings;

#-- Common ---------------------------------------------------------
use Redis::CappedCollection qw(
    $DEFAULT_SERVER
    $DEFAULT_PORT

    $E_NO_ERROR
    $E_MISMATCH_ARG
    $E_DATA_TOO_LARGE
    $E_NETWORK
    $E_MAXMEMORY_LIMIT
    $E_MAXMEMORY_POLICY
    $E_COLLECTION_DELETED
    $E_REDIS
);

# Error handling
sub exception {
    my $coll    = shift;
    my $err     = shift;

    die $err unless $coll;
    if ( $coll->last_errorcode == $E_NO_ERROR ) {
        # For example, to ignore
        return unless $err;
    } elsif ( $coll->last_errorcode == $E_MISMATCH_ARG ) {
        # Necessary to correct the code
    } elsif ( $coll->last_errorcode == $E_DATA_TOO_LARGE ) {
        # Limit data length
    } elsif ( $coll->last_errorcode == $E_NETWORK ) {
        # For example, sleep
        #sleep 60;
        # and return code to repeat the operation
        #return 'to repeat';
    } elsif ( $coll->last_errorcode == $E_MAXMEMORY_LIMIT ) {
        # For example, return code to restart the server
        #return 'to restart the redis server';
    } elsif ( $coll->last_errorcode == $E_MAXMEMORY_POLICY ) {
        # Correct Redis server 'maxmemory-policy' setting
    } elsif ( $coll->last_errorcode == $E_COLLECTION_DELETED ) {
        # For example, return code to ignore
        #return "to ignore $err";
    } elsif ( $coll->last_errorcode == $E_REDIS ) {
        # Independently analyze the $err
    } elsif ( $coll->last_errorcode == $E_DATA_ID_EXISTS ) {
        # For example, return code to reinsert the data
        #return "to reinsert with new data ID";
    } elsif ( $coll->last_errorcode == $E_OLDER_THAN_ALLOWED ) {
        # Independently analyze the situation
    } else {
        # Unknown error code
    }
    die $err if $err;
}

my ( $list_id, $coll, @data );

eval {
    $coll = Redis::CappedCollection->create(
        redis   => $DEFAULT_SERVER.':'.$DEFAULT_PORT,
        name    => 'Some name',
    );
};
exception( $coll, $@ ) if $@;
say "'", $coll->name, "' collection created.";

#-- Producer -------------------------------------------------------
#-- New data

eval {
    $list_id = $coll->insert(
        'Some List_id', # list id
        123,            # data id
        'Some data',
    );
    say "Added data in a list with '", $list_id, "' id" );

    # Change the "zero" element of the list with the ID $list_id
    if ( $coll->update( $list_id, 0, 'New data' ) ) {
        say 'Data updated successfully';
    } else {
        say 'Failed to update element';
    }
};
exception( $coll, $@ ) if $@;

#-- Consumer -------------------------------------------------------
#-- Fetching the data

eval {
    @data = $coll->receive( $list_id );
    say "List '$list_id' has '$_'" foreach @data;
    # or to obtain records in the order they were placed
    while ( my ( $list_id, $data ) = $coll->pop_oldest ) {
        say "List '$list_id' had '$data'";
    }
};
exception( $coll, $@ ) if $@;

#-- Utility --------------------------------------------------------
#-- Getting statistics

my ( $lists, $items );
eval {
    my $info = $coll->collection_info;
    say 'An existing collection uses ', $info->{cleanup_bytes}, " byte of 'cleanup_bytes', ",
        'in ', $info->{items}, ' items are placed in ',
        $info->{lists}, ' lists';

    say "The collection has '$list_id' list"
        if $coll->list_exists( 'Some_id' );
};
exception( $coll, $@ ) if $@;

#-- Closes and cleans up -------------------------------------------

eval {
    $coll->quit;

    # Before use, make sure that the collection
    # is not being used by other clients
    #$coll->drop_collection;
};
exception( $coll, $@ ) if $@;

CappedCollection data structure

Using currently selected database (default = 0).

CappedCollection package creates the following data structures on Redis:

#-- To store collection status:
# HASH    Namespace:S:Collection_id
# For example:
$ redis-cli
redis 127.0.0.1:6379> KEYS C:S:*
1) "C:S:Some collection name"
#   | |                  |
#   | +-------+          +------------+
#   |         |                       |
#  Namespace  |                       |
#  Fixed symbol of a properties hash  |
#                         Capped Collection id
...
redis 127.0.0.1:6379> HGETALL "C:S:Some collection name"
1) "lists"              # hash key
2) "1"                  # the key value
3) "items"              # hash key
4) "1"                  # the key value
5) "older_allowed"      # hash key
6) "0"                  # the key value
7) "cleanup_bytes"      # hash key
8) "0"                  # the key value
9) "cleanup_items"      # hash key
10) "0"                  # the key value
11) "max_list_items"     # hash key
12) "100"               # the key value
13) "memory_reserve"    # hash key
14) "0.05"              # the key value
15) "data_version"      # hash key
16) "3"                 # the key value
17) "last_removed_time" # hash key
18) "0"                 # the key value
...

#-- To store collection queue:
# ZSET    Namespace:Q:Collection_id
# For example:
redis 127.0.0.1:6379> KEYS C:Q:*
1) "C:Q:Some collection name"
#   | |                  |
#   | +------+           +-----------+
#   |        |                       |
#  Namespace |                       |
#  Fixed symbol of a queue           |
#                        Capped Collection id
...
redis 127.0.0.1:6379> ZRANGE "C:Q:Some collection name" 0 -1 WITHSCORES
1) "Some list id" ----------+
2) "1348252575.6651001"     |
#           |               |
#  Score: oldest data_time  |
#                   Member: Data List id
...

#-- To store CappedCollection data:
# HASH    Namespace:D:Collection_id:DataList_id
# If the amount of data in the list is greater than 1
# ZSET    Namespace:T:Collection_id:DataList_id
# For example:
redis 127.0.0.1:6379> KEYS C:[DT]:*
1) "C:D:Some collection name:Some list id"
# If the amount of data in the list is greater than 1
2) "C:T:Some collection name:Some list id"
#   | |                  |             |
#   | +-----+            +-------+     + ---------+
#   |       |                    |                |
# Namespace |                    |                |
# Fixed symbol of a list of data |                |
#                    Capped Collection id         |
#                                         Data list id
...
redis 127.0.0.1:6379> HGETALL "C:D:Some collection name:Some list id"
1) "0"                      # hash key: Data id
2) "Some stuff"             # the key value: Data
...
# If the amount of data in the list is greater than 1
redis 127.0.0.1:6379> ZRANGE "C:T:Some collection name:Some list id" 0 -1 WITHSCORES
1) "0" ---------------+
2) "1348252575.5906"  |
#           |         |
#   Score: data_time  |
#              Member: Data id
...

DEPENDENCIES

In order to install and use this package Perl version 5.010 or better is required. Redis::CappedCollection module depends on other packages that are distributed separately from Perl. We recommend the following packages to be installed before installing Redis::CappedCollection :

Const::Fast
Digest::SHA1
Mouse
Params::Util
Redis
Try::Tiny

The Redis::CappedCollection module has the following optional dependencies:

Data::UUID
JSON::XS
Net::EmptyPort
Test::Exception
Test::NoWarnings
Test::RedisServer

If the optional modules are missing, some "prereq" tests are skipped.

The installation of the missing dependencies can either be accomplished through your OS package manager or through CPAN (or downloading the source for all dependencies and compiling them manually).

BUGS AND LIMITATIONS

Redis server version 2.8 or higher is required.

The use of maxmemory-policy all* in the redis.conf file could lead to a serious (and hard to detect) problem as Redis server may delete the collection element. Therefore the Redis::CappedCollection does not work with mode maxmemory-policy all* in the redis.conf.

It may not be possible to use this module with the cluster of Redis servers because full name of some Redis keys may not be known at the time of the call the Redis Lua script ('EVAL' or 'EVALSHA' command). So the Redis server may not be able to correctly forward the request to the appropriate node in the cluster.

We strongly recommend setting maxmemory option in the redis.conf file.

WARN: Not use maxmemory less than for example 70mb (60 connections) for avoid 'used_memory > maxmemory' problem.

Old data with the same time will be forced out in no specific order.

The collection API does not support deleting a single data item.

UTF-8 data should be serialized before passing to Redis::CappedCollection for storing in Redis.

According to Redis documentation:

This module consider that any data sent to the Redis server is a raw octets string, even if it has utf8 flag set. And it doesn't do anything when getting data from the Redis server.

TODO: implement tests for
- memory errors (working with internal ROLLBACK commands)
- working when maxmemory = 0 (in the redis.conf file)
WARNING: According to initServer() function in redis.c :
```
/* 32 bit instances are limited to 4GB of address space, so if there is
 * no explicit limit in the user provided configuration we set a limit
 * at 3 GB using maxmemory with 'noeviction' policy'. This avoids
 * useless crashes of the Redis instance for out of memory. */
```
The Redis::CappedCollection module was written, tested, and found working on recent Linux distributions.

There are no known bugs in this package.

Please report problems to the "AUTHOR".

Patches are welcome.

SOURCE CODE

Redis::CappedCollection is hosted on GitHub: https://github.com/TrackingSoft/Redis-CappedCollection

AUTHOR

Sergey Gladkov, <sgladkov@trackingsoft.com>

Please use GitHub project link above to report problems or contact authors.

CONTRIBUTORS

Alexander Solovey

Jeremy Jordan

Vlad Marchenko

COPYRIGHT AND LICENSE

This package is free software; you can redistribute it and/or modify it under the same terms as Perl itself. See perlartistic at http://dev.perl.org/licenses/artistic.html.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

To install Redis::CappedCollection, copy and paste the appropriate command in to your terminal.

cpanm

cpanm Redis::CappedCollection

CPAN shell

perl -MCPAN -e shell
install Redis::CappedCollection

For more information on module installation, please visit the detailed CPAN module installation guide.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	go to github issues (only if github is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

	Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)

NAME

VERSION

SYNOPSIS

ABSTRACT

DESCRIPTION

EXPORT

$DEFAULT_SERVER

$DEFAULT_PORT

$DEFAULT_CONNECTION_TIMEOUT

$DEFAULT_OPERATION_TIMEOUT

$NAMESPACE

$MIN_MEMORY_RESERVE, $MAX_MEMORY_RESERVE

$DEFAULT_CLEANUP_ITEMS

$DATA_VERSION

CONSTRUCTOR

create

open

METHODS

name

redis

reconnect_on_error

connection_timeout

operation_timeout

cleanup_bytes

cleanup_items

max_list_items

max_datasize

older_allowed

memory_reserve

last_errorcode

insert

update

upsert

receive

pop_oldest

redis_config_ok

collection_info

list_info

oldest_time

list_exists

collection_exists

lists

resize

drop_collection

drop_list

clear_collection

ping

quit

DIAGNOSTICS

Debug mode

An Example

CappedCollection data structure

DEPENDENCIES

BUGS AND LIMITATIONS

MORE DOCUMENTATION

SEE ALSO

SOURCE CODE

AUTHOR

CONTRIBUTORS

COPYRIGHT AND LICENSE

Module Install Instructions