Sereal::Splitter - splits a Sereal blob in chunks of roughly the same size
use Sereal::Splitter qw(SRL_ZLIB create_header_data_template); my $splitter = Sereal::Splitter->new( { input => $data, chunk_size => 1, compress => SRL_ZLIB, header_data_template => create_header_data_template( { date => time(), elements_count => '__$CNT__' } ) } ); while (defined( my $chunk = $splitter->next_chunk())) { # do stuff with $chunk; }
This library implements an efficient way of splitting a Sereal blob into smaller chunks. Currently, it only works with ArrayRefs Sereal blobs, like this:
[ $element_1, $element_2, ..., $element_n ]
In the future, it may also work with HashRefs.
Takes a HashRef with options:
Mandatory, String, the Sereal blob to split
Mandatory, positive Int, the approximate size of the uncompressed chunk
Optional, Int, one of SRL_UNCOMPRESSED, SRL_SNAPPY or SRL_ZLIB. These constant can be exported at use time. If set, indicates how chunks must be compressed. Defaults to SRL_UNCOMPRESSED.
Optional, Str, the header_data to inject in each chunk. This header_data can contain special scalar values, that will be replaced by values. Special scalar values are:
This will be replaced by the number of elements that the chunks contains. It must be encoded as SHORT_BINARY_08. It'll be replaced by a VARINT.
REMARK: In theory, it should be a string of lentgh 11, because varint max size are 11 bytes. However, the Sereal decoder code, can't cope with varint bigger than 8 bytes, because of a bug, and even if the varint is forged like 0x8180808080808080808080.
To make things easier, you can use create_header_data_template (see below) to create it for you.
create_header_data_template
returns the next chunk as a String, or Undef if it was the last chunk
Given a structure, will return a Sereal *body*, that can be used as value for the header_data_template constructor option.
header_data_template
This function loads Sereal::Encoder if it's not already loaded.
Sereal::Encoder
To install Sereal::Splitter, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Sereal::Splitter
CPAN shell
perl -MCPAN -e shell install Sereal::Splitter
For more information on module installation, please visit the detailed CPAN module installation guide.