NAME

Kafka::Message - interface to the Kafka message properties.

VERSION

This documentation refers to Kafka::Message version 0.800_3 .

SYNOPSIS

    use 5.010;
    use strict;
    use warnings;

    use Kafka qw(
        $DEFAULT_MAX_BYTES
    );
    use Kafka::Connection;
    use Kafka::Consumer;

    #-- Connection
    my $connect = Kafka::Connection->new( host => 'localhost' );

    #-- Consumer
    my $consumer = Kafka::Consumer->new( Connection  => $connect );

    # The Kafka consumer response has an ARRAY reference type.
    # For the fetch response array has the class name Kafka::Message elements.

    # Consuming messages
    my $messages = $consumer->fetch(
        'mytopic',          # topic
        0,                  # partition
        0,                  # offset
        $DEFAULT_MAX_BYTES  # Maximum size of MESSAGE(s) to receive
    );
    if ( $messages ) {
        foreach my $message ( @$messages ) {
            if( $message->valid ) {
                say 'key        : ', $message->key;
                say 'payload    : ', $message->payload;
                say 'offset     : ', $message->offset;
                say 'next_offset: ', $message->next_offset;
            }
            else {
                say 'error      : ', $message->error;
            }
        }
    }

DESCRIPTION

This module is not intended to be used by the end user.

Kafka::Message class implements API for Kafka message.

Reference to an array of instances of class Kafka::Message returned by the fetch method of the Consumer client.

The main features of the Kafka::Message class are:

• Provides representing the Apache Kafka Message structure (with no compression codec attribute now). Description of the structure is available at https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-Messagesets

• Support for working with 64 bit elements on 32 bit systems. offset and next_offset methods return the Math::BigInt integer on 32 bit systems.

CONSTRUCTOR

new ( \%arg )

Creates a Kafka::Message, which is a newly created message object. new() takes an argument, this argument is a HASH reference with the currently used methods entries.

Returns the created message as a Kafka::Message object.

METHODS

payload

A simple message received from the Apache Kafka server.

key

The key is an optional message key that was used for partition assignment. The key can be an empty string.

valid

Indicates whether received message is valid or not.

error

A description why message is invalid (currently happens only when message is compressed).

offset

The offset of the message in the Apache Kafka server.

next_offset

The offset of the next message in the Apache Kafka server.

Attributes

This holds metadata attributes about the message. The last 3 bits contain the compression codec used for the message. The other bits are currently unused.

HighwaterMarkOffset

The offset at the end of the log for this partition. This can be used by the client to determine how many messages behind the end of the log they are.

MagicByte

This is version id used to allow backwards compatible evolution of the message binary format.

DIAGNOSTICS

In order to achieve better performance, constructor of this module does not perform validation of arguments.

SEE ALSO

The basic operation of the Kafka package modules:

Kafka - constants and messages used by the Kafka package modules.

Kafka::Connection - interface to connect to a Kafka cluster.

Kafka::Producer - interface for producing client.

Kafka::Consumer - interface for consuming client.

Kafka::Message - interface to access Kafka message properties.

Kafka::Int64 - functions to work with 64 bit elements of the protocol on 32 bit systems.

Kafka::Protocol - functions to process messages in the Apache Kafka's Protocol.

Kafka::IO - low level interface for communication with Kafka server.

Kafka::Internals - Internal constants and functions used by several package modules.

A wealth of detail about the Apache Kafka and the Kafka Protocol:

Main page at http://kafka.apache.org/

Kafka Protocol at https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol

AUTHOR

Sergey Gladkov, <sgladkov@trackingsoft.com>

CONTRIBUTORS

Alexander Solovey

Jeremy Jordan

Vlad Marchenko

COPYRIGHT AND LICENSE

Copyright (C) 2012-2013 by TrackingSoft LLC.

This package is free software; you can redistribute it and/or modify it under the same terms as Perl itself. See perlartistic at http://dev.perl.org/licenses/artistic.html.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

cpanm Kafka

perl -MCPAN -e shell
install Kafka