The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.
 / 
Fluent-LibFluentBit-0.03
River stage zero No dependents
/ Fluent::LibFluentBit

NAME

Fluent::LibFluentBit - Perl interface to libfluent-bit.so

VERSION

version 0.03

SYNOPSIS

  use Fluent::LibFluentBit -config => {
    log_level => 'trace',
    outputs => [{
      name => 'datadog',
      match => '*',
      host => "http-intake.logs.datadoghq.com",
      tls => 'on',
      compress => 'gzip',
      apikey => $ENV{DATADOG_API_KEY},
      dd_service => 'example',
      dd_source => 'perl-fluentbit',
    }]
  );
  my $logger= Fluent::LibFluentBit->new_logger;
  $logger->info("Message");
  $logger->error({ log => "Message", key1 => "value1", key2 => "value2" });

DESCRIPTION

Fluent is a software tool that collects log data from a wide variety of sources and delivers them to a wide variety of destinations (with 1000+ plugins that cover just about any conceivable source or destination) and buffers them in a server "fluentd" to act as a central point of configuration and to smooth over any network interruptions.

Fluent-Bit is a smaller single-process implementation of the same idea. It is written in C, for performance and low overhead, and available as both a standalone program and a C library. It supports fewer plugins (but still an impressive 100+) but does not need an intermediate server for the buffering. When used as a C library, the main application gets to write log data un-blocked while a background thread in libfluent-bit does the work of writing to remote destinations.

To integrate fluent-bit with a Perl application, you have several options, including:

  • Write to log files, then run fluent-bit as a log processor that watches for new lines in the files.

  • Pipe the perl process output into stdin of fluent-bit, as either JSON or parsed plaintext.

  • Use this module to feed data directly into fluent-bit within the same process (but separate thread)

There are a time and a place for each of these options. The main use case for this module (as I see it) is when it would be inconvenient to direct the output of the process into a pipe, and where you trust the perl script to do its logging via an API and not accidentally via stdout (which wouldn't be seen by libfluent-bit).

CONSTRUCTOR

default_instance

You probably only want one instance of fluent-bit running per program, so all the methods of this package can be called as class methods and they will operate on this default instance. The instance gets created the first time you call default_instance or if you pass -config to the 'use' line.

new

This creates a non-default instance of the library. You probably don't need this; see "default_instance" above.

Arguments to new get passed to "configure".

ATTRIBUTES

All attributes are read-only and should be modified using "configure"

inputs

Arrayref of Fluent::LibFluentBit::Input.

outputs

Arrayref of Fluent::LibFluentBit::Output.

filters

Arrayref of Fluent::LibFluentBit::Filter.

started

Boolean, whether the background thread is running.

METHODS

All methods may be called on the class, in which case they will use "default_instance".

configure

  $flb->configure( $key => $value, ... );

This accepts any attribute (case-insensitive) that you could write in the [SERVICE] section of the fluent-bit config file. Invalid attributes generate warnings instead of exceptions.

You may also pass a list of inputs => [...], outputs => [...], and filters => [...] which will generate calls to "add_input", "add_output", and "add_filter" respectively.

add_input

  $inp= $flb->add_input($type => \%config);
  $inp= $flb->add_input({ name => $type, %config... });

Create and configure a new input. You probably don't need this if you are only using the loggers from this library as input. %config Attributes are not case-sensitive, and are the same keys and values you would write in the [INPUT] sections of the config file.

Returns an instance of Fluent::LibFluentBit::Input which is also added to the "inputs" attribute.

add_filter

Same as add_input, for filters.

add_output

Same as add_input, for outputs.

start

Start the fluent-bit engine. This should probably only occur after all configurations of inputs and filters and outputs.

This is a no-op if the engine is already started. It can die if flb_start returns an error.

stop

Stop the fluent-bit engine, if it is started. This relies on the "started" attribute and does not consult the library. (maybe that's a bug?)

new_logger

Return a new instance of Fluent::LibFluentBit::Logger which feeds messages to the 'lib' input of the library. Currently these all use the same input handle, creted the first time the logger gets used, and which triggers a call to "start".

EXPORTS

The following can be exported into your namespace for a more C-like experience:

libfluent-bit API

flb_create
flb_service_set
flb_input
flb_input_set
flb_filter
flb_filter_set
flb_output
flb_output_set
flb_start
flb_stop
flb_destroy
flb_lib_push
flb_lib_config_file

Constants

FLB_LIB_ERROR
FLB_LIB_NONE
FLB_LIB_OK
FLB_LIB_NO_CONFIG_MAP

AUTHOR

Michael Conrad <mconrad@intellitree.com>

COPYRIGHT AND LICENSE

This software is copyright (c) 2023 by Michael Conrad.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.