The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.
 / 
Search-ESsearcher-0.4.4
River stage zero No dependents
++
/ Search::ESsearcher

NAME

Search::ESsearcher - Provides a handy system for doing templated elasticsearch searches.

VERSION

Version 0.4.4

SYNOPSIS

    use Search::ESsearcher;

    my $ess = Search::ESsearcher->new();

METHODS

new

This initiates the object.

    my $ss=Search::ESsearcher->new;

This returns what Elasticsearch config will be used.

    my $elastic=$ess->elastic_get;

This sets the name of the config file to use.

One option is taken and name of the config file to load.

Undef sets it back to the default, "default".

    $ess->elastic_set('foo');

    $ess->elastic_set(undef);

fetch_help

This fetches the help for the current search and returns it. Failsure to find one, results in a empty message being returned.

    my $help=$ess->fetch_help;

get_options

This fetches the options for use later when filling in the search template.

    $ess->get_options;

load_options

This loads the currently set options.

    $ess->load_options;

load_elastic

This loads the currently specified config file containing the Elasticsearch config JSON.

    $ess->load_elastic;

load_output

This loads the currently specified output template.

While this is save internally, the template is also returned as a string.

    my $outpot_template=$ess->load_output;

This loads the currently specified search template.

While this is save internally, the template is also returned as a string.

    my $search_template=$ess->load_search;

name_valide

This validates a config name.

One option is taken and that is the name to valid.

The returned value is a perl boolean based on if it it is valid or not.

    if ( ! $ess->name_validate( $name ) ){
        print "Name is not valid.\n";
    }

This returns the currently set options config name.

    my $options=$ess->options_get;

This sets the options config name to use.

One option is taken and this is the config name. If it is undefiend, then the default is used.

    $ess->options_set( $name );

This returns the currently set output template name.

    my $output=$ess->output_get;

This sets the output template name to use.

One option is taken and this is the template name. If it is undefiend, then the default is used.

    $ess->output_set( $name );

results_process

This processes the results from search_run.

One option is taken and that is the return from search_run.

The returned value from this is array of each document found after it has been formated using the set output template.

    my $results=$ess->search_run;
    my @formated=$ess->results_process( $results );
    @formated=reverse(@formated);
    print join("\n", @formated)."\n";

This returns the currently set search template name.

    my $search=$ess->search_get;

search_fill_in

This fills in the loaded search template.

The results are saved internally as well as returned.

    my $filled_in=$ess->search_fill_in;

search_run

This is used to run the search after search_fill_in has been called.

The returned value is of the type that would be returned by Search::Elasticsearch->search.

    my $results=$ess->search_run;

This sets the search template name to use.

One option is taken and this is the template name. If it is undefiend, then the default is used.

    $ess->search_sets( $name );

Configuration And Usage

Configs, help, and templates are looked for in the following manner and order, with the following of the elasticsearch config.

   $ENV{HOME}."/.config/essearcher/".$item."/".$name
   $base.'/etc/essearcher/".$item."/".$name
   Search::ESsearcher::Templates::$name->$item
   ERROR

Item can be any of the following.

    elastic
    help
    output
    options
    search

The basic idea is you have matching output, options and search that you can use to perform queries and print the results.

Each template/config is its own file under the directory named after its purpose. So the options template fail2ban would be 'options/fail2ban'.

elastic

This directory contains JSON formatted config files for use with connecting to the Elasticsearch server.

This is then read in and converted to a hash and feed to Search::Elasticsearch->new.

By default it will attempt to connect to it on "127.0.0.1:9200". The JSON equivalent would be...

    { "nodes": [ "127.0.0.1:9200" ] }

options

This is a file that will be used as a string for with Getopt::Long. They will be passed to the templates as a hash.

help

This contains information on the options the search uses.

This is just a text file containing information and is not required.

If you are writing a module, it should definitely be present.

This is a Template template that will be filled in using the data from the passed command line options and used to run the search.

The end result should be valid JSON that can be turned into a hash for feeding Search::Elasticsearch->search.

When writing search templates, it is highly suggested to use Template::Plugin::JSON for when inserting variables as it will automatically escape them.

output

This is a Template template that will be filled in using the data from the passed command line options and the returned results.

It will be used for each returned document. bin/essearcher will then join the array with "\n".

TEMPLATES

o

This is a hash that contains the parsed options.

Below is a example with the option --program and transforming it a JSON save value.

    [% USE JSON ( pretty => 1 ) %]
    [% DEFAULT o.program = "*" %]
    
    {"query_string": {
        "default_field": "program",
        "query": [% o.program.json %]
        }
    },

aon

This is AND, OR, or NOT sub that handles the following in a string, transforming them from the punctuation to the logic.

    , OR
    + AND
    ! NOT

So the string "postfix,spamd" would become "postfix OR spamd".

Can be used like below.

    [% USE JSON ( pretty => 1 ) %]
    
    [% IF o.program %]
    {"query_string": {
        "default_field": "program",
        "query": [% aon( o.program ).json %]
        }
    },
    [% END %]

This function is only available for the search template.

aonHost

This is AND, OR, or NOT sub that handles the following in a string, transforming them from the punctuation to the logic.

    , OR
    + AND
    ! NOT

So the string "foo.,mail.bar." would become "/foo./ OR /mail.bar./".

This is best used with $field.keyword.

Can be used like below.

    [% USE JSON ( pretty => 1 ) %]

    [% IF o.host %]
    {"query_string": {
        "default_field": "host.keyword",
        "query": [% aonHost( o.host ).json %]
        }
    },
    [% END %]

This function is only available for the search template.

c

This wraps Term::ANSIColor->color.

    [% c("cyan") %][% f.timestamp %] [% c("bright_blue") %][% f.logsource %]

This function is only available for the output template.

pd

This is a time helper.

/^-/ appends "now" to it. So "-5m" becomes "now-5m".

/^u\:/ takes what is after ":" and uses Time::ParseDate to convert it to a unix time value.

Any thing not matching maching any of the above will just be passed on.

    [% IF o.dgt %]
        {"range": {
            "@timestamp": {
                "gt": [% pd( o.dgt ).json %]
            }
        }
        },
    [% END %]

Modules

Additonal modules bundling help, options, search, and output can be made. The requirement for these are as below.

They need to exist below Search::ESsearcher::Templates.

Provide the following functions that return strings.

    help
    options
    search
    output

Basic information as to what is required to make it work in logstash or the like is also good as well.

ERROR CODES/FLAGS

All error handling is done via Error::Helper.

1 / IOerror

Failed to perform some sort of file operation.

2 / NOfile

The specified template/config does not exist.

3 / nameIsInvalid

Invalid name specified.

4 / searchNotUsable

Errored while processing the template.

For more information on writing templates, see Template.

5 / elasticnotloadable

The Elasticsearch config does not parse as JSON, preventing it from being loaded.

6 / notResults

The return value passed to results_process deos not appear to be a results return. Most likely the search errored and returned undef or a blank value.

AUTHOR

Zane C. Bowers-Hadley, <vvelox at vvelox.net>

BUGS

Please report any bugs or feature requests to bug-search-essearcher at rt.cpan.org, or through the web interface at https://rt.cpan.org/NoAuth/ReportBug.html?Queue=Search-ESsearcher. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.

SUPPORT

You can find documentation for this module with the perldoc command.

    perldoc Search::ESsearcher

You can also look for information at:

ACKNOWLEDGEMENTS

LICENSE AND COPYRIGHT

This software is Copyright (c) 2019 by Zane C. Bowers-Hadley.

This is free software, licensed under:

  The Artistic License 2.0 (GPL Compatible)

8 POD Errors

The following errors were encountered while parsing the POD:

Around line 102:

Unknown directive: =head

Around line 121:

Unknown directive: =head

Around line 596:

Unknown directive: =head

Around line 615:

Unknown directive: =head

Around line 650:

Unknown directive: =head

Around line 670:

Unknown directive: =head

Around line 785:

Unknown directive: =head

Around line 922:

Unknown directive: =head