NAME

es-alias-manager.pl - Allow easy alias management for daily indexes

VERSION

version 2.6

SYNOPSIS

es-alias-manager.pl --local --config /etc/elasticsearch/aliases.yml

Options:

    --help              print help
    --manual            print full manual
    --config            Location of Config File, default /etc/elasticsearch/aliases.yml

From App::ElasticSearch::Utilities:

    --local         Use localhost as the elasticsearch host
    --host          ElasticSearch host to connect to
    --port          HTTP port for your cluster
    --noop          Any operations other than GET are disabled
    --timeout       Timeout to ElasticSearch, default 30
    --keep-proxy    Do not remove any proxy settings from %ENV
    --index         Index to run commands against
    --base          For daily indexes, reference only those starting with "logstash"
                     (same as --pattern logstash-* or logstash-DATE)
    --datesep       Date separator, default '.' also (--date-separator)
    --pattern       Use a pattern to operate on the indexes
    --days          If using a pattern or base, how many days back to go, default: all

ARGUMENT GLOBALS

Some options may be specified in the /etc/es-utils.yaml or $HOME/.es-utils.yaml file:

    ---
    host: esproxy.example.com
    port: 80
    timeout: 10

From CLI::Helpers:

    --color             Boolean, enable/disable color, default use git settings
    --verbose           Incremental, increase verbosity
    --debug             Show developer output
    --quiet             Show no output (for cron)

DESCRIPTION

This script assists in maintaining the aliases for relative or daily indexes across multiple datacenters.

Use with cron:

    22 4 * * * es-alias-manager.pl --local --config /etc/elasticsearch/aliases.yml

This will allow you to split your cluster between datacenters (or whatever) and alias the split clusters to a homogenous index that standard LogStash/Kibana interfaces will understand.

If I create the following in /etc/elasticsearch/aliases.yml

    ---
    logstash:
      pattern: \*-logstash-{{DATE}}
      daily: logstash-{{DATE}}
      relative:
        alias: logstash-{{PERIOD}}
        periods:
          today:
            from:
              days: 0
            to:
              days: 0
          lastweek:
            from:
              days: 14
            to:
              days: 7

Assuming today is the 2013.07.18 and I have 3 datacenters (IAD, NYC, AMS) with the following indices:

    iad-logstash-2013.07.17
    iad-logstash-2013.07.18
    nyc-logstash-2013.07.17
    nyc-logstash-2013.07.18
    ams-logstash-2013.07.17
    ams-logstash-2013.07.18

The following aliases would be created

    logstash-2013.07.17
        |- iad-logstash-2013.07.17
        |- nyc-logstash-2013.07.17
        `- ams-logstash-2013.07.17

    logstash-2013.07.18
        |- iad-logstash-2013.07.18
        |- nyc-logstash-2013.07.18
        `- ams-logstash-2013.07.18

    logstash-today
        |- iad-logstash-2013.07.18
        |- nyc-logstash-2013.07.18
        `- ams-logstash-2013.07.18

This lets you use index templates and the index.routing.allocation to isolate data by datacenter or another parameter to certain nodes while allowing all the nodes to work together as cleanly as possible. This also facilitates the default expectations of Kibana to have a single index per day when you may need more.

PATTERN VARIABLES

Patterns are used to match an index to the aliases it should have. A few symbols are expanded into regular expressions. Those patterns are:

    The '*' expands to match any number of any characters.
    The '?' expands to match any single character.
    {{DATE}} expands to match YYYY.MM.DD, YYYY-MM-DD, or YYYYMMDD

ALIAS VARIABLES

For daily indices, the following variables are available:

    {{DATE}} - Expands to YYYY.MM.DD for the current day of the current index

For relative period indices, the following variable is required.

    {{PERIOD}} - Name of the period

OPTIONS

config

Location of the config file, default is /etc/elasticsearch/aliases.yml

AUTHOR

Brad Lhotsky <brad@divisionbyzero.net>

COPYRIGHT AND LICENSE

This software is Copyright (c) 2012 by Brad Lhotsky.

This is free software, licensed under:

  The (three-clause) BSD License

cpanm App::ElasticSearch::Utilities

perl -MCPAN -e shell
install App::ElasticSearch::Utilities