==================================================
Changes from 2016-09-18 00:00:00 +0000 to present.
==================================================
----------------------------------------
version 6.1 at 2018-09-18 03:10:49 +0000
----------------------------------------
Change: 020fddc36e35230b1a275c87c8edddae2fbb7c7b
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-09-17 20:10:27 +0000
Updates for the documentation for 6.1 release
Change: 263e77f276c95dbf1f90bba6b77264547e4b7390
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-09-17 20:09:05 +0000
Add --bg-filter option for es-search.pl
If significant_terms is used for the --top aggregation, this option is able
to inject a background_filter into the significant_terms aggregation.
Change: e76897d8eefee23d70b9fd5b3e6bd26bd7494b95
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-09-17 20:08:24 +0000
Update to using GitHub::Meta and rearrange Pod::Weaver configs
----------------------------------------
version 6.0 at 2018-07-28 00:45:25 +0000
----------------------------------------
Change: e58dcf2f190a1db3e1e10f8799b67c7f9a8abcc4
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-27 17:45:25 +0000
Fix a bug in conditional checks for index age when no date is found in the
index name
Change: f6e05fcf90fa575f5e5b5ccafb0bf94667ef9bd5
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-23 14:31:41 +0000
New es_local_index_meta() function to allow looking up the timestamp field
in es-search.pl much easier and more straight-forward.
----------------------------------------
version 5.9 at 2018-07-21 01:10:38 +0000
----------------------------------------
Change: 9b00c64735a21b74223fddcde44fff74772efd17
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-20 18:10:38 +0000
Bumping Release to 5.9
Change: 415c84da87571aa488a0f246d423e24fbf446e00
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-18 17:58:55 +0000
Fix another uninitialized error in the code
----------------------------------------
version 5.8 at 2018-07-16 19:14:45 +0000
----------------------------------------
Change: 3f276cdc5e5b8a1fc6215275b8b3c24e3d8ef066
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-16 12:14:45 +0000
Docs update for 5.8 release
Change: 12336ccb1ee03eaa294af46a1340af3cbb47f4c1
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-16 11:45:58 +0000
Fix a warning and improve output
* Fix a "use of uninitialized" warning for when an index is missing a
date string but matches the base pattern * Fix logic bug that meant
'--delete --delete-days 1' was a noop because
of using '<' instead of '<='
Change: e3b0899af37835e7e9d372788c31339cc3cf8f01
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-16 11:43:38 +0000
Bubble up errors so we know why we failed
Change: 9a07d69d5a7571d58951682c3d190083cf0a84df
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-16 11:25:16 +0000
Show shards breakdown by primary and replica
----------------------------------------
version 5.7 at 2018-07-06 05:34:05 +0000
----------------------------------------
Change: e90729b13bb4fc45d68e9711ac139c8f8ab1dad8
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-05 22:34:05 +0000
Documentation Updates for 5.7 release
Change: 7498a4ccea72cd4cc8860316bf3aa175e278c158
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-05 22:23:55 +0000
Add support for statistical aggregations via --with
Change: 8a86345e449b355257358de7f81414d278442a38
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-05 22:05:43 +0000
Fix joins when using "NOT" for negations
Change: 4e103a8feb362f9b5722736c3fdda66d9f36819a
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-05 20:20:40 +0000
Fixup broken query strings without joining keywords
This addresses the most used case where a query_string is missing the
keyword 'AND' in the token joining. This also makes it easier to produce
longer queries with less thinking.
Change: 1b43ea3120ae6ff51bc9f0d7c04e46f05499e7bb
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-02 23:13:54 +0000
Fix documentation for the ::Ranges plugin
Change: c7cb7c238a01c235823161c7d4455afe00665870
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-02 22:58:21 +0000
Add a --filter option to es-search.pl
Make use of the newly exposed 'filter' context from ::QueryString and add
positive matches as either 'must' (query context) or 'filter' (filter
context).
Change: 498af1d1759cf2788a06526a847d331e81e028d6
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-02 22:55:00 +0000
Allow toggling the context between 'query' and 'filter'
By overriding the ::QueryString context, it's possible to toggle between
collecting the positive matches as 'filter' elements instead of 'must'.
This allows the use of unscored queries which may improve performance for
some queries dramatically.
Change: 601bb7eea49cc3a8dbe0c9579c6a0b94e5675824
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-02 22:53:07 +0000
Early return for ::QueryString:: plugins
Both the ::Nested and ::Ranges plugins would emit uninitialized warnings
when run due to forgetting to check if the token matched the plugin.
Change: 67342d859955cc6f4ca4b6a75363beb08f7af1d8
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-02 21:41:22 +0000
Fix parameter specification in the library too
Change: 1039688ce0316e52f73f3ff57ec7b2ddedf0636b
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-02 21:36:26 +0000
Fix the parameter specifications and remove unsupported script
Change: 60aaa2dc1ea4ad19f0d5837a3620e9f98029196e
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-02 21:21:54 +0000
Add ::QueryString::Ranges
Translates queries with field:<,<=,>,>-= to equivalent range queries
Change: ca4e73f856bd9f48818e3fd2a7b959998dee8047
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-02 08:58:05 +0000
Fix --base in es-search.pl, make --index less policey.
* --base was broken as es-search.pl is looking for open indices.
Prepend the wildcard to the '/_stats/docs' request. * Don't validate the
--index parameter, instead pass it through. This
gives the user the power to create their own index strings
Change: a880e6ff664b99e64da8d18589f66bc51f1e089c
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-02 08:58:05 +0000
Replace es-storage-data.pl with es-storage-overview.pl
Change: 7bb0c293fd3f433183002c75be6a7e0e6e0f53fc
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-06-18 20:49:18 +0000
Remove size parameter if set to zero.
For statistical aggregations, the size parameter is invalid. This is a
shortcut to remove the parameter from the sub aggregation.
Change: 1e6372612b5e86627f6df2703c8a49e344bd129d
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-06-08 16:13:52 +0000
Fix failing tests
Change: 3053cd3ee7c1f30138f5fe2f51b6e77dc6639bba
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-05-10 21:40:51 +0000
Make the fast things fast
* Reorder the --bases for es-search.pl to skip index meta checks. * Use
es_indices(check_date => 0) instead of _all => 1 to skip closed
indices. * Make es_index_bases() simpler, i.e.:
Old behavior:
(a, a-b, a-b-c, b-c, c) = es_index_bases('a-b-c');
New behavior:
(a, a-b, a-b-c) = es_index_bases('a-b-c');
* Use _stats/docs to resolve all index names as indexes without explicit
aliases don't show up in all versions of ElasticSearch.
Change: ce08b1c0cc3c9432780612d13dd1214dc60be04e
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-05-10 21:06:34 +0000
New script: es-storage-overview.pl
Implements the simplest possible overview of storage. Does not provide node
or index data, but gives a high level overview.
Change: 3837060be0e0145b2e23987da6fab11af9d54a4e
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-05-10 20:29:24 +0000
es-search.pl - more concise output
When we encounter a record that is an array, but is holding only 1 item,
derefernce and extract the single value.
Change: b2473b3b412444a8148dfd96b0a07eb0dc466054
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-04-30 19:57:37 +0000
Made es_request() errors fatal and sped up index lookup
----------------------------------------
version 5.6 at 2018-04-13 18:20:03 +0000
----------------------------------------
Change: fcb981c7ea59b5e3ac5f9c5b2ae3cf34e1fad5a3
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-04-13 20:20:03 +0000
Documentation fixup with the latest pod tooling
Change: 2d9fce9813a585ff3115bfdce4b6b46a09192bf7
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-04-13 10:57:03 +0000
Don't track the local TODO file
Change: 7727fd81af17556c54e9c911d507efcd20266765
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-04-13 10:54:44 +0000
Ready for 5.6 Release
* Remove overzealous debugging messages * Tweak es-storage-data.pl to
provide full details via --all * Change the way
es-daily-index-maintenance.pl complains when --master--only is set. *
Documentation updated
Change: 1b06f8ce6a574e34acea51938193d7590c6c24f9
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-03-17 00:49:08 +0000
Docs updated added --view base to es-storage-data.pl
Change: 0cf82d70c23104b590f29b4551612578541cbb6a
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-03-17 00:39:18 +0000
Get es-storage-data.pl working on modern ElasticSearch
----------------------------------------
version 5.5 at 2018-03-17 06:01:25 +0000
----------------------------------------
Change: 757dec9e648d077759da9a19ebdf3ed343df82f1
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-03-16 23:01:25 +0000
Remove no longer necessary test
Change: d63a1b823fcd72116e731cbfc45387290b60e769
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-03-16 22:49:15 +0000
Documentation updated for 5.5 release
Change: b1709e428bf9031e99b196ed6c865b02535ddb66
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-03-16 22:43:04 +0000
Bugfix and better replica management
* BUGFIX: es_index_strip_date() now removes everything after the date
string.
es-daily-index-maintenance.pl
* --replicas-* options aliased to --replica-* * Smarter handling of math
around replica calculation * Disable --bloom option entirely as it's
deprecated by Elastic
Change: aabba5ad9a1c26ee5aa631f4ea81d887bf98463a
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-02-21 23:04:18 +0000
Pod and Perl::Critic fixes
Change: 1dcaaa7ea8b7b8b89387692dd969a18fd385bd23
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-02-21 20:58:15 +0000
Conversion to Ref::Util and Types::Tiny
Fix paths for the shebang line so the installer will be able to install the
scripts with the correct Perl binary path. Add a first pass at nested query
syntax
Change: 722e812a7365f9a81aa0d1df5d61a3a1dcb695ed
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-02-21 20:52:06 +0000
Update the QS::IP expander to make it more intelligent.
If a field ends in *_ip, allow CIDR queries against the field. This skips
renaming the field from "src_ip" to "src_ip_numeric" and is now an inline
replacement. This enhances simplifies the CLI queries.
The docs were not accurate, so now the example and documentation are valid.
Change: 8e7e34fe1690349a0485ec271c85e966c699b0d9
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2017-12-03 23:42:00 +0000
Disallow tabs in my files.
Change: 938d5f9e9bf387d304388bf524348dd4b0d51fac
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2017-10-25 17:07:47 +0000
Allow --show to be more flexible
* Unbind from the index fields since this operates on the source
document * Change to an array so you can add multiple --show's
----------------------------------------
version 5.4 at 2017-10-13 19:36:20 +0000
----------------------------------------
Change: c31b5f39953589c497f4374be9c22f36a2514dc2
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2017-10-13 12:36:20 +0000
Documentation update for the 5.4 release.
Change: 3b6a379937869eaf46e80eea6f7d54e5c4f2220a
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2017-07-12 20:50:57 +0000
Default to UTC, warn when using incompatible options.
Change: 570de7b2ffa8896b723a3d5c739cec25f4fe8297
Author: Lisa Hare <lhare@inview.co.uk>
Date : 2017-06-23 08:50:42 +0000
add older option to es-apply-settings
when older argument is passed to es_indices in Utilities.pm if days is also
specified then it will only return indexes older that the specified days
Refs #24
Change: 04a72068727474471abd709e993b9095bcf4e489
Author: Brad Lhotsky <blhotsky@craigslist.org>
Date : 2017-06-07 11:05:12 +0000
Fixup the optimize call for newer ElasticSearch releases.
Change: 25ad9e95697a92093c30375836041c7fb2f691e8
Author: Markus Linnala <Markus.Linnala@cybercom.com>
Date : 2017-04-05 19:33:52 +0000
es-alias-manager.pl: relax index name checker to allow _
----------------------------------------
version 5.3 at 2017-03-23 01:49:52 +0000
----------------------------------------
Change: 3becaead7ab98d420511e67ac55724eb19c3bd76
Author: Brad Lhotsky <blhotsky@craigslist.org>
Date : 2017-03-22 18:49:52 +0000
Version 5.3 Docs
Change: 2fb6c954347fbbd3a5c8dc372effe8e84116b908
Author: Brad Lhotsky <blhotsky@craigslist.org>
Date : 2017-03-22 18:48:01 +0000
ElasticSearch 5.2.x support and enhancements
* Elasticsearch no longer supports aliased keys in the index.
Removed the uniqueness parsing for the last section of the key name. ES
v2.x and on requires the keys be fully qualified.
* es-search.pl subaggs output adjusted
Prior version injected the sub aggregations before the values, this now
append the sub aggregations to the listed output in a more readable
manner.
* Allow aggregations to be wrapped and use it to provide date_histogram
wrappers for es-search.pl
es-search.pl now supports --interval to wrap aggregations in a
date_histogram's so you can better understand what is happening with
your data.
* es_master() and --master-only added, documentation clean up.
----------------------------------------
version 5.2 at 2016-12-29 22:44:10 +0000
----------------------------------------
Change: f9be6c93da4bb0d734440c5f5ccabef81888adc2
Author: Brad Lhotsky <blhotsky@craigslist.org>
Date : 2016-12-29 14:44:10 +0000
Fixes for 5.x
* <index>/_status deprecated since 1.2, make rewrite to _stats the
default * <index>/_stats no longer supports all parameter, it's the default
since 1.x * Replace the last call to <index>/_status in my tools with
<index>/stats
----------------------------------------
version 5.1 at 2016-11-16 02:30:42 +0000
----------------------------------------
Change: f20cbf1927d19c4aeac2c1bebf792cd77e4f7986
Author: Brad Lhotsky <blhotsky@craigslist.org>
Date : 2016-11-15 18:30:42 +0000
Added perl 5.24 testing to travis and bumped version numbers.
Change: df93facfcc57614bc9ed011a6dadea4a0510f832
Author: Brad Lhotsky <blhotsky@craigslist.org>
Date : 2016-11-15 18:26:35 +0000
Improvements to the UI and the compatibility layer
* es-search.pl now supports sub aggregations using the --with parameter *
Fix a bug in the handling of es_indices with ES 2.x clusters. * Remove the
warning from the App::ElasticSearch::Utilities::HTTPRequest
object as I can see this moving out of this module soon. *
App::ElasticSearch::Utilities::Connection handles JSON error documents *
Fix case sensitivity bug in the --top option of es-search.pl
=================================================
Plus 57 releases after 2016-09-18 00:00:00 +0000.
=================================================