==================================================
Changes from 2017-05-26 00:00:00 +0000 to present.
==================================================
----------------------------------------
version 6.6 at 2019-05-26 21:14:01 +0000
----------------------------------------
Change: 762f7e14e368ade8b947f92563023a5a416ae152
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2019-05-26 14:12:14 +0000
Dccoumentation regenerated for 6.6 release
Change: d9d69b0e228ee5d07c7a8b18ee6051044192e6d5
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2019-05-26 13:29:52 +0000
Fix documentation of es_index_fields()
Change: 9b032b8ee358990d929f7c120a8edc160151df19
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2019-05-26 13:26:50 +0000
Use field metadata to automatically set timestamp
Since we have the field metadata, we can use that metadata to autodiscover
the date field for sorting.
Change: 63e56193f8eef5f8176843be49709004cec10eaa
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2019-05-26 13:24:37 +0000
es_index_fields() patched to return field metadata
Instead of just flagging the presence of a field, the field types are
returned along with other meta-data including if the field is nested.
`es-search.pl` now displays field data types and whether a field is
contained within a nested path.
Change: 8202a0722d7a29e5fc6dc701806e8ced5dafcc14
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2019-05-21 20:24:47 +0000
Fix formatting
Change: 38aad622f61630b8da086364c7aea778e10081f1
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2019-05-21 20:20:13 +0000
Regenerate .mkdown prepping for the 6.6 release
Change: 2770f819958265a79677c0c0d200ebbf9c5fc7b5
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2019-05-21 20:16:43 +0000
Fix the defined or so 0 isn't translated to -1
Change: ccbcb6c100f218b1c60e619a90d2cd57ac63f49e
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2019-05-21 20:13:41 +0000
Add .json support to the ::QueryString::FileExpansion plugin
Change: 69a1b7dc9d4e4bed31bf0ed103e3ff2e98c78a38
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2019-05-21 08:38:41 +0000
Documentation Update and set default --days to 1
Reviewing the documentation found some spelling errors and incorrect
wording. Fixed that up. Also changed the default --days a query runs
against to 1 as that's a better default setting.
Change: d91a9c6a8815a326eafc26f497dc282e9f27c46a
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2019-05-19 12:26:56 +0000
Documentation consistency patches
Change: 06b670f490597d78951fdd5b784f8168f6c98cc7
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2019-05-19 12:12:08 +0000
Patch documentation for INSTALL to be secure
----------------------------------------
version 6.5 at 2019-05-08 02:42:02 +0000
----------------------------------------
Change: 0db6a92889a94daf54f17432b53f26a91153e4f7
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2019-05-07 19:42:02 +0000
Docs ready for the 6.5 release
Change: b0ae2e1c9a859b37c8345f1cd24152f6c74c14ef
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2019-05-07 19:41:07 +0000
Fix priority for ::QueryString::AutoEscape
Since this plugin is keyed to '=<field>:<value>', I want to make sure it
fires before the nested plugin and before any other expansion plugins to
avoid issues.
Change: 044643c4fcb1183b1ffa46a3575bd01ab028e048
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2019-03-25 21:45:24 +0000
Collect node local shard dimensions and safe guard some calls to be
compatible with ES 5.x and 6.x
Change: b13c4a5eb6861898d3b35d567da19838372fadcc
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2019-03-25 17:30:36 +0000
Report number of indexes with a block set to true.
Change: b037da82daf1f23f4940cbb0955e8beaecb69e1b
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2019-01-30 21:29:48 +0000
Fix the es-copy-index.pl to migrate indices from other clusters.
----------------------------------------
version 6.4 at 2019-01-02 22:20:36 +0000
----------------------------------------
Change: 470feb7d6e00998906849cbfb49d1904eb79cf6b
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2019-01-02 14:20:36 +0000
Updated Documentation for the 6.4 release
Change: 095da97cc9e9fb29ebcea13851d548e20c6166d6
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2019-01-02 13:40:05 +0000
Add in the new QueryString Extensions to the documentation
Change: 714c9856d382bcc1ee95fc2ecd8ee7257e13aaec
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2019-01-02 12:12:56 +0000
Scroll search corrected to use the correct API
Finally migrated off the 0.x version of the scrolled search API which is no
longer supported in 6.5.
Change: 4a9e798426a21f2ab3543317bea5a56937536b56
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2019-01-02 11:18:28 +0000
Convert ::QueryString::AutoEscape to use terms filter
Rather than attempting to escape characters, use the Elastic `terms` filter
to perform the lookup if the user specifies `=field:value`.
This will convert to:
{ terms: { field: 'value' } }
And that will perform an exact match to the documents in a more predictable
and future-proof manner.
----------------------------------------
version 6.3 at 2019-01-01 01:00:09 +0000
----------------------------------------
Change: 0a032698e44212328cb3bef1acae61812db12353
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-12-31 17:00:09 +0000
Fixes for the 6.x version of ElasticSearch
* Ensure we set a Content-Type header for requests sending JSON * Handle
special casing for index creation in ::Connection * Ensure all _bulk
requests end with a "\n" * Fix missing Ref::Util import in es-copy-index.pl
* es-copy-index.pl: only send _mappings if defined
Change: 4683229bdb8811d7d1122f4a9bd665c61d84f23d
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-12-31 16:49:20 +0000
Fix authordeps
Change: 75b43dee4c8313e16891e2bb7bfe8e44b99593a9
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-12-31 16:45:25 +0000
Reset author/copyright_holder because my git settings trip up the
auto-discovery.
Change: 510aa532e05a9c2cb44af384145c2beaa06c65e2
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-12-27 22:37:01 +0000
Fix up the AutoEscape plugin to run at the correct priority
Change: 90d61d84d2c2ece75b7bc5d13690ca1b282c7ad7
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-12-27 22:20:31 +0000
Added an AutoEscape Query String Parser
While searching for UserAgents or file paths, you need to know the secret
Lucene handshakes to match as you expect. This plugin automatically
converts common characters users may not expect to cause special behavior.
----------------------------------------
version 6.2 at 2018-12-21 02:53:52 +0000
----------------------------------------
Change: 81bae677ac93da9eba1c2a3b4859eded645782f9
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-12-20 18:53:52 +0000
Documentation and release fixes
* Removed Test::Coverage::TrustPod and Test::Critic from requires and
moved them to the authordeps sections * Fixed Running of Extra Tests on
Release * Added a perlcritic.rc
Change: 0eab9bf3babfb37064e4af13c579e42013a45057
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-12-20 18:37:13 +0000
Add 'jsonpretty' as a format
When using `--format 'json'`, default to setting `--no-decorators` since
we'll mostly use this with `jq`.
Change: 1f6a51a1cca972d441fc9aca2dfd3169c815a87f
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-12-20 18:21:56 +0000
Output tweaks for better usability
* Added `--no-decorators` option as an override for `--no-header` * When
using `--no-decorators`, disable outputting the footers as well * Only
display summary information when we're dealing with multi-day output * When
an agg is `terms` or `significant_terms` and no other collision
exists, suppress the `$type` prefix to be prettier
Change: 1a8a5394e2c99f4cf59d670d36705ce06b41ae92
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-09-26 17:28:02 +0000
Allow the user to change the max-batch-size/scroll size
Change: 298902bf52642fcc081059f9f85853ba7449d865
Author: Mohammad S Anwar <mohammad.anwar@yahoo.com>
Date : 2018-07-21 10:00:45 +0000
Autogenerate META.yml as suggested by CPANTS.
----------------------------------------
version 6.1 at 2018-09-18 03:10:27 +0000
----------------------------------------
Change: 020fddc36e35230b1a275c87c8edddae2fbb7c7b
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-09-17 20:10:27 +0000
Updates for the documentation for 6.1 release
Change: 263e77f276c95dbf1f90bba6b77264547e4b7390
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-09-17 20:09:05 +0000
Add --bg-filter option for es-search.pl
If significant_terms is used for the --top aggregation, this option is able
to inject a background_filter into the significant_terms aggregation.
Change: e76897d8eefee23d70b9fd5b3e6bd26bd7494b95
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-09-17 20:08:24 +0000
Update to using GitHub::Meta and rearrange Pod::Weaver configs
----------------------------------------
version 6.0 at 2018-07-28 00:45:25 +0000
----------------------------------------
Change: e58dcf2f190a1db3e1e10f8799b67c7f9a8abcc4
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-27 17:45:25 +0000
Fix a bug in conditional checks for index age when no date is found in the
index name
Change: f6e05fcf90fa575f5e5b5ccafb0bf94667ef9bd5
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-23 14:31:41 +0000
New es_local_index_meta() function to allow looking up the timestamp field
in es-search.pl much easier and more straight-forward.
----------------------------------------
version 5.9 at 2018-07-21 01:10:38 +0000
----------------------------------------
Change: 9b00c64735a21b74223fddcde44fff74772efd17
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-20 18:10:38 +0000
Bumping Release to 5.9
Change: 415c84da87571aa488a0f246d423e24fbf446e00
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-18 17:58:55 +0000
Fix another uninitialized error in the code
----------------------------------------
version 5.8 at 2018-07-16 19:14:45 +0000
----------------------------------------
Change: 3f276cdc5e5b8a1fc6215275b8b3c24e3d8ef066
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-16 12:14:45 +0000
Docs update for 5.8 release
Change: 12336ccb1ee03eaa294af46a1340af3cbb47f4c1
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-16 11:45:58 +0000
Fix a warning and improve output
* Fix a "use of uninitialized" warning for when an index is missing a
date string but matches the base pattern * Fix logic bug that meant
'--delete --delete-days 1' was a noop because
of using '<' instead of '<='
Change: e3b0899af37835e7e9d372788c31339cc3cf8f01
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-16 11:43:38 +0000
Bubble up errors so we know why we failed
Change: 9a07d69d5a7571d58951682c3d190083cf0a84df
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-16 11:25:16 +0000
Show shards breakdown by primary and replica
----------------------------------------
version 5.7 at 2018-07-06 05:34:05 +0000
----------------------------------------
Change: e90729b13bb4fc45d68e9711ac139c8f8ab1dad8
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-05 22:34:05 +0000
Documentation Updates for 5.7 release
Change: 7498a4ccea72cd4cc8860316bf3aa175e278c158
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-05 22:23:55 +0000
Add support for statistical aggregations via --with
Change: 8a86345e449b355257358de7f81414d278442a38
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-05 22:05:43 +0000
Fix joins when using "NOT" for negations
Change: 4e103a8feb362f9b5722736c3fdda66d9f36819a
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-05 20:20:40 +0000
Fixup broken query strings without joining keywords
This addresses the most used case where a query_string is missing the
keyword 'AND' in the token joining. This also makes it easier to produce
longer queries with less thinking.
Change: 1b43ea3120ae6ff51bc9f0d7c04e46f05499e7bb
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-02 23:13:54 +0000
Fix documentation for the ::Ranges plugin
Change: c7cb7c238a01c235823161c7d4455afe00665870
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-02 22:58:21 +0000
Add a --filter option to es-search.pl
Make use of the newly exposed 'filter' context from ::QueryString and add
positive matches as either 'must' (query context) or 'filter' (filter
context).
Change: 498af1d1759cf2788a06526a847d331e81e028d6
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-02 22:55:00 +0000
Allow toggling the context between 'query' and 'filter'
By overriding the ::QueryString context, it's possible to toggle between
collecting the positive matches as 'filter' elements instead of 'must'.
This allows the use of unscored queries which may improve performance for
some queries dramatically.
Change: 601bb7eea49cc3a8dbe0c9579c6a0b94e5675824
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-02 22:53:07 +0000
Early return for ::QueryString:: plugins
Both the ::Nested and ::Ranges plugins would emit uninitialized warnings
when run due to forgetting to check if the token matched the plugin.
Change: 67342d859955cc6f4ca4b6a75363beb08f7af1d8
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-02 21:41:22 +0000
Fix parameter specification in the library too
Change: 1039688ce0316e52f73f3ff57ec7b2ddedf0636b
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-02 21:36:26 +0000
Fix the parameter specifications and remove unsupported script
Change: 60aaa2dc1ea4ad19f0d5837a3620e9f98029196e
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-02 21:21:54 +0000
Add ::QueryString::Ranges
Translates queries with field:<,<=,>,>-= to equivalent range queries
Change: ca4e73f856bd9f48818e3fd2a7b959998dee8047
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-02 08:58:05 +0000
Fix --base in es-search.pl, make --index less policey.
* --base was broken as es-search.pl is looking for open indices.
Prepend the wildcard to the '/_stats/docs' request. * Don't validate the
--index parameter, instead pass it through. This
gives the user the power to create their own index strings
Change: a880e6ff664b99e64da8d18589f66bc51f1e089c
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-07-02 08:58:05 +0000
Replace es-storage-data.pl with es-storage-overview.pl
Change: 7bb0c293fd3f433183002c75be6a7e0e6e0f53fc
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-06-18 20:49:18 +0000
Remove size parameter if set to zero.
For statistical aggregations, the size parameter is invalid. This is a
shortcut to remove the parameter from the sub aggregation.
Change: 1e6372612b5e86627f6df2703c8a49e344bd129d
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-06-08 16:13:52 +0000
Fix failing tests
Change: 3053cd3ee7c1f30138f5fe2f51b6e77dc6639bba
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-05-10 21:40:51 +0000
Make the fast things fast
* Reorder the --bases for es-search.pl to skip index meta checks. * Use
es_indices(check_date => 0) instead of _all => 1 to skip closed
indices. * Make es_index_bases() simpler, i.e.:
Old behavior:
(a, a-b, a-b-c, b-c, c) = es_index_bases('a-b-c');
New behavior:
(a, a-b, a-b-c) = es_index_bases('a-b-c');
* Use _stats/docs to resolve all index names as indexes without explicit
aliases don't show up in all versions of ElasticSearch.
Change: ce08b1c0cc3c9432780612d13dd1214dc60be04e
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-05-10 21:06:34 +0000
New script: es-storage-overview.pl
Implements the simplest possible overview of storage. Does not provide node
or index data, but gives a high level overview.
Change: 3837060be0e0145b2e23987da6fab11af9d54a4e
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-05-10 20:29:24 +0000
es-search.pl - more concise output
When we encounter a record that is an array, but is holding only 1 item,
derefernce and extract the single value.
Change: b2473b3b412444a8148dfd96b0a07eb0dc466054
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-04-30 19:57:37 +0000
Made es_request() errors fatal and sped up index lookup
----------------------------------------
version 5.6 at 2018-04-13 18:20:03 +0000
----------------------------------------
Change: fcb981c7ea59b5e3ac5f9c5b2ae3cf34e1fad5a3
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-04-13 20:20:03 +0000
Documentation fixup with the latest pod tooling
Change: 2d9fce9813a585ff3115bfdce4b6b46a09192bf7
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-04-13 10:57:03 +0000
Don't track the local TODO file
Change: 7727fd81af17556c54e9c911d507efcd20266765
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-04-13 10:54:44 +0000
Ready for 5.6 Release
* Remove overzealous debugging messages * Tweak es-storage-data.pl to
provide full details via --all * Change the way
es-daily-index-maintenance.pl complains when --master--only is set. *
Documentation updated
Change: 1b06f8ce6a574e34acea51938193d7590c6c24f9
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-03-17 00:49:08 +0000
Docs updated added --view base to es-storage-data.pl
Change: 0cf82d70c23104b590f29b4551612578541cbb6a
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-03-17 00:39:18 +0000
Get es-storage-data.pl working on modern ElasticSearch
----------------------------------------
version 5.5 at 2018-03-17 06:01:25 +0000
----------------------------------------
Change: 757dec9e648d077759da9a19ebdf3ed343df82f1
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-03-16 23:01:25 +0000
Remove no longer necessary test
Change: d63a1b823fcd72116e731cbfc45387290b60e769
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-03-16 22:49:15 +0000
Documentation updated for 5.5 release
Change: b1709e428bf9031e99b196ed6c865b02535ddb66
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-03-16 22:43:04 +0000
Bugfix and better replica management
* BUGFIX: es_index_strip_date() now removes everything after the date
string.
es-daily-index-maintenance.pl
* --replicas-* options aliased to --replica-* * Smarter handling of math
around replica calculation * Disable --bloom option entirely as it's
deprecated by Elastic
Change: aabba5ad9a1c26ee5aa631f4ea81d887bf98463a
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-02-21 23:04:18 +0000
Pod and Perl::Critic fixes
Change: 1dcaaa7ea8b7b8b89387692dd969a18fd385bd23
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-02-21 20:58:15 +0000
Conversion to Ref::Util and Types::Tiny
Fix paths for the shebang line so the installer will be able to install the
scripts with the correct Perl binary path. Add a first pass at nested query
syntax
Change: 722e812a7365f9a81aa0d1df5d61a3a1dcb695ed
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2018-02-21 20:52:06 +0000
Update the QS::IP expander to make it more intelligent.
If a field ends in *_ip, allow CIDR queries against the field. This skips
renaming the field from "src_ip" to "src_ip_numeric" and is now an inline
replacement. This enhances simplifies the CLI queries.
The docs were not accurate, so now the example and documentation are valid.
Change: 8e7e34fe1690349a0485ec271c85e966c699b0d9
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2017-12-03 23:42:00 +0000
Disallow tabs in my files.
Change: 938d5f9e9bf387d304388bf524348dd4b0d51fac
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2017-10-25 17:07:47 +0000
Allow --show to be more flexible
* Unbind from the index fields since this operates on the source
document * Change to an array so you can add multiple --show's
----------------------------------------
version 5.4 at 2017-10-13 19:36:20 +0000
----------------------------------------
Change: c31b5f39953589c497f4374be9c22f36a2514dc2
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2017-10-13 12:36:20 +0000
Documentation update for the 5.4 release.
Change: 3b6a379937869eaf46e80eea6f7d54e5c4f2220a
Author: Brad Lhotsky <brad@divisionbyzero.net>
Date : 2017-07-12 20:50:57 +0000
Default to UTC, warn when using incompatible options.
Change: 570de7b2ffa8896b723a3d5c739cec25f4fe8297
Author: Lisa Hare <lhare@inview.co.uk>
Date : 2017-06-23 08:50:42 +0000
add older option to es-apply-settings
when older argument is passed to es_indices in Utilities.pm if days is also
specified then it will only return indexes older that the specified days
Refs #24
Change: 04a72068727474471abd709e993b9095bcf4e489
Author: Brad Lhotsky <blhotsky@craigslist.org>
Date : 2017-06-07 11:05:12 +0000
Fixup the optimize call for newer ElasticSearch releases.
Change: 25ad9e95697a92093c30375836041c7fb2f691e8
Author: Markus Linnala <Markus.Linnala@cybercom.com>
Date : 2017-04-05 19:33:52 +0000
es-alias-manager.pl: relax index name checker to allow _
=================================================
Plus 60 releases after 2017-05-26 00:00:00 +0000.
=================================================