Cherry-pick #16081 to 7.x: Elasticsearch index must be lowercase #16142
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Index names must be lowercase When indexing into Elasticsearch index names must always be lowercase. If the index or indices setting are configured to produce non-lowercase strings (e.g. by extracting part of the index name from the event contents), we need to normalize them to be lowercase. This change ensure that index names are always converted to lowercase. Static strings are converted to lowercase upfront, while dynamic strings will be post-processed. * update kafka/redis/LS output to guarantee lowercase index * add godoc (cherry picked from commit 7ddcb1e)
kvch
reviewed
Feb 6, 2020
CHANGELOG.next.asciidoc
Outdated
| @@ -71,6 +71,8 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d | |||
| - Fix issue where default go logger is not discarded when either * or stdout is selected. {issue}10251[10251] {pull}15708[15708] | |||
| - Fix issue where TLS settings would be ignored when a forward proxy was in use. {pull}15516{15516} | |||
| - Remove superfluous use of number_of_routing_shards setting from the default template. {pull}16038[16038] | |||
| - Upgrade go-ucfg to latest v0.8.1. {pull}15937{15937} | |||
There was a problem hiding this comment.
This entry does not belong here.
kaiyan-sheng
approved these changes
Feb 6, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cherry-pick of PR #16081 to 7.x branch. Original message:
What does this PR do?
This change ensure that index names are always converted to lowercase.
Static strings are converted to lowercase upfront, while dynamic strings
will be post-processed.
Why is it important?
When indexing into Elasticsearch index names must always be lowercase.
If the index or indices setting are configured to produce non-lowercase
strings (e.g. by extracting part of the index name from the event
contents), we need to normalize them to be lowercase.
Checklist
- [ ] I have made corresponding changes to the documentation- [ ] I have made corresponding change to the default configuration filesAuthor's Checklist
How to test this PR locally
Index with different index/indices configurations into Elasticsearch:
indexsettingindiceswhere the target index is upper caseRun same checks with Redis, Kafka, Logstash outputs (the index name should consistently be lower case for all outputs).
Related issues
Use cases
Screenshots
Logs