Index pattern #2078
Closed
Index pattern #2078
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The "index" setting name has historical reasons, from the times that all outputs shared the same configuration options. Since the meaning and contents of the "index" setting are starting to vary per output, it's time to clean this up. The Redis "index" is now called "key". Before this change, the index was still somehow special, since it was set to the beatName outside of the output. This removes this hack, and sets index to beatName only in the Elasticsearch and Logstash modules. This also removes the `index` setting for the file output, but that was never documented. Part of elastic#2074.
tsg
added
in progress
This transforms the fixed index into a pattern, somehow similar to how
Logstash is doing it. However, logstash is using the Joda format, for which
we don't have Go libraries. So instead of writing `filebeat-%{+YYYY.MM.dd}` you
would need to write `filebeat-%{+2006.01.02}` (i.e. layout by example).
Because the Go layouts don't support ISO 8601 weeks, in order to support weekly
indices, the special `isoweek` keyword was introduced, which is the equivalent
of Joda `xxxx.ww`. The layout `filebeat-%{+isoweek}` results in `filebeat-2016.29`
for today.
Part of elastic#2074. Closes elastic#921.
|
Closing, to be reimplemented on top of the format string support, which already supports timestamps. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This transforms the fixed index into a pattern, somehow similar to how
Logstash is doing it. However, logstash is using the Joda format, for which
we don't have Go libraries. So instead of writing
filebeat-%{+YYYY.MM.dd}youwould need to write
filebeat-%{+2006.01.02}(i.e. layout by example).Because the Go layouts don't support ISO 8601 weeks, in order to support weekly
indices, the special
isoweekkeyword was introduced, which is the equivalentof Joda
xxxx.ww. The layoutfilebeat-%{+isoweek}results infilebeat-2016.29for today.
Part of #2074. Closes #921.
Also, depends on #2077.