A set of Elasticsearch index templates designed to store and enable querying of Elasticsearch operational statistics. The templates are specifically designed to work well in combination with watches CLI tool (see below).
Provided index templates assume JSON data is formatted as
nested data.
JSON data provided by Elasticsearch stats REST endpoints may need transformation to be nested.
If you use watches to poll data from Elasticsearch then you can use
--transform
option to handle the data transformation accordingly.
There are provided useful scripts
and fluentd
configuration demonstrating how to setup
and use index templates correctly.
In this tutorial we will setup Elasticsearch cluster to poll operational stats from and store this data back to the same cluster. More specifically, we will be polling ES REST APIs by watches to log files. These log files are then tailed by fluentd and pushed back to the same Elasticsearch cluster. This creates infamous feedback-loop and is not recommended setup for production but is ok for the demo. For production it is recommended to point the fluentd to different Elasticsearch cluster.
+-----------+
+-----| watches |-----+
| +-----------+ |
| |
+-------------------+ V
| | +-------------+
| Elasticsearch | +-------------+ |
| | | Log files |-+
| (index templates) | +-------------+
+-------------------+ |
A |
| +-----------+ |
+-----| fluentd |-----+
+-----------+
- the latest
watches
cli tool (1.0.2-dev
or higher), see watches docs for installation instructions - installed Elasticsearch 2.4.4, either download it manually or install using package manager
- installed
fluentd
andfluent-plugin-elasticsearch
, see fluentd docs for more details - (optional) configure
ES_BIN
env variable to point tobin/elasticsearch
startup script
After Elasticsearch, fluentd and watches is installed you can follow instructions:
# ----------------------------------
# Clone git repo and navigate to 'scripts' folder
# ----------------------------------
git clone https://github.com/lukas-vlcek/watches_index_templates.git
cd watches_index_templates/scripts
# ----------------------------------
# Start and initialize Elasticsearch
# ----------------------------------
# Make sure ${ES_BIN} points to valid Elasticsearch startup script.
# There is no need to setup ${ES_BIN} if 'elasticsearch' CLI command works OOB.
${ES_BIN} --version
$ Version: 2.4.4, Build: fcbb46d/2017-01-03T11:33:16Z, JVM: 1.8.0_65
# Start local Elasticsearch node
./start_elasticsearch.sh
# Push index templates
./push_templates.sh
# ----------------------------------
# Start fluentd
# ----------------------------------
# This process will tail log files (these are populated by watches, see below)
# and push the data back to Elasticsearch cluster.
./start_fluentd.sh
# ----------------------------------
# Start watches processes
# ----------------------------------
# Watches will start polling ES stats and store the data to log files.
# By default watches processes will run 60 seconds and then terminate.
# You can change this duration by passing a number as the first parameter to this script.
#
# Example: ./start_watches.sh 10
# This will terminate all processes after 10 seconds.
# You can also terminate this script manually at any time.
./start_watches.sh
Scripts start_elasticsearch.sh
, push_templates.sh
and start_fluentd.sh
share common configuration file
_configure_env.sh
which can be adjusted or included env variables can be setup to override the default values. These scripts also
accept -v
parameter to force debug output.
To get more realistic stats data from Elasticsearch you can consider running parallel intensive indexing task, for example you can use stream2es tool to index Wikipedia dump.
See optional indexer documentation for more details.
Indexed data can be queried to get more insight into Elasticsearch cluster operational statistics. Stay tuned for more information.
Copyright 2017 Lukáš Vlček
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.