Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[7.7][docs] Backport: Add hints based autodiscover docs for heartbeat (#14365) #17817

Merged
merged 1 commit into from
Apr 18, 2020
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
131 changes: 102 additions & 29 deletions heartbeat/docs/autodiscover-hints.asciidoc
Original file line number Diff line number Diff line change
@@ -1,42 +1,115 @@
{beatname_uc} supports autodiscover based on hints from the both Docker and Kubernetes. The hints system looks for
{beatname_uc} supports autodiscover based on hints from the provider. The hints system looks for
hints in Kubernetes Pod annotations or Docker labels that have the prefix `co.elastic.monitor`. As soon as
the container starts, {beatname_uc} will check if it contains any hints and launch the proper config for
it. Hints tell {beatname_uc} how to get logs for the given container.
it. Hints tell {beatname_uc} how to get logs for the given container. By default monitors will be created
for the container that exposes the port being requested to be monitored. You can use hints to modify this behavior. This is the full
list of supported hints:

Hints for {beatname_uc} take the form `co.elastic.monitor/MONITOR_INDEX.CONFIG_OPTION=VALUE`, where `MONITOR_INDEX` represents
the index of monitor, and `CONFIG_OPTION` is any of the <<configuration-heartbeat-options>> options, with `VALUE` being the value
you'd like to set that option to.

As an example let's convert the regular syntax {beatname_uc} monitors below into autodiscover hints.
[float]
===== `co.elastic.monitor/type`

[source,yml]
----------------------------------------------------------------------
-- heartbeat.monitors:
- type: tcp
hosts: '${data.host}:6379'
schedule: @every 10s
- type: icmp
hosts: '${data.host}'
schedule: @every 10s
----------------------------------------------------------------------
Define the monitor type to use. Ex: http, tcp, icmp

To configure this with autodiscover, add the following line to your Dockerfile to add the requisite labels.
[float]
===== `co.elastic.monitor/hosts`

----------------------------------------------------------------------
LABEL co.elastic.monitor/1.type=tcp co.elastic.monitor/1.hosts='${data.host}:6379' co.elastic.monitor/1.schedule='@every 10s'
LABEL co.elastic.monitor/2.type=icmp co.elastic.monitor/2.hosts='${data.host}' co.elastic.monitor/2.schedule='@every 10s'
----------------------------------------------------------------------
The URIs to monitor. Example:
[source,yaml]
-----
co.elastic.monitor/type: icmp
co.elastic.monitor/hosts: ${data.host}
-----

Then, add the following to `heartbeat.yml`:
This would ensure that each host has an ICMP monitor enabled on it.

[source,yml]
----------------------------------------------------------------------
[float]
===== `co.elastic.monitor/schedule`

Define the schedule on which the monitor should be executed.
-----
co.elastic.monitor/schedule: "@every 5s"
-----

[float]
===== `co.elastic.monitor/processors`

Define a processor to be added to the {beatname_uc} monitor configuration. See <<filtering-and-enhancing-data>> for the list
of supported processors.

In order to provide ordering of the processor definition, numbers can be provided. If not, the hints builder will do
arbitrary ordering:

[source,yaml]
-----
co.elastic.monitor/processors.1.drop_fields.fields: "field1, field2"
co.elastic.monitor/processors.drop_fields.fields: "field3"
-----

In the above sample the processor definition tagged with `1` would be executed first.

[float]
==== Kubernetes

Kubernetes autodiscover provider supports hints in Pod annotations. To enable it just set `hints.enabled`:

[source,yaml]
-----
heartbeat.autodiscover:
providers:
- type: docker
hints.enabled: true
----------------------------------------------------------------------
- type: kubernetes
hints.enabled: true
-----

You can annotate Kubernetes Pods with useful info to spin up {beatname_uc} monitors:

Then, start {beatname_uc}. This should discover the container you built and ping it once with ICMP, another time with TCP.
[source,yaml]
-----
annotations:
co.elastic.monitor/type: icmp
co.elastic.monitor/hosts: ${data.host}
co.elastic.monitor/schedule: "@every 5s"
-----


[float]
===== Multiple containers

When a pod has multiple containers, the settings are shared unless you add the container name in the
hint. For example, these hints configure the container exposing port 8080 to do a HTTP check and have the `sidecar`
container to have an TCP check.


[source,yaml]
-----
annotations:
co.elastic.monitor/type: http
co.elastic.monitor/hosts: ${data.host}:8080/healtlz
co.elastic.monitor/schedule: "@every 5s"
co.elastic.monitor.sidecar/type: tcp
co.elastic.monitor.sidecar/hosts: ${data.host}:8081
co.elastic.monitor.sidecar/schedule: "@every 5s"
-----



[float]
==== Docker

Docker autodiscover provider supports hints in labels. To enable it just set `hints.enabled`:

[source,yaml]
-----
heartbeat.autodiscover:
providers:
- type: docker
hints.enabled: true
-----

You can label Docker containers with useful info to spin up {beatname_uc} monitors similar to the Kubernetes example:
----------------------------------------------------------------------
LABEL co.elastic.monitor/1.type=tcp co.elastic.monitor/1.hosts='${data.host}:6379' co.elastic.monitor/1.schedule='@every 10s'
LABEL co.elastic.monitor/2.type=icmp co.elastic.monitor/2.hosts='${data.host}' co.elastic.monitor/2.schedule='@every 10s'
----------------------------------------------------------------------

For a full list of variables available, see the provider-specific docs in <<configuration-autodiscover>>.