Skip to content

Commit

Permalink
Heartbeat event format (#4091)
Browse files Browse the repository at this point in the history 
See PR #4091 for detailed list of event format changes and sample events.

- export transport.DialWith
- update heartbeat look:
  - add look.Status
  - add some missing godoc
- rewrite heartbeat dialchain package
  - simplify package
  - have connection layers add standardized connection data to the final event
  - add some more helpers
  - add some godocs
- update event format. See PR #4091
- add more detailed duration measure to HTTP module
- update job settings
  - introduce explicit structs for job settings
  - job settings can contain static monitor fields to be added to every event
  - helpers can add more static monitor fields to settings
- update fields.yml structure
- update kibana dashboard
  • Loading branch information
@ruflin
Steffen Siering authored and ruflin committed Apr 28, 2017
1 parent 1344e4f commit 6b01fa9
Show file tree
Hide file tree
Showing 27 changed files with 1,321 additions and 549 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ https://github.com/elastic/beats/compare/v5.1.1...master[Check the HEAD diff]
- Add Icinga module. {pull}3904[3904]

*Heartbeat*
- Event format and field naming changes in Heartbeat and sample Dashboard. {pull}4091[4091]

*Metricbeat*
- Linux cgroup metrics are now enabled by default for the system process
Expand Down
335 changes: 243 additions & 92 deletions heartbeat/_meta/fields.yml
View file
Original file line number Diff line number Diff line change
@@ -1,132 +1,283 @@
- key: common
title: "Common monitoring fields"
title: "Common Heartbeat Monitor"
description:
fields:
- name: type
type: keyword
required: true
description: >
The monitor type.
- name: monitor
type: keyword
type: group
description: >
Monitor job name.
Common monitor fields.
- name: scheme
type: keyword
description: >
Address url scheme. For example `tcp`, `ssl`, `http`, and `https`.
fields:
- name: type
type: keyword
description: >
The monitor type.
- name: host
type: keyword
description: >
Hostname of service being monitored. Can be missing, if service is
monitored by IP.
- name: name
type: keyword
description: >
The monitors configured name
- name: port
type: integer
description: >
Service port number.
- name: id
type: keyword
description: >
The monitors full job ID as used by heartbeat.
- name: url
type: text
description: >
Service url used by monitor.
- name: duration
type: group
description: total monitoring test duration
fields:
- name: us
type: long
description: Duration in microseconds

- name: ip
type: ip
description: >
IP of service being monitored. If service is monitored by hostname,
the `ip` field contains the resolved ip address for the current host.
- name: scheme
type: keyword
description: >
Address url scheme. For example `tcp`, `tls`, `http`, and `https`.
- name: duration
type: group
description: total monitoring test duration
fields:
- name: us
type: long
description: Duration in microseconds
- name: host
type: keyword
description: >
Hostname of service being monitored. Can be missing, if service is
monitored by IP.
- name: resolve_rtt
type: group
description: Duration required to resolve an IP from hostname.
fields:
- name: us
type: long
description: Duration in microseconds
- name: ip
type: ip
description: >
IP of service being monitored. If service is monitored by hostname,
the `ip` field contains the resolved ip address for the current host.
- name: icmp_rtt
type: group
description: ICMP Echo Request and Reply round trip time
fields:
- name: us
type: long
description: Duration in microseconds
- name: status
required: true
type: keyword
description: >
Indicator if monitor could validate the service to be available.
- name: tcp_connect_rtt
- key: resolve
title: "Host Lookup"
description:
fields:
- name: resolve
type: group
description: >
Duration required to establish a TCP connection based on already
available IP address.
Host lookup fields.
fields:
- name: us
type: long
description: Duration in microseconds
- name: host
type: keyword
description: >
Hostname of service being monitored.
- name: ip
type: ip
description: >
IP address found for the given host.
- name: socks5_connect_rtt
- name: rtt
type: group
description: Duration required to resolve an IP from hostname.
fields:
- name: us
type: long
description: Duration in microseconds

- key: icmp
title: "ICMP"
description:
fields:
- name: icmp
type: group
description: >
Time required to establish a connection via SOCKS5 to endpoint based on available
connection to SOCKS5 proxy.
IP ping fields.
fields:
- name: us
type: long
description: Duration in microseconds
- name: requests
type: integer
description: >
Number if ICMP EchoRequests send.
- name: tls_handshake_rtt
- name: rtt
type: group
description: ICMP Echo Request and Reply round trip time
fields:
- name: us
type: long
description: Duration in microseconds

- key: tcp
title: "TCP Layer"
description:
fields:
- name: tcp
type: group
description: >
Time required to finish TLS handshake based on already available network
connection.
TCP network layer related fields.
fields:
- name: us
type: long
description: Duration in microseconds
- name: port
type: integer
description: >
Service port number.
- name: rtt
type: group
description: >
TCP layer round trip times.
fields:
- name: connect
type: group
description: >
Duration required to establish a TCP connection based on already
available IP address.
fields:
- name: us
type: long
description: Duration in microseconds

- name: validate
type: group
description: >
Duration of validation step based on existing TCP connection.
fields:
- name: us
type: long
description: Duration in microseconds

- name: http_rtt
- key: socks5
title: "SOCKS5 Proxy"
description:
fields:
- name: socks5
type: group
description: >
Time required between sending the HTTP request and first by from HTTP
response being read. Duration based on already available network connection.
SOCKS5 proxy related fields:
fields:
- name: us
type: long
description: Duration in microseconds
- name: rtt
type: group
description: >
TLS layer round trip times.
fields:
- name: connect
type: group
description: >
Time required to establish a connection via SOCKS5 to endpoint
based on available connection to SOCKS5 proxy.
fields:
- name: us
type: long
description: Duration in microseconds


- name: validate_rtt
- key: tls
title: "TLS Encryption Layer"
description:
fields:
- name: tls
type: group
description: >
Time required for validating the connection if connection checks are configured.
TLS layer related fields.
fields:
- name: us
type: long
description: Duration in microseconds
- name: rtt
type: group
description: >
TLS layer round trip times.
fields:
- name: handshake
type: group
description: >
Time required to finish TLS handshake based on already available network
connection.
fields:
- name: us
type: long
description: Duration in microseconds

- name: response
- key: http
title: "HTTP Monitor"
description:
fields:
- name: http
type: group
description: >
Service response parameters.
HTTP related fields.
fields:
- name: status
type: integer
- name: url
type: text
description: >
Response status code.
Service url used by monitor.
- name: up
required: true
type: boolean
description: >
Boolean indicator if monitor could validate the service to be available.
- name: response
type: group
description: >
Service response parameters.
fields:
- name: status
type: integer
description: >
Response status code.
- name: rtt
type: group
description: >
HTTP layer round trip times.
fields:
- name: validate
type: group
description: |
Duration between first byte of HTTP request being written and
response being processed by validator. Duration based on already
available network connection.
Note: if validator is not reading body or only a prefix, this
number does not fully represent the total time needed
to read the body.
fields:
- name: us
type: long
description: Duration in microseconds

- name: validate_body
type: group
description: |
Duration of validator required to read and validate the response
body.
Note: if validator is not reading body or only a prefix, this
number does not fully represent the total time needed
to read the body.
fields:
- name: us
type: long
description: Duration in microseconds

- name: write_request
type: group
description:
Duration of sending the complete HTTP request. Duration based on
already available network connection.
fields:
- name: us
type: long
description: Duration in microseconds

- name: response_header
type: group
description:
Time required between sending the start of sending the HTTP
request and first by from HTTP response being read. Duration
based on already available network connection.
fields:
- name: us
type: long
description: Duration in microseconds

- name: total
type: group
description: |
Duration required to process the HTTP transaction. Starts with
the initial TCP connection attempt. Ends with after validator
did check the response.
Note: if validator is not reading body or only a prefix, this
number does not fully represent the total time needed.
fields:
- name: us
type: long
description: Duration in microseconds
Loading

0 comments on commit 6b01fa9

Please sign in to comment.