From 8a4d743dae084277af8a4ca1dd7f75d04bbab63d Mon Sep 17 00:00:00 2001 From: Andrew Cholakian Date: Mon, 2 Dec 2019 16:02:07 -0600 Subject: [PATCH 1/3] [Heartbeat] Add changelog/docs for new scheduler We added a new scheduler in https://github.com/elastic/beats/commit/fa6344553bd0d0b712023150a68cf4374f706e9b but forgot to add the docs / changelog. This fixes that. --- CHANGELOG.next.asciidoc | 3 +++ heartbeat/docs/heartbeat-options.asciidoc | 7 +++++++ 2 files changed, 10 insertions(+) diff --git a/CHANGELOG.next.asciidoc b/CHANGELOG.next.asciidoc index a132b7159b0e..43e607262cc8 100644 --- a/CHANGELOG.next.asciidoc +++ b/CHANGELOG.next.asciidoc @@ -54,6 +54,8 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d - Removed `host.name` field that should never have been included. Heartbeat uses `observer.*` fields instead. {pull}14140[14140] - Changed default user-agent to be `Elastic-Heartbeat/VERSION (PLATFORM_INFO)` as the current default `Go-http-client/1.1` is often blacklisted. {pull}14291[14291] - JSON/Regex checks against HTTP bodies will only consider the first 100MiB of the HTTP body to prevent excessive memory usage. {pull}14223[pull] +- Heartbeat now starts monitors scheduled with the '@every X' syntax instantaneously on startup, rather than waiting for the +given interval to pass before running them. {pull}14890[14890] *Journalbeat* @@ -216,6 +218,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d - Fix integer comparison on JSON responses. {pull}13348[13348] - Fix storage of HTTP bodies to work when JSON/Regex body checks are enabled. {pull}14223[14223] - Fix recording of SSL cert metadata for Expired/Unvalidated x509 certs. {pull}13687[13687] +- The heartbeat scheduler no longer drops scheduled items when under very high load causing missed deadlines. {pull}14890[14890] *Journalbeat* diff --git a/heartbeat/docs/heartbeat-options.asciidoc b/heartbeat/docs/heartbeat-options.asciidoc index 317be236bffb..2c889c122cee 100644 --- a/heartbeat/docs/heartbeat-options.asciidoc +++ b/heartbeat/docs/heartbeat-options.asciidoc @@ -139,6 +139,13 @@ started. The `schedule` option uses a cron-like syntax based on https://github.com/gorhill/cronexpr#implementation[this `cronexpr` implementation], but adds the `@every` keyword. +For stats on the execution of scheduled tasks you can enable the HTTP stats server with `http.enabled: true` in heartbeat.yml, then run `curl http://localhost:5066/stats | jq .heartbeat.scheduler` to view the scheduler's stats. Stats are provided for both jobs and tasks. Each time a monitor is scheduled is considered to be a single job, while portions of the work a job does, like DNS lookups and executing network requests are defined as tasks. The stats provided are: + +* **jobs.active:** The number of actively running jobs/monitors.b +* **jobs.missed_deadline:** The number of jobs that executed after their scheduled time. This can be caused either by overlong long timeouts from the previous job or high load preventing heartbeat from keeping up with work. +* **tasks.active:** The number of tasks currently running. +* **tasks.waiting:** If the global `schedule.limit` option is set, this number will reflect the number of tasks that are ready to execute, but have not been started in order to prevent exceeding `schedule.limit`. + [float] [[monitor-ipv4]] ==== `ipv4` From a0dd59eaa001371ea34fffe15fa4cafe617df544 Mon Sep 17 00:00:00 2001 From: Andrew Cholakian Date: Mon, 2 Dec 2019 16:11:40 -0600 Subject: [PATCH 2/3] Fix typo --- heartbeat/docs/heartbeat-options.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/heartbeat/docs/heartbeat-options.asciidoc b/heartbeat/docs/heartbeat-options.asciidoc index 2c889c122cee..3aa6356e45bd 100644 --- a/heartbeat/docs/heartbeat-options.asciidoc +++ b/heartbeat/docs/heartbeat-options.asciidoc @@ -141,7 +141,7 @@ but adds the `@every` keyword. For stats on the execution of scheduled tasks you can enable the HTTP stats server with `http.enabled: true` in heartbeat.yml, then run `curl http://localhost:5066/stats | jq .heartbeat.scheduler` to view the scheduler's stats. Stats are provided for both jobs and tasks. Each time a monitor is scheduled is considered to be a single job, while portions of the work a job does, like DNS lookups and executing network requests are defined as tasks. The stats provided are: -* **jobs.active:** The number of actively running jobs/monitors.b +* **jobs.active:** The number of actively running jobs/monitors. * **jobs.missed_deadline:** The number of jobs that executed after their scheduled time. This can be caused either by overlong long timeouts from the previous job or high load preventing heartbeat from keeping up with work. * **tasks.active:** The number of tasks currently running. * **tasks.waiting:** If the global `schedule.limit` option is set, this number will reflect the number of tasks that are ready to execute, but have not been started in order to prevent exceeding `schedule.limit`. From 7e5ce6bd03baa5004bcbc9361809ab4fdbd03015 Mon Sep 17 00:00:00 2001 From: Andrew Cholakian Date: Mon, 2 Dec 2019 16:14:31 -0600 Subject: [PATCH 3/3] Fix line ending --- CHANGELOG.next.asciidoc | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/CHANGELOG.next.asciidoc b/CHANGELOG.next.asciidoc index 43e607262cc8..8663f974f692 100644 --- a/CHANGELOG.next.asciidoc +++ b/CHANGELOG.next.asciidoc @@ -54,8 +54,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d - Removed `host.name` field that should never have been included. Heartbeat uses `observer.*` fields instead. {pull}14140[14140] - Changed default user-agent to be `Elastic-Heartbeat/VERSION (PLATFORM_INFO)` as the current default `Go-http-client/1.1` is often blacklisted. {pull}14291[14291] - JSON/Regex checks against HTTP bodies will only consider the first 100MiB of the HTTP body to prevent excessive memory usage. {pull}14223[pull] -- Heartbeat now starts monitors scheduled with the '@every X' syntax instantaneously on startup, rather than waiting for the -given interval to pass before running them. {pull}14890[14890] +- Heartbeat now starts monitors scheduled with the '@every X' syntax instantaneously on startup, rather than waiting for the given interval to pass before running them. {pull}14890[14890] *Journalbeat*