New Task Management API #525
Comments
devoxel
added a commit
to devoxel/elasticsearch_exporter
that referenced
this issue
Sep 20, 2023
This commit adds simple aggregation of Elasticsearch Tasks API. There are 4 new metrics; though 3 are just bookkeeping. elasticsearch_task_stats_action_total is a gague reporting the total number of tasks running for a given action. Because there are no stats endpoints available for this, this change introduces an aggregation step to group the number of tasks by action name. This metric is useful for ensuring long running actions of a specific kind stay within a specific limit. Of particular use to me is the action: 'indices:data/write/delete/byquery'. In my usecase, our ES access patterns mean we have a predefined limit of these actions running on the cluster. This change also adds two new CLI flags to manage the collection of tasks API: --es.tasks (to enable task collection) --es.tasks.actions (to filter tasks by action param) Issue prometheus-community#525 proposed addition of collection of these tasks.
devoxel
added a commit
to devoxel/elasticsearch_exporter
that referenced
this issue
Sep 20, 2023
This commit adds simple aggregation of Elasticsearch Tasks API. There are 4 new metrics; though 3 are just bookkeeping. elasticsearch_task_stats_action_total is a gague reporting the total number of tasks running for a given action. Because there are no stats endpoints available for this, this change introduces an aggregation step to group the number of tasks by action name. This metric is useful for ensuring long running actions of a specific kind stay within a specific limit. Of particular use to me is the action: 'indices:data/write/delete/byquery'. In my usecase, our ES access patterns mean we have a predefined limit of these actions running on the cluster. This change also adds two new CLI flags to manage the collection of tasks API: --es.tasks (to enable task collection) --es.tasks.actions (to filter tasks by action param) Issue prometheus-community#525 proposed addition of collection of these tasks.
devoxel
added a commit
to devoxel/elasticsearch_exporter
that referenced
this issue
Sep 20, 2023
This commit adds simple aggregation of Elasticsearch Tasks API. There are 4 new metrics; though 3 are just bookkeeping. elasticsearch_task_stats_action_total is a gague reporting the total number of tasks running for a given action. Because there are no stats endpoints available for this, this change introduces an aggregation step to group the number of tasks by action name. This metric is useful for ensuring long running actions of a specific kind stay within a specific limit. Of particular use to me is the action: 'indices:data/write/delete/byquery'. In my usecase, our ES access patterns mean we have a predefined limit of these actions running on the cluster. This change also adds two new CLI flags to manage the collection of tasks API: --es.tasks (to enable task collection) --es.tasks.actions (to filter tasks by action param) Issue prometheus-community#525 proposed addition of collection of these tasks.
devoxel
added a commit
to devoxel/elasticsearch_exporter
that referenced
this issue
Sep 20, 2023
This commit adds simple aggregation of Elasticsearch Tasks API. There are 4 new metrics; though 3 are just bookkeeping. elasticsearch_task_stats_action_total is a gague reporting the total number of tasks running for a given action. Because there are no stats endpoints available for this, this change introduces an aggregation step to group the number of tasks by action name. This metric is useful for ensuring long running actions of a specific kind stay within a specific limit. Of particular use to me is the action: 'indices:data/write/delete/byquery'. In my usecase, our ES access patterns mean we have a predefined limit of these actions running on the cluster. This change also adds two new CLI flags to manage the collection of tasks API: --es.tasks (to enable task collection) --es.tasks.actions (to filter tasks by action param) Issue prometheus-community#525 proposed addition of collection of these tasks. Signed-off-by: Aaron Delaney <apd@arista.com>
sysadmind
pushed a commit
that referenced
this issue
Mar 14, 2024
* collector: add tasks API collection This commit adds simple aggregation of Elasticsearch Tasks API. There are 4 new metrics; though 3 are just bookkeeping. elasticsearch_task_stats_action_total is a gague reporting the total number of tasks running for a given action. Because there are no stats endpoints available for this, this change introduces an aggregation step to group the number of tasks by action name. This metric is useful for ensuring long running actions of a specific kind stay within a specific limit. Of particular use to me is the action: 'indices:data/write/delete/byquery'. In my usecase, our ES access patterns mean we have a predefined limit of these actions running on the cluster. This change also adds two new CLI flags to manage the collection of tasks API: --es.tasks (to enable task collection) --es.tasks.actions (to filter tasks by action param) Issue #525 proposed addition of collection of these tasks. Signed-off-by: Aaron Delaney <apd@arista.com> * collector: use collector interface for tasks Signed-off-by: Aaron Delaney <apd@arista.com> * all: fix issues reported by golangci-lint Signed-off-by: Aaron Delaney <apd@arista.com> * collector: make task structs private to package Signed-off-by: Aaron Delaney <apd@arista.com> * Fix task stats metric name Signed-off-by: Aaron Delaney <apd@arista.com> * Fix tasks test Signed-off-by: Aaron Delaney <apd@arista.com> --------- Signed-off-by: Aaron Delaney <apd@arista.com>
jaimeyh
pushed a commit
to sysdiglabs/elasticsearch_exporter
that referenced
this issue
Jun 14, 2024
* collector: add tasks API collection This commit adds simple aggregation of Elasticsearch Tasks API. There are 4 new metrics; though 3 are just bookkeeping. elasticsearch_task_stats_action_total is a gague reporting the total number of tasks running for a given action. Because there are no stats endpoints available for this, this change introduces an aggregation step to group the number of tasks by action name. This metric is useful for ensuring long running actions of a specific kind stay within a specific limit. Of particular use to me is the action: 'indices:data/write/delete/byquery'. In my usecase, our ES access patterns mean we have a predefined limit of these actions running on the cluster. This change also adds two new CLI flags to manage the collection of tasks API: --es.tasks (to enable task collection) --es.tasks.actions (to filter tasks by action param) Issue prometheus-community#525 proposed addition of collection of these tasks. Signed-off-by: Aaron Delaney <apd@arista.com> * collector: use collector interface for tasks Signed-off-by: Aaron Delaney <apd@arista.com> * all: fix issues reported by golangci-lint Signed-off-by: Aaron Delaney <apd@arista.com> * collector: make task structs private to package Signed-off-by: Aaron Delaney <apd@arista.com> * Fix task stats metric name Signed-off-by: Aaron Delaney <apd@arista.com> * Fix tasks test Signed-off-by: Aaron Delaney <apd@arista.com> --------- Signed-off-by: Aaron Delaney <apd@arista.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Elasticsearch has a new API in beta that shows pending inter-cluster tasks like searches and cleanup activities. These are different than the tasks Elasticsearch Exporter currently exports, which are pending administrative tasks.
https://www.elastic.co/guide/en/elasticsearch/reference/current/tasks.html
elastic/elasticsearch#51628
Would it be possible to add support for this new API so that we can monitor these tasks as another health indicator.
The text was updated successfully, but these errors were encountered: