Cherry-pick #21751 to 7.x: Add proxy metricset (#21834)

CHANGELOG.next.asciidoc

@@ -529,6 +529,7 @@ field. You can revert this change by configuring tags for the module and omittin
 - Add istiod metricset. {pull}21519[21519]
 - Release `add_cloudfoundry_metadata` as GA. {pull}21525[21525]
 - Add support for OpenStack SSL metadata APIs in `add_cloud_metadata`. {pull}21590[21590]
+- Add proxy metricset for istio module. {pull}21751[21751]
 
 *Auditbeat*
 

metricbeat/docs/modules/istio.asciidoc

@@ -11,26 +11,37 @@ beta[]
 This is the Istio module.
 This module is compatible with versions before `1.5` of Istio where microservices architecture is used. If using
 versions priot to `1.5` then `mesh`, `mixer`, `pilot`, `galley`, `citadel` metricsets should be used.
-wehre the Istio module collects metrics from the
+where the Istio module collects metrics from the
 Istio https://istio.io/v1.4/docs/tasks/observability/metrics/querying-metrics/#about-the-prometheus-add-on[prometheus exporters endpoints].
-For versions after `1.5`, `istiod` metricset can be used which collects metrics directly from Istio Daemon.
+
+For versions after `1.5`, `istiod` and `proxy` metricsets can be used.
+
+`istiod` collects metrics directly from Istio Daemon while `proxy` collects from each of the proxy sidecars.
 
 The default metricsets are `mesh`, `mixer`, `pilot`, `galley`, `citadel`.
 
 [float]
 === Compatibility
 
 The Istio module is tested with Istio `1.4` for `mesh`, `mixer`, `pilot`, `galley`, `citadel`.
-The Istio module is tested with Istio `1.7` for `istiod`.
+The Istio module is tested with Istio `1.7` for `istiod` and `proxy`.
 
 [float]
 === Dashboard
 
-The Istio module includes a predefined dashboard with overview information about Istio Daemon.
-This dashboard is only compatible with versions of Istio after `1.5` which should be monitored with `istiod` metricset.
+The Istio module includes predefined dashboards:
+
+1. overview information about Istio Daemon.
+
+2. Traffic information collected from istio-proxies.
+
+These dashboards are only compatible with versions of Istio after `1.5` which should be monitored with `istiod`
+and `proxy` metricsets.
 
 image::./images/metricbeat-istio-overview.png[]
 
+image::./images/metricbeat-istio-traffic.png[]
+
 
 [float]
 === Example configuration
@@ -82,6 +93,13 @@ metricbeat.modules:
   period: 10s
   # use istiod.istio-system:15014, when deploying Metricbeat in a kubernetes cluster as Pod or Daemonset
   hosts: ['localhost:15014']
+
+# Istio proxy to monitor Envoy sidecars for versions after 1.5 of Istio.
+- module: istio
+  metricsets: ['proxy']
+  period: 10s
+  # it's recommended to deploy this metricset with autodiscovery, see metricset's docs for more info
+  hosts: ['localhost:15090']
 ----
 
 [float]
@@ -93,19 +111,27 @@ The following metricsets are available:
 
 * <<metricbeat-metricset-istio-galley,galley>>
 
+* <<metricbeat-metricset-istio-istiod,istiod>>
+
 * <<metricbeat-metricset-istio-mesh,mesh>>
 
 * <<metricbeat-metricset-istio-mixer,mixer>>
 
 * <<metricbeat-metricset-istio-pilot,pilot>>
 
+* <<metricbeat-metricset-istio-proxy,proxy>>
+
 include::istio/citadel.asciidoc[]
 
 include::istio/galley.asciidoc[]
 
+include::istio/istiod.asciidoc[]
+
 include::istio/mesh.asciidoc[]
 
 include::istio/mixer.asciidoc[]
 
 include::istio/pilot.asciidoc[]
 
+include::istio/proxy.asciidoc[]
+

metricbeat/docs/modules/istio/istiod.asciidoc

@@ -0,0 +1,18 @@
+////
+This file is generated! See scripts/mage/docs_collector.go
+////
+
+[[metricbeat-metricset-istio-istiod]]
+[role="xpack"]
+=== Istio istiod metricset
+
+beta[]
+
+include::../../../../x-pack/metricbeat/module/istio/istiod/_meta/docs.asciidoc[]
+
+
+==== Fields
+
+For a description of each field in the metricset, see the
+<<exported-fields-istio,exported fields>> section.
+

metricbeat/docs/modules/istio/proxy.asciidoc

@@ -0,0 +1,18 @@
+////
+This file is generated! See scripts/mage/docs_collector.go
+////
+
+[[metricbeat-metricset-istio-proxy]]
+[role="xpack"]
+=== Istio proxy metricset
+
+beta[]
+
+include::../../../../x-pack/metricbeat/module/istio/proxy/_meta/docs.asciidoc[]
+
+
+==== Fields
+
+For a description of each field in the metricset, see the
+<<exported-fields-istio,exported fields>> section.
+

metricbeat/docs/modules_list.asciidoc

@@ -133,11 +133,13 @@ This file is generated! See scripts/mage/docs_collector.go
 |<<metricbeat-metricset-iis-webserver,webserver>> beta[]  
 |<<metricbeat-metricset-iis-website,website>> beta[]  
 |<<metricbeat-module-istio,Istio>>  beta[]   |image:./images/icon-yes.png[Prebuilt dashboards are available]    |  
-.5+| .5+|  |<<metricbeat-metricset-istio-citadel,citadel>> beta[]  
+.7+| .7+|  |<<metricbeat-metricset-istio-citadel,citadel>> beta[]  
 |<<metricbeat-metricset-istio-galley,galley>> beta[]  
+|<<metricbeat-metricset-istio-istiod,istiod>> beta[]  
 |<<metricbeat-metricset-istio-mesh,mesh>> beta[]  
 |<<metricbeat-metricset-istio-mixer,mixer>> beta[]  
 |<<metricbeat-metricset-istio-pilot,pilot>> beta[]  
+|<<metricbeat-metricset-istio-proxy,proxy>> beta[]  
 |<<metricbeat-module-jolokia,Jolokia>>     |image:./images/icon-no.png[No prebuilt dashboards]    |  
 .1+| .1+|  |<<metricbeat-metricset-jolokia-jmx,jmx>>   
 |<<metricbeat-module-kafka,Kafka>>     |image:./images/icon-yes.png[Prebuilt dashboards are available]    |  

x-pack/metricbeat/metricbeat.reference.yml

@@ -702,6 +702,13 @@ metricbeat.modules:
   # use istiod.istio-system:15014, when deploying Metricbeat in a kubernetes cluster as Pod or Daemonset
   hosts: ['localhost:15014']
 
+# Istio proxy to monitor Envoy sidecars for versions after 1.5 of Istio.
+- module: istio
+  metricsets: ['proxy']
+  period: 10s
+  # it's recommended to deploy this metricset with autodiscovery, see metricset's docs for more info
+  hosts: ['localhost:15090']
+
 #------------------------------- Jolokia Module -------------------------------
 - module: jolokia
   #metricsets: ["jmx"]

x-pack/metricbeat/module/istio/_meta/config.reference.yml

@@ -39,3 +39,10 @@
   period: 10s
   # use istiod.istio-system:15014, when deploying Metricbeat in a kubernetes cluster as Pod or Daemonset
   hosts: ['localhost:15014']
+
+# Istio proxy to monitor Envoy sidecars for versions after 1.5 of Istio.
+- module: istio
+  metricsets: ['proxy']
+  period: 10s
+  # it's recommended to deploy this metricset with autodiscovery, see metricset's docs for more info
+  hosts: ['localhost:15090']

x-pack/metricbeat/module/istio/_meta/config.yml

@@ -39,3 +39,10 @@
   period: 10s
   # use istiod.istio-system:15014, when deploying Metricbeat in a kubernetes cluster as Pod or Daemonset
   hosts: ['localhost:15014']
+
+# Istio proxy to monitor Envoy sidecars for versions after 1.5 of Istio.
+- module: istio
+  metricsets: ['proxy']
+  period: 10s
+  # it's recommended to deploy this metricset with autodiscovery, see metricset's docs for more info
+  hosts: ['localhost:15090']

x-pack/metricbeat/module/istio/_meta/docs.asciidoc

@@ -1,22 +1,33 @@
 This is the Istio module.
 This module is compatible with versions before `1.5` of Istio where microservices architecture is used. If using
 versions priot to `1.5` then `mesh`, `mixer`, `pilot`, `galley`, `citadel` metricsets should be used.
-wehre the Istio module collects metrics from the
+where the Istio module collects metrics from the
 Istio https://istio.io/v1.4/docs/tasks/observability/metrics/querying-metrics/#about-the-prometheus-add-on[prometheus exporters endpoints].
-For versions after `1.5`, `istiod` metricset can be used which collects metrics directly from Istio Daemon.
+
+For versions after `1.5`, `istiod` and `proxy` metricsets can be used.
+
+`istiod` collects metrics directly from Istio Daemon while `proxy` collects from each of the proxy sidecars.
 
 The default metricsets are `mesh`, `mixer`, `pilot`, `galley`, `citadel`.
 
 [float]
 === Compatibility
 
 The Istio module is tested with Istio `1.4` for `mesh`, `mixer`, `pilot`, `galley`, `citadel`.
-The Istio module is tested with Istio `1.7` for `istiod`.
+The Istio module is tested with Istio `1.7` for `istiod` and `proxy`.
 
 [float]
 === Dashboard
 
-The Istio module includes a predefined dashboard with overview information about Istio Daemon.
-This dashboard is only compatible with versions of Istio after `1.5` which should be monitored with `istiod` metricset.
+The Istio module includes predefined dashboards:
+
+1. overview information about Istio Daemon.
+
+2. Traffic information collected from istio-proxies.
+
+These dashboards are only compatible with versions of Istio after `1.5` which should be monitored with `istiod`
+and `proxy` metricsets.
 
 image::./images/metricbeat-istio-overview.png[]
+
+image::./images/metricbeat-istio-traffic.png[]