Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[vSphere] Add new Cluster metricset #40536

Merged
merged 31 commits into from
Aug 31, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
c1e34ec
add cluster metricset
niraj-crest Aug 15, 2024
2ffae91
update changelog
niraj-crest Aug 15, 2024
5009309
fix lint errors
niraj-crest Aug 16, 2024
895c19d
minor changes
niraj-crest Aug 27, 2024
7e3982d
add unit test
niraj-crest Aug 27, 2024
58c2ff6
update fields.yml
niraj-crest Aug 27, 2024
3173898
mage update
niraj-crest Aug 27, 2024
6c94fce
minor changes
niraj-crest Aug 28, 2024
a26ac0f
Merge branch 'main' of https://github.com/niraj-elastic/beats into vs…
niraj-crest Aug 28, 2024
7289c01
minor changes
niraj-crest Aug 28, 2024
b10f6e8
mage update
niraj-crest Aug 28, 2024
3f3e3ad
Merge branch 'main' into vsphere_cluster_metricset
shmsr Aug 28, 2024
8751d46
Address review comment
niraj-crest Aug 29, 2024
e59e349
Merge branch 'vsphere_cluster_metricset' of https://github.com/niraj-…
niraj-crest Aug 29, 2024
03fb2dd
Address review comments
niraj-crest Aug 29, 2024
659f419
Merge branch 'main' of https://github.com/niraj-elastic/beats into vs…
niraj-crest Aug 29, 2024
9fc4e41
address review comments
niraj-crest Aug 29, 2024
ae5baa0
update data.json
niraj-crest Aug 29, 2024
ecad3e0
update fields.yml
niraj-crest Aug 29, 2024
b4a51a8
update data.json
niraj-crest Aug 29, 2024
bcf98c8
Merge branch 'main' into vsphere_cluster_metricset
niraj-elastic Aug 29, 2024
7a2ed26
lint changes
niraj-crest Aug 29, 2024
cc90c76
Merge branch 'vsphere_cluster_metricset' of https://github.com/niraj-…
niraj-crest Aug 29, 2024
5a84640
Merge branch 'main' into vsphere_cluster_metricset
niraj-elastic Aug 29, 2024
4c17b58
mage update
niraj-crest Aug 29, 2024
2381255
Merge branch 'main' of https://github.com/niraj-elastic/beats into vs…
niraj-crest Aug 29, 2024
5b44f90
mage update for x-pack
niraj-crest Aug 29, 2024
f5bba7f
Merge branch 'main' of https://github.com/niraj-elastic/beats into vs…
niraj-crest Aug 29, 2024
c1fa02d
Mage update after conflicts
niraj-crest Aug 29, 2024
89e76da
update fields.go
niraj-crest Aug 29, 2024
91cea3d
Address review comments
niraj-crest Aug 30, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -291,6 +291,7 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
- Improve logging in Okta Entity Analytics provider. {issue}40106[40106] {pull}40347[40347]
- Document `winlog` input. {issue}40074[40074] {pull}40462[40462]
- Added retry logic to websocket connections in the streaming input. {issue}40271[40271] {pull}40601[40601]
- Add new metricset cluster for the vSphere module. {pull}40536[40536]
- Disable event normalization for netflow input {pull}40635[40635]

*Auditbeat*
Expand Down
101 changes: 101 additions & 0 deletions metricbeat/docs/fields.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -66854,6 +66854,107 @@ vSphere module



[float]
=== cluster

Cluster information.




*`vsphere.cluster.datastore.names`*::
+
--
List of all the Datastore names associated with the cluster.


type: keyword

--

*`vsphere.cluster.datastore.count`*::
+
--
Number of Datastores associated with the cluster.


type: long

--


*`vsphere.cluster.das_config.admission.control.enabled`*::
+
--
Indicates whether strict admission control is enabled.


type: boolean

--

*`vsphere.cluster.das_config.enabled`*::
+
--
Indicates whether vSphere HA feature is enabled.


type: boolean

--


*`vsphere.cluster.host.count`*::
+
--
Number of Hosts associated with the cluster.


type: long

--

*`vsphere.cluster.host.names`*::
+
--
List of all the Host names associated with the cluster.


type: keyword

--

*`vsphere.cluster.name`*::
+
--
Cluster name.


type: keyword

--


*`vsphere.cluster.network.count`*::
+
--
Number of Networks associated with the cluster.


type: long

--

*`vsphere.cluster.network.names`*::
+
--
List of all the Network names associated with the cluster.


type: keyword

--

[float]
=== datastore

Expand Down
8 changes: 6 additions & 2 deletions metricbeat/docs/modules/vsphere.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ This file is generated! See scripts/mage/docs_collector.go

The vSphere module uses the https://github.com/vmware/govmomi[Govmomi] library to collect metrics from any Vmware SDK URL (ESXi/VCenter). This library is built for and tested against ESXi and vCenter 5.5, 6.0 and 6.5.

By default it enables the metricsets `network`, `resourcepool`, `datastore`, `host` and `virtualmachine`.
By default it enables the metricsets `cluster`, `network`, `resourcepool`, `datastore`, `host` and `virtualmachine`.

[float]
=== Dashboard
Expand All @@ -35,7 +35,7 @@ in <<configuration-metricbeat>>. Here is an example configuration:
metricbeat.modules:
- module: vsphere
enabled: true
metricsets: ["datastore", "host", "virtualmachine", "network", "resourcepool"]
metricsets: ["cluster", "datastore", "host", "virtualmachine", "network", "resourcepool"]
# Real-time data collection – An ESXi Server collects data for each performance counter every 20 seconds.
period: 20s
hosts: ["https://localhost/sdk"]
Expand All @@ -53,6 +53,8 @@ metricbeat.modules:

The following metricsets are available:

* <<metricbeat-metricset-vsphere-cluster,cluster>>

* <<metricbeat-metricset-vsphere-datastore,datastore>>

* <<metricbeat-metricset-vsphere-host,host>>
Expand All @@ -63,6 +65,8 @@ The following metricsets are available:

* <<metricbeat-metricset-vsphere-virtualmachine,virtualmachine>>

include::vsphere/cluster.asciidoc[]

include::vsphere/datastore.asciidoc[]

include::vsphere/host.asciidoc[]
Expand Down
29 changes: 29 additions & 0 deletions metricbeat/docs/modules/vsphere/cluster.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
////
This file is generated! See scripts/mage/docs_collector.go
////
:edit_url: https://github.com/elastic/beats/edit/main/metricbeat/module/vsphere/cluster/_meta/docs.asciidoc


[[metricbeat-metricset-vsphere-cluster]]
=== vSphere cluster metricset

beta[]

include::../../../module/vsphere/cluster/_meta/docs.asciidoc[]

This is a default metricset. If the host module is unconfigured, this metricset is enabled by default.

:edit_url:

==== Fields

For a description of each field in the metricset, see the
<<exported-fields-vsphere,exported fields>> section.

Here is an example document generated by this metricset:

[source,json]
----
include::../../../module/vsphere/cluster/_meta/data.json[]
----
:edit_url!:
3 changes: 2 additions & 1 deletion metricbeat/docs/modules_list.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -311,7 +311,8 @@ This file is generated! See scripts/mage/docs_collector.go
|<<metricbeat-module-uwsgi,uWSGI>> |image:./images/icon-yes.png[Prebuilt dashboards are available] |
.1+| .1+| |<<metricbeat-metricset-uwsgi-status,status>>
|<<metricbeat-module-vsphere,vSphere>> |image:./images/icon-yes.png[Prebuilt dashboards are available] |
.5+| .5+| |<<metricbeat-metricset-vsphere-datastore,datastore>>
.6+| .6+| |<<metricbeat-metricset-vsphere-cluster,cluster>> beta[]
|<<metricbeat-metricset-vsphere-datastore,datastore>>
|<<metricbeat-metricset-vsphere-host,host>>
|<<metricbeat-metricset-vsphere-network,network>> beta[]
|<<metricbeat-metricset-vsphere-resourcepool,resourcepool>> beta[]
Expand Down
1 change: 1 addition & 0 deletions metricbeat/include/list_common.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion metricbeat/metricbeat.reference.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1007,7 +1007,7 @@ metricbeat.modules:
#------------------------------- VSphere Module -------------------------------
- module: vsphere
enabled: true
metricsets: ["datastore", "host", "virtualmachine", "network", "resourcepool"]
metricsets: ["cluster", "datastore", "host", "virtualmachine", "network", "resourcepool"]
# Real-time data collection – An ESXi Server collects data for each performance counter every 20 seconds.
period: 20s
hosts: ["https://localhost/sdk"]
Expand Down
1 change: 1 addition & 0 deletions metricbeat/module/vsphere/_meta/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ Now setup your metricbeat config to connect to Govcsim:
```
- module: vsphere
metricsets:
- cluster
- datastore
- host
- virtualmachine
Expand Down
2 changes: 1 addition & 1 deletion metricbeat/module/vsphere/_meta/config.reference.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
- module: vsphere
enabled: true
metricsets: ["datastore", "host", "virtualmachine", "network", "resourcepool"]
metricsets: ["cluster", "datastore", "host", "virtualmachine", "network", "resourcepool"]
# Real-time data collection – An ESXi Server collects data for each performance counter every 20 seconds.
period: 20s
hosts: ["https://localhost/sdk"]
Expand Down
1 change: 1 addition & 0 deletions metricbeat/module/vsphere/_meta/config.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
- module: vsphere
#metricsets:
# - cluster
# - datastore
# - host
# - virtualmachine
Expand Down
2 changes: 1 addition & 1 deletion metricbeat/module/vsphere/_meta/docs.asciidoc
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
The vSphere module uses the https://github.com/vmware/govmomi[Govmomi] library to collect metrics from any Vmware SDK URL (ESXi/VCenter). This library is built for and tested against ESXi and vCenter 5.5, 6.0 and 6.5.

By default it enables the metricsets `network`, `resourcepool`, `datastore`, `host` and `virtualmachine`.
By default it enables the metricsets `cluster`, `network`, `resourcepool`, `datastore`, `host` and `virtualmachine`.

[float]
=== Dashboard
Expand Down
47 changes: 47 additions & 0 deletions metricbeat/module/vsphere/cluster/_meta/data.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
{
"@timestamp": "2017-10-12T08:05:34.853Z",
"event": {
"dataset": "vsphere.cluster",
"duration": 115000,
"module": "vsphere"
},
"metricset": {
"name": "cluster",
"period": 10000
},
"service": {
"address": "127.0.0.1:33365",
"type": "vsphere"
},
"vsphere": {
"cluster": {
"name": "Cluster_1",
"das_config": {
"enabled": false,
"admission": {
"control": {
"enabled": true
}
}
},
"host": {
"count": 1,
"names": [
"Host_1"
]
},
"datastore": {
"count": 1,
"names": [
"Datastore_1"
]
},
"network": {
"count": 1,
"names": [
"Network_1"
]
}
}
}
}
1 change: 1 addition & 0 deletions metricbeat/module/vsphere/cluster/_meta/docs.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
This is the `cluster` metricset of the vSphere module.
54 changes: 54 additions & 0 deletions metricbeat/module/vsphere/cluster/_meta/fields.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
- name: cluster
type: group
release: beta
description: >
Cluster information.
fields:
- name: datastore
type: group
fields:
- name: names
type: keyword
description: >
List of all the Datastore names associated with the cluster.
- name: count
type: long
description: >
Number of Datastores associated with the cluster.
- name: das_config
type: group
fields:
- name: admission.control.enabled
type: boolean
description: >
Indicates whether strict admission control is enabled.
- name: enabled
type: boolean
description: >
Indicates whether vSphere HA feature is enabled.
- name: host
type: group
fields:
- name: count
type: long
description: >
Number of Hosts associated with the cluster.
- name: names
type: keyword
description: >
List of all the Host names associated with the cluster.
- name: name
type: keyword
description: >
Cluster name.
- name: network
type: group
fields:
- name: count
type: long
description: >
Number of Networks associated with the cluster.
- name: names
type: keyword
description: >
List of all the Network names associated with the cluster.
Loading
Loading