Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Duplicate fields from system.process to process.* #22845

Merged
merged 6 commits into from
Dec 5, 2020
Merged

Duplicate fields from system.process to process.* #22845

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
1f3eb28
Duplicate fields from system.process to process.*
kaiyan-sheng Dec 2, 2020
955c29d
remove checking for windows
kaiyan-sheng Dec 2, 2020
4e2cbbe
add getAndCopy function
kaiyan-sheng Dec 3, 2020
415c7ce
Merge remote-tracking branch 'upstream/master' into process_metrics
kaiyan-sheng Dec 4, 2020
0799839
add changelog
kaiyan-sheng Dec 4, 2020
7ccf2c2
regenerate data.json for system.process
kaiyan-sheng Dec 4, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -866,6 +866,7 @@ same journal. {pull}18467[18467]
- Add io.ops in fields exported by system.diskio. {pull}22066[22066]
- `kibana` module: `stats` metricset no-longer collects usage-related data. {pull}22732[22732]
- Adjust the Apache status fields in the fleet mode. {pull}22821[22821]
- Add process.state, process.cpu.pct, process.cpu.start_time and process.memory.pct. {pull}22845[22845]

*Packetbeat*

Expand Down
51 changes: 51 additions & 0 deletions metricbeat/docs/fields.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40884,6 +40884,57 @@ System status metrics, like CPU and memory usage, that are collected from the op



[float]
=== process

Process metrics.



*`process.state`*::
+
--
The process state. For example: "running".


type: keyword

--

*`process.cpu.pct`*::
+
--
The percentage of CPU time spent by the process since the last event. This value is normalized by the number of CPU cores and it ranges from 0 to 1.


type: scaled_float

format: percent

--

*`process.cpu.start_time`*::
+
--
The time when the process was started.


type: date

--

*`process.memory.pct`*::
+
--
The percentage of memory the process occupied in main memory (RAM).


type: scaled_float

format: percent

--

[float]
=== system

Expand Down
25 changes: 25 additions & 0 deletions metricbeat/module/system/_meta/fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,31 @@
short_config: true
release: ga
fields:
- name: process
type: group
description: >
Process metrics.
fields:
- name: state
type: keyword
description: >
The process state. For example: "running".
- name: cpu.pct
type: scaled_float
format: percent
description: >
The percentage of CPU time spent by the process since the last event.
This value is normalized by the number of CPU cores and it ranges
from 0 to 1.
- name: cpu.start_time
type: date
description: >
The time when the process was started.
- name: memory.pct
type: scaled_float
format: percent
description: >
The percentage of memory the process occupied in main memory (RAM).
- name: system
type: group
description: >
Expand Down
2 changes: 1 addition & 1 deletion metricbeat/module/system/fields.go
View file
Open in desktop

Large diffs are not rendered by default.

185 changes: 15 additions & 170 deletions metricbeat/module/system/process/_meta/data.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,202 +15,47 @@
"splash"
],
"command_line": "/sbin/init splash",
"cpu": {
"pct": 0,
"start_time": "2020-12-03T21:01:50.000Z"
},
"memory": {
"pct": 0.0004
},
"name": "systemd",
"pgid": 1,
"pid": 1,
"ppid": 0
"ppid": 0,
"state": "sleeping"
},
"service": {
"type": "system"
},
"system": {
"process": {
"cgroup": {
"blkio": {
"id": "init.scope",
"path": "/init.scope",
"total": {
"bytes": 573440,
"ios": 140
}
},
"cpu": {
"cfs": {
"period": {
"us": 100000
},
"quota": {
"us": 0
},
"shares": 1024
},
"id": "init.scope",
"path": "/init.scope",
"rt": {
"period": {
"us": 0
},
"runtime": {
"us": 0
}
},
"stats": {
"periods": 0,
"throttled": {
"ns": 0,
"periods": 0
}
}
},
"cpuacct": {
"id": "init.scope",
"path": "/init.scope",
"percpu": {
"1": 571119313,
"10": 282248474,
"11": 363773369,
"12": 855691122,
"2": 735154670,
"3": 458174215,
"4": 281192882,
"5": 563599363,
"6": 415888016,
"7": 396270851,
"8": 550242795,
"9": 419970811
},
"stats": {
"system": {
"ns": 1700000000
},
"user": {
"ns": 1920000000
}
},
"total": {
"ns": 5893325881
}
},
"id": "init.scope",
"memory": {
"id": "init.scope",
"kmem": {
"failures": 0,
"limit": {
"bytes": 9223372036854771712
},
"usage": {
"bytes": 16871424,
"max": {
"bytes": 17829888
}
}
},
"kmem_tcp": {
"failures": 0,
"limit": {
"bytes": 9223372036854771712
},
"usage": {
"bytes": 0,
"max": {
"bytes": 0
}
}
},
"mem": {
"failures": 0,
"limit": {
"bytes": 9223372036854771712
},
"usage": {
"bytes": 21827584,
"max": {
"bytes": 30121984
}
}
},
"memsw": {
"failures": 0,
"limit": {
"bytes": 0
},
"usage": {
"bytes": 0,
"max": {
"bytes": 0
}
}
},
"path": "/init.scope",
"stats": {
"active_anon": {
"bytes": 4055040
},
"active_file": {
"bytes": 405504
},
"cache": {
"bytes": 643072
},
"hierarchical_memory_limit": {
"bytes": 9223372036854771712
},
"hierarchical_memsw_limit": {
"bytes": 0
},
"inactive_anon": {
"bytes": 270336
},
"inactive_file": {
"bytes": 0
},
"major_page_faults": 0,
"mapped_file": {
"bytes": 0
},
"page_faults": 117447,
"pages_in": 52008,
"pages_out": 50851,
"rss": {
"bytes": 4009984
},
"rss_huge": {
"bytes": 0
},
"swap": {
"bytes": 0
},
"unevictable": {
"bytes": 0
}
}
},
"path": "/init.scope"
},
"cmdline": "/sbin/init splash",
"cpu": {
"start_time": "2020-11-03T17:55:56.000Z",
"start_time": "2020-12-03T21:01:50.000Z",
"total": {
"norm": {
"pct": 0
},
"pct": 0,
"value": 5580
"value": 4220
}
},
"memory": {
"rss": {
"bytes": 13414400,
"bytes": 12042240,
"pct": 0.0004
},
"share": 8531968,
"size": 173076480
"share": 8523776,
"size": 172105728
},
"state": "sleeping"
}
},
"user": {
"name": "root"
}
}
}
26 changes: 23 additions & 3 deletions metricbeat/module/system/process/process.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,9 +139,19 @@ func (m *MetricSet) Fetch(r mb.ReporterV2) error {
}

// Duplicate system.process.cmdline with ECS name process.command_line
if v, ok := proc["cmdline"]; ok {
rootFields.Put("process.command_line", v)
}
rootFields = getAndCopy(proc, "cmdline", rootFields, "process.command_line")

// Duplicate system.process.state with process.state
rootFields = getAndCopy(proc, "state", rootFields, "process.state")

// Duplicate system.process.cpu.start_time with process.cpu.start_time
rootFields = getAndCopy(proc, "cpu.start_time", rootFields, "process.cpu.start_time")

// Duplicate system.process.cpu.total.norm.pct with process.cpu.pct
rootFields = getAndCopy(proc, "cpu.total.norm.pct", rootFields, "process.cpu.pct")

// Duplicate system.process.memory.rss.pct with process.memory.pct
rootFields = getAndCopy(proc, "memory.rss.pct", rootFields, "process.memory.pct")

if cwd := getAndRemove(proc, "cwd"); cwd != nil {
rootFields.Put("process.working_directory", cwd)
Expand Down Expand Up @@ -180,3 +190,13 @@ func getAndRemove(from common.MapStr, field string) interface{} {
}
return nil
}

func getAndCopy(from common.MapStr, field string, to common.MapStr, toField string) common.MapStr {
v, err := from.GetValue(field)
if err != nil {
return to
}

_, err = to.Put(toField, v)
return to
}