Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate system process metricset fields to ECS #10332

Merged
merged 21 commits into from
Jan 31, 2019
Merged
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d

*Metricbeat*

- Migrate system process metricset fields to ECS. {pull}10332[10332]
- Refactor Prometheus metric mappings {pull}9948[9948]
- Removed Prometheus stats metricset in favor of just using Prometheus collector {pull}9948[9948]
- Adjust Redis.info metricset fields to ECS. {pull}10319[10319]
Expand Down Expand Up @@ -231,6 +232,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Release use of xpack.enabled: true flag in Elasticsearch and Kibana modules as GA. {pull}10222[10222]
- Add support for MySQL 8.0 and tests also for Percona and MariaDB. {pull}10261[10261]
- Rename 'db' Metricset to 'transaction_log' in MSSQL Metricbeat module {pull}10109[10109]
- Add process arguments and the path to its executable file in the system process metricset {pull}10332[10332]
- Added 'server' Metricset to Zookeeper Metricbeat module {issue}8938[8938] {pull}10341[10341]
- Release AWS module as GA. {pull}10345[10345]

Expand Down
27 changes: 27 additions & 0 deletions dev-tools/ecs-migration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1223,6 +1223,33 @@
alias: true
beat: metricbeat

### System

- from: system.process.name
to: process.name
alias: true
beat: metricbeat

- from: system.process.pid
to: process.pid
alias: true
beat: metricbeat

- from: system.process.ppid
to: process.ppid
alias: true
beat: metricbeat

- from: system.process.pgid
to: process.pgid
alias: true
beat: metricbeat

- from: system.process.username
to: user.name
alias: true
beat: metricbeat

### HTTP
- from: http.request.body
to: http.request.body.content
Expand Down
53 changes: 34 additions & 19 deletions libbeat/metric/system/process/process.go
Original file line number Diff line number Diff line change
Expand Up @@ -45,14 +45,16 @@ type ProcsMap map[int]*Process
// Process is the structure which holds the information of a process running on the host.
// It includes pid, gid and it interacts with gosigar to fetch process data from the host.
type Process struct {
Pid int `json:"pid"`
Ppid int `json:"ppid"`
Pgid int `json:"pgid"`
Name string `json:"name"`
Username string `json:"username"`
State string `json:"state"`
CmdLine string `json:"cmdline"`
Cwd string `json:"cwd"`
Pid int `json:"pid"`
Ppid int `json:"ppid"`
Pgid int `json:"pgid"`
Name string `json:"name"`
Username string `json:"username"`
State string `json:"state"`
Args []string `json:"args"`
CmdLine string `json:"cmdline"`
Cwd string `json:"cwd"`
Executable string `json:"executable"`
jsoriano marked this conversation as resolved.
Show resolved Hide resolved
Mem sigar.ProcMem
Cpu sigar.ProcTime
SampleTime time.Time
Expand Down Expand Up @@ -98,15 +100,16 @@ func newProcess(pid int, cmdline string, env common.MapStr) (*Process, error) {
}

proc := Process{
Pid: pid,
Ppid: state.Ppid,
Pgid: state.Pgid,
Name: state.Name,
Username: state.Username,
State: getProcState(byte(state.State)),
CmdLine: cmdline,
Cwd: exe.Cwd,
Env: env,
Pid: pid,
Ppid: state.Ppid,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see, we also have it as part of the monitoring reporting?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think this is also used for beats monitoring.

Pgid: state.Pgid,
Name: state.Name,
Username: state.Username,
State: getProcState(byte(state.State)),
CmdLine: cmdline,
Cwd: exe.Cwd,
Executable: exe.Name,
Env: env,
}

return &proc, nil
Expand All @@ -130,12 +133,16 @@ func (proc *Process) getDetails(envPredicate func(string) bool) error {
return fmt.Errorf("error getting process cpu time for pid=%d: %v", proc.Pid, err)
}

if proc.CmdLine == "" {
if len(proc.Args) == 0 {
args := sigar.ProcArgs{}
if err := args.Get(proc.Pid); err != nil && !sigar.IsNotImplemented(err) {
return fmt.Errorf("error getting process arguments for pid=%d: %v", proc.Pid, err)
}
proc.CmdLine = strings.Join(args.List, " ")
proc.Args = args.List
}

if proc.CmdLine == "" && len(proc.Args) > 0 {
proc.CmdLine = strings.Join(proc.Args, " ")
}

if fd, err := getProcFDUsage(proc.Pid); err != nil {
Expand Down Expand Up @@ -283,6 +290,10 @@ func (procStats *Stats) getProcessEvent(process *Process) common.MapStr {
},
}

if len(process.Args) > 0 {
proc["args"] = process.Args
}

if process.CmdLine != "" {
proc["cmdline"] = process.CmdLine
}
Expand All @@ -291,6 +302,10 @@ func (procStats *Stats) getProcessEvent(process *Process) common.MapStr {
proc["cwd"] = process.Cwd
}

if process.Executable != "" {
proc["exe"] = process.Executable
}

if len(process.Env) > 0 {
proc["env"] = process.Env
}
Expand Down
5 changes: 5 additions & 0 deletions metricbeat/_meta/fields.common.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,11 @@
description: >
The name of the metricset that generated the event.

- name: process.pgid
type: long
description: >
Process group id.

- name: service.address
description: >
Connection address of the machine from which the metricset was collected. This
Expand Down
35 changes: 20 additions & 15 deletions metricbeat/docs/fields.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -1975,6 +1975,16 @@ The name of the module that generated the event.
The name of the metricset that generated the event.


--

*`process.pgid`*::
+
--
type: long

Process group id.


--

*`service.address`*::
Expand Down Expand Up @@ -23334,10 +23344,9 @@ The number of outgoing packets that were dropped. This value is always 0 on Darw
*`system.process.name`*::
+
--
type: keyword

The process name.
type: alias

alias to: process.name

--

Expand All @@ -23354,30 +23363,27 @@ The process state. For example: "running".
*`system.process.pid`*::
+
--
type: long

The process pid.
type: alias

alias to: process.pid

--

*`system.process.ppid`*::
+
--
type: long

The process parent pid.
type: alias

alias to: process.ppid

--

*`system.process.pgid`*::
+
--
type: long

The process group id.
type: alias

alias to: process.pgid

--

Expand All @@ -23394,10 +23400,9 @@ The full command-line used to start the process, including the arguments separat
*`system.process.username`*::
+
--
type: keyword

The username of the user that created the process. If the username cannot be determined, the field will contain the user's numeric identifier (UID). On Windows, this field includes the user's domain and is formatted as `domain\username`.
type: alias

alias to: user.name

--

Expand Down
2 changes: 1 addition & 1 deletion metricbeat/include/fields.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion metricbeat/include/fields/fields.go

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion metricbeat/module/system/fields.go

Large diffs are not rendered by default.

Loading