Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Elastic Agent] Add stream.* to metrics and logs collected #18261

Merged
merged 2 commits into from
May 5, 2020

Conversation

ph
Copy link
Contributor

@ph ph commented May 5, 2020

This PR ensure that the stream.type, stream.dataset and stream.namespace
are added to each logs or metrics collected by the Agent.

Fixes: #18203

What does this PR do?

Why is it important?

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

This PR ensure that the stream.type, stream.dataset and stream.namespace
are added to each logs or metrics collected by the Agent.

Fixes: elastic#18203
@ph ph added review [zube]: In Review elastic-agent Ingest Management:alpha1 Group issues for ingest management alpha1 labels May 5, 2020
@ph ph requested review from ruflin and michalpristas May 5, 2020 17:57
@ph ph self-assigned this May 5, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/ingest-management (Team:Ingest Management)

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label May 5, 2020
@ph ph added needs_backport PR is waiting to be backported to other branches. and removed needs_team Indicates that the issue/PR needs a Team:* label labels May 5, 2020
@@ -199,6 +211,18 @@ func (o *Operator) getMonitoringMetricbeatConfig(output interface{}) (map[string
"period": "10s",
"hosts": hosts,
"index": "metrics-agent-default",
"processors": []map[string]interface{}{
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a slice of map make me always cringe a little.

Copy link
Contributor

@ruflin ruflin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code LGTM, but did not run it locally.

This PR made me think if we should have a final pipeline that enriches events with these fields if they are missing based on the index name 🤔

@ph
Copy link
Contributor Author

ph commented May 5, 2020

This PR made me think if we should have a final pipeline that enriches events with these fields if they are missing based on the index name

This seems to be a good default, that would be a painless script? Maybe create an issue.

Copy link
Contributor

@michalpristas michalpristas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM i will test tomorrow morning, if you need to backport let's get this in

@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview stats

Expand to view the summary

Build stats

@ph ph merged commit 7b87db5 into elastic:master May 5, 2020
ph added a commit to ph/beats that referenced this pull request May 5, 2020
…8261)

* [Elastic Agent] Add stream.* to metrics and logs collected

This PR ensure that the stream.type, stream.dataset and stream.namespace
are added to each logs or metrics collected by the Agent.

Fixes: elastic#18203
(cherry picked from commit 7b87db5)
@ph ph added v7.8.0 and removed needs_backport PR is waiting to be backported to other branches. labels May 5, 2020
ph added a commit that referenced this pull request May 5, 2020
…18264)

* [Elastic Agent] Add stream.* to metrics and logs collected

This PR ensure that the stream.type, stream.dataset and stream.namespace
are added to each logs or metrics collected by the Agent.

Fixes: #18203
(cherry picked from commit 7b87db5)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Agent] Does not send stream.* fields for monitoring data
4 participants