Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce event.duration and event.dataset for backward compatiblity #9393

Merged
merged 5 commits into from
Dec 12, 2018

Conversation

ruflin
Copy link
Contributor

@ruflin ruflin commented Dec 5, 2018

The field event.duration is introduced as a replacement for metricset.rtt. event.duration is in nano seconds, metricset.rtt in nano seconds so the two are not compatible. metricset.rtt will be removed in 7.0.

event.dataset is introduced as {module}.{metricset} for better backward compatiblity in 7.0.

Further changes

  • Fix tests

@ruflin ruflin added v5.6.0 v6.6.0 Team:Integrations Label for the Integrations team and removed v5.6.0 labels Dec 5, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/infrastructure

@ruflin
Copy link
Contributor Author

ruflin commented Dec 5, 2018

jenkins, test this

@webmat
Copy link
Contributor

webmat commented Dec 5, 2018

In the description, you meant to say the current metricset.rtt is in microseconds, right?

@webmat
Copy link
Contributor

webmat commented Dec 5, 2018

I'm pretty sure the test failures are legit, because they're the same on Travis and on Jenkins. So perhaps related to the incorrect field type.

Copy link
Contributor

@webmat webmat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to:

  • Fix the event.dataset type to keyword
  • Make sure there's no more test failures (if not caused by the point above)

@ruflin
Copy link
Contributor Author

ruflin commented Dec 11, 2018

jenkins, test this

@ruflin
Copy link
Contributor Author

ruflin commented Dec 11, 2018

@webmat This should be ready to go now.

The field event.duration is introduced as a replacement for metricset.rtt. event.duration is in nano seconds, metricset.rtt in nano seconds so the two are not compatible. `metricset.rtt` will be removed in 7.0.

`event.dataset` is introduced as `{module}.{metricset}` for better backward compatiblity in 7.0.

Further changes

* Fix tests
@ruflin ruflin force-pushed the introduce-dataset-fields branch from 5279894 to 09e70f8 Compare December 11, 2018 20:22
Copy link
Contributor

@webmat webmat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost got to LGTM, only one missing thing I noticed. The metricset.rtt is not actually marked as deprecated in the documentation.

I also have a nit about an innocuous race condition in setting event.duration vs metricset.rtt.

if event.Took > 0 {
// rtt is deprecated and will be removed in 7.0. Replaced by event.duration.
info.Put("metricset.rtt", event.Took/time.Microsecond)
info.Put("event.duration", event.Took/time.Nanosecond)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: here you have an innocuous race condition. The time.Nanosecond could happen later enough that trunc(event.duration) != metricset.rtt. It would be cleaner to populate event.duration, then truncate it to populate metricset.rtt.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure i can follow. event.Took is a value and it's not calculated here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, didn't look up what event.Took was, sorry. Never mind this :-)

@@ -131,6 +132,8 @@ https://github.com/elastic/beats/compare/v6.5.0...6.x[Check the HEAD diff]

*Metricbeat*

- Deprecate field `metricset.rtt`. Replaced by `event.duration` which is in nano instead of micro seconds.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no update to the definition for metricset.rtt to mark it as deprecated. I think it would make sense to add it in this PR. WDYT?

Copy link
Contributor Author

@ruflin ruflin Dec 11, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, added it to the fields definition.

@@ -38,3 +39,13 @@
example: elasticsearch
description: >
Name of the service metricbeat fetches the data from.

- name: event.duration
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious, why this is not named with the unit, like event.duration.ns?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This field comes from ECS: https://github.com/elastic/ecs#event

A more general note: I have second thoughts for how we deal with units in our current convention and hope in the future Elasticsearch can partially deal with it: elastic/elasticsearch#31244

@ruflin
Copy link
Contributor Author

ruflin commented Dec 12, 2018

jenkins, test this

@ruflin ruflin merged commit 84ffadf into elastic:6.x Dec 12, 2018
@ruflin ruflin deleted the introduce-dataset-fields branch December 12, 2018 13:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ecs Metricbeat Metricbeat review Team:Integrations Label for the Integrations team v6.6.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants