Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wavefront parser #4402

Merged
merged 6 commits into from
Aug 13, 2018
Merged

Wavefront parser #4402

merged 6 commits into from
Aug 13, 2018

Conversation

puckpuck
Copy link
Contributor

@puckpuck puckpuck commented Jul 9, 2018

Required for all PRs:

  • Signed CLA.
  • Has appropriate unit tests.

Parser for Wavefront Data Format

Does require a small fix to Wavefront output plugin in order to anticipate the presence of a source tag which will overrule any other host, hostname, etc. tag. Source/Host is a special class tag in Wavefront.

Note: No readme.md was created, since other parsers don't have one.

@danielnelson
Copy link
Contributor

Haven't had time to review yet, but the source tag caught my idea. It sounds similar to something we have thought about adding in the past. I assume this just ideally contains the hostname of the remote server when monitoring another system or the hostname of the local server Telegraf is running on?

Have you thought at all about the format of this field? Would it be only a fqdn or would it potentially have things such as a port number or schema?

@danielnelson danielnelson added this to the 1.8.0 milestone Jul 11, 2018
@puckpuck
Copy link
Contributor Author

yeah that's what the source tag represents in Wavefront. Where the data came from. This used to be called host, but now we see data from serverless stuff, some odd funky monitoring things, sometimes it's even the name of your car (yes no lie), so host isn't really appropriate anymore. In Wavefront source/host is a required tag, since it becomes part of the main index in the TSDB.

Wether it's a short name or fqdn Wavefront doesn't care. It's just a string representing an entity. We prefer to let the user/customer select which one they want to use, though most seem to lean towards short names, however some larger enterprises want an fqdn

@danielnelson
Copy link
Contributor

If we started using a standard name it would just be convention only in the same way the host tag is now. I would be happy with source as the tag name but what really triggers me is differences between formats like:

unix:///var/run/foo.sock
http://example.org
http://example.org/foo
example.org
example.org:1234

Obviously this is a bikeshed issue to a large degree but I'm having a hard time deciding if the tag needs to be unique to identify the source or if it would make sense to only specify the system it is on, maybe all of the examples could just be source=example.org? The downside of this is we may need to have metrics with redundancy to provide uniqueness:

apache,host=lappy386,source=example.org,url=http://example.org/server_status value=42

@puckpuck
Copy link
Contributor Author

I'm not sure what to do with unix:///var/run/foo.sock ... but for the other 4, source/host should be example. The other attributes (port, endpoint, etc) should be point tags.

@danielnelson danielnelson requested a review from glinton August 9, 2018 22:36
Copy link
Contributor

@glinton glinton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, approving since only changes are minor formatting

if s, ok := mTags["source"]; ok {
source = s
delete(mTags, "source")

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: no empty line here

import (
"bufio"
"bytes"
"github.com/influxdata/telegraf"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you move non-stdlib imports below, separated by an empty line

package wavefront

import (
"github.com/influxdata/telegraf"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same, move non-stdlib imports below

@puckpuck
Copy link
Contributor Author

made minor formatting changes as requested.

Also added docs. Realized just now it should be documented. Parser has no config so the docs are pretty simple.

@glinton glinton merged commit 6454319 into influxdata:master Aug 13, 2018
@randallt
Copy link

randallt commented Oct 1, 2018

@puckpuck, I don't see the new Wavefront parser show up in the release notes (other than this issue being listed) or on the main Telegraf readme (parsers). Did it actually get released with 1.8?

@puckpuck
Copy link
Contributor Author

puckpuck commented Oct 1, 2018

@randallt it is part of 1.8. It did make it to the Features section of the release notes, but not the Parsers section. @danielnelson ?

@randallt
Copy link

randallt commented Oct 1, 2018

The main README.md also needs to be updated:

Telegraf is able to parse the following input data formats into metrics, these formats may be used with input plugins supporting the data_format option:

InfluxDB Line Protocol
JSON
Graphite
Value
Nagios
Collectd
Dropwizard

@puckpuck
Copy link
Contributor Author

puckpuck commented Oct 1, 2018

I'll create a PR for the main readme. The input data format in docs is good though.

@puckpuck
Copy link
Contributor Author

puckpuck commented Oct 1, 2018

Come to think of it, since the docs for parsers/serializers were split all the links on the main readme need to be updated as well.

@danielnelson
Copy link
Contributor

Sorry about that, I'll get it fixed.

@danielnelson
Copy link
Contributor

317f4d2
f393c5e

rgitzel pushed a commit to rgitzel/telegraf that referenced this pull request Oct 17, 2018
@puckpuck puckpuck deleted the wavefront-parser branch February 21, 2019 03:53
otherpirate pushed a commit to otherpirate/telegraf that referenced this pull request Mar 15, 2019
otherpirate pushed a commit to otherpirate/telegraf that referenced this pull request Mar 15, 2019
dupondje pushed a commit to dupondje/telegraf that referenced this pull request Apr 22, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants