-
Notifications
You must be signed in to change notification settings - Fork 5.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(outputs.graphite): Retry connecting to servers with failed send attempts #11439
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One small log change, thank you!
Co-authored-by: Joshua Powers <powersj@fastmail.com>
Download PR build artifacts for linux_amd64.tar.gz, darwin_amd64.tar.gz, and windows_amd64.zip. 📦 Click here to get additional PR build artifactsArtifact URLs |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me. Thanks @sspaink!
resolve: #11429
The Graphite output plugin would only retry a server connection if it failed to send on all connections, this behavior wasn't obvious and if one server always remains up the other servers would never reconnect. This pull requests updates the retry logic to track the failed servers and reconnect to them after each send. I could also add a
retry_delay
configuration option similar to the inputs.cloud_pubsub and inputs.jti_openconfig_telemetry plugins, but from the issue the author mentioned this might not be necessary so I thought maybe we could add the config option if someone does require it?I also updated the
log.Errorf
tolog.Debugf
for connection related errors, seeing as these servers will retry anyway this would help remove some noise in the logs which was also a concern in reported in #11429. If all servers fail it should print an error to the log though.