Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Telegraf AMQP Output - Persistent messages #3526

Closed
hanej opened this issue Nov 29, 2017 · 2 comments · Fixed by #3528
Closed

Telegraf AMQP Output - Persistent messages #3526

hanej opened this issue Nov 29, 2017 · 2 comments · Fixed by #3528
Labels
feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin
Milestone

Comments

@hanej
Copy link

hanej commented Nov 29, 2017

Feature Request

Proposal:

Please allow the option to set messages to be persistent on the amqp output plugin.

Current behavior:

Messages are not persistent and are lost when a broker restarts

Desired behavior:

Messages are retained on the broker until they are processed

Use case: [Why is this important (helps with prioritizing requests)]

When batch loading large amounts of messages (logparser messages to be precise), the messages are very large and take a while to process. On occasion, the broker will run out of memory and crash. All messages are lost and have to be reprocessed.

@danielnelson
Copy link
Contributor

I might be wrong, but I think when you create the queue you need to make it durable. This would be done outside of Telegraf.

@hanej
Copy link
Author

hanej commented Nov 29, 2017

Making the queue durable just means the queue itself survives a broker restart, not the messages. Messages need to be put on the queue with a DeliveryMode of 2 to make them persistent. Currently 1 is the default (transient).

https://github.com/streadway/amqp/blob/master/types.go#L104-L116

@danielnelson danielnelson added the feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin label Nov 29, 2017
@danielnelson danielnelson added this to the 1.5.0 milestone Nov 29, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants