Cherry-pick #16215 to 7.x: azure-event hub: improve error handling and stop input if the event has not been processed correctly #17195
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…as not been processed correctly (elastic#16215) * work on GA * update changelog * go vet * work on error * error handling * error message * temp * temp * undo tests * update * integration * fmt update * upgrade * upgrade * mage vendor * notice (cherry picked from commit 1d3bc8c)
narph
added
[zube]: In Review
backport
Team:Integrations
|
Pinging @elastic/integrations (Team:Integrations) |
|
Pinging @elastic/integrations-platforms (Team:Platforms) |
ChrsMark
reviewed
Mar 24, 2020
go.sum
Outdated
| github.com/antihax/optional v0.0.0-20180407024304-ca021399b1a6/go.mod h1:V8iCPQYkqmusNa815XgQio277wI47sdRh1dUOLdyC6Q= | ||
| github.com/antlr/antlr4 v0.0.0-20200225173536-225249fdaef5 h1:nkZ9axP+MvUFCu8JRN/MCY+DmTfs6lY7hE0QnJbxSdI= |
There was a problem hiding this comment.
Maybe this should not be here?
There was a problem hiding this comment.
ran mage vendor , updated go.sum
There was a problem hiding this comment.
I'm mostly concerned about https://github.com/elastic/beats/pull/16215/files#diff-f949e2d81c8076ebbf8af38fcbb72c1fR111 and https://github.com/elastic/beats/pull/16215/files#diff-f949e2d81c8076ebbf8af38fcbb72c1fR734 from the original PR which are absent now. Is this normal?
ChrsMark
approved these changes
Mar 24, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cherry-pick of PR #16215 to 7.x branch. Original message:
Should help with issue #15671
Previously, if the event was not successfully processed, the checkpoint would have still been updated and the input would move to the next set of messages.
compositeHandlersmethod returns nil either way https://github.com/elastic/beats/blob/master/vendor/github.com/Azure/azure-event-hubs-go/v3/eph/eph.go#L406so the checkpoint is updated https://github.com/elastic/beats/blob/master/vendor/github.com/Azure/azure-event-hubs-go/v3/receiver.go#L242.
In the new release 3.1.2 for the azure-event-hub sdk, the error is returned https://github.com/Azure/azure-event-hubs-go/blob/v3.1.2/eph/eph.go#L423 so the checkpoint is not updated anymore.
This is still not e2e ack as this does not stop if the events were not sent to ES.