Harvester Cleanup and FileEvent #2090
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
ruflin
commented
Jul 25, 2016
•
ruflin
commented
- Rename FileEvent to Event
- Remove double information between FileEvent and State (Source, Fileinfo)
- Remove readLine function as not needed anymore
ruflin
added
in progress
ruflin
force-pushed
the
cleanup-reader-start
branch
2 times, most recently
from
f393b1e to
190db27
Compare
* Rename FileEvent to Event * Remove double information between FileEvent and State (Source, Fileinfo) * Remove readLine function as not needed anymore
| @@ -63,8 +64,7 @@ func (h *Harvester) Harvest() { | |||
| default: | |||
| } | |||
|
|
|||
| // Partial lines return error and are only read on completion | |||
| ts, text, bytesRead, jsonFields, err := readLine(r) | |||
| message, err := r.Next() | |||
There was a problem hiding this comment.
@urso The behaviour here should be identical to before, but hopefully better readable. An event is still created if Byte == 0 but this is filtered out as a state update only by the publisher. I assume this happens when an event only consists of a newline. So the state will be updated no event will be sent to the output. Alternatively we could also check inside shouldExportLine for bytes == 0.
There was a problem hiding this comment.
normally new lines characters are added to byte count, so to have correct offset.
ruflin
force-pushed
the
cleanup-reader-start
branch
from
190db27 to
ac4a155
Compare
ruflin
added
review
and removed
in progress
|
LGTM. waiting for travis. |
Merged
ruflin
added a commit
to ruflin/beats
that referenced
this pull request
Jul 26, 2016
Empty lines should not be sent as an event. To make sure this does not happen a test was added. This was probably broken in elastic#2090
tsg
pushed a commit
that referenced
this pull request
Jul 26, 2016
Empty lines should not be sent as an event. To make sure this does not happen a test was added. This was probably broken in #2090
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.