Fix isolation of modules when merging local and global field settings. #5808
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 4 commits
December 5, 2017 15:08
The user global and local fields configurations must be merged on pipeline setup. The DeepUpdate method did only keep references to the globally shared fields, such that local fields settings do overwrite the shared global state. If more then one beat.Client instance did set local fields, the global state (and all fields added to an event) will be given by the last client being instantiated. The fix Clones the global fields before merging, such that the beat.Client fields operate fully in isolation.
urso
force-pushed
the
bug/field-overwrites
branch
from
899026e to
69eb2c6
Compare
urso
added
bug
libbeat
needs_backport
kvch
reviewed
Dec 5, 2017
| @@ -68,7 +68,8 @@ type pipelineProcessors struct { | |||
|
|
|||
| processors beat.Processor | |||
|
|
|||
| disabled bool // disabled is set if outputs have been disabled via CLI | |||
| disabled bool // disabled is set if outputs have been disabled via CLI | |||
| alwaysCopy bool | |||
There was a problem hiding this comment.
Where does this variable get its value?
There was a problem hiding this comment.
I introduced it for the unit tests enforce the copy of fields. This way, when adding more tests we can see if copy vs. non-copy of fields makes a difference.
ruflin
reviewed
Dec 5, 2017
I reproduced + verified the issue via unit tests. Really need an additional system test here? |
|
@urso Not required, only nice to have. Glad we have the unit tests. |
ruflin
approved these changes
Dec 6, 2017
| @@ -39,7 +39,9 @@ type processorFn struct { | |||
| // 8. (P) pipeline processors list | |||
| // 9. (P) (if publish/debug enabled) log event | |||
| // 10. (P) (if output disabled) dropEvent | |||
| func (p *Pipeline) newProcessorPipeline( | |||
| func newProcessorPipeline( | |||
| info beat.Info, | |||
There was a problem hiding this comment.
Why did you decide to pass info instead of using it as variable as before?
There was a problem hiding this comment.
The info is used for the debug print. It used to be part of the pipeline variable. I removed the method and created a constructor passing only those information required to build the processor pipeline. The decoupling simplified the construction of the processor pipeline in the unit tests.
|
@urso Can you take care of the backports? |
|
@urso I think we also need to backport this to 6.0? |
urso
pushed a commit
to urso/beats
that referenced
this pull request
Dec 7, 2017
elastic#5808) The user global and local fields configurations must be merged on pipeline setup. The DeepUpdate method did only keep references to the globally shared fields, such that local fields settings do overwrite the shared global state. If more then one beat.Client instance did set local fields, the global state (and all fields added to an event) will be given by the last client being instantiated. The fix Clones the global fields before merging, such that the beat.Client fields operate fully in isolation. (cherry picked from commit 08eb2f5)
urso
pushed a commit
to urso/beats
that referenced
this pull request
Dec 7, 2017
elastic#5808) The user global and local fields configurations must be merged on pipeline setup. The DeepUpdate method did only keep references to the globally shared fields, such that local fields settings do overwrite the shared global state. If more then one beat.Client instance did set local fields, the global state (and all fields added to an event) will be given by the last client being instantiated. The fix Clones the global fields before merging, such that the beat.Client fields operate fully in isolation. (cherry picked from commit 08eb2f5)
ruflin
pushed a commit
that referenced
this pull request
Dec 7, 2017
#5808) (#5836) The user global and local fields configurations must be merged on pipeline setup. The DeepUpdate method did only keep references to the globally shared fields, such that local fields settings do overwrite the shared global state. If more then one beat.Client instance did set local fields, the global state (and all fields added to an event) will be given by the last client being instantiated. The fix Clones the global fields before merging, such that the beat.Client fields operate fully in isolation. (cherry picked from commit 08eb2f5)
ruflin
pushed a commit
that referenced
this pull request
Dec 8, 2017
#5808) (#5825) The user global and local fields configurations must be merged on pipeline setup. The DeepUpdate method did only keep references to the globally shared fields, such that local fields settings do overwrite the shared global state. If more then one beat.Client instance did set local fields, the global state (and all fields added to an event) will be given by the last client being instantiated. The fix Clones the global fields before merging, such that the beat.Client fields operate fully in isolation. (cherry picked from commit 08eb2f5)
leweafan
pushed a commit
to leweafan/beats
that referenced
this pull request
Apr 28, 2023
elastic#5808) (elastic#5825) The user global and local fields configurations must be merged on pipeline setup. The DeepUpdate method did only keep references to the globally shared fields, such that local fields settings do overwrite the shared global state. If more then one beat.Client instance did set local fields, the global state (and all fields added to an event) will be given by the last client being instantiated. The fix Clones the global fields before merging, such that the beat.Client fields operate fully in isolation. (cherry picked from commit a16dc30)
leweafan
pushed a commit
to leweafan/beats
that referenced
this pull request
Apr 28, 2023
elastic#5808) (elastic#5836) The user global and local fields configurations must be merged on pipeline setup. The DeepUpdate method did only keep references to the globally shared fields, such that local fields settings do overwrite the shared global state. If more then one beat.Client instance did set local fields, the global state (and all fields added to an event) will be given by the last client being instantiated. The fix Clones the global fields before merging, such that the beat.Client fields operate fully in isolation. (cherry picked from commit a16dc30)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves: #5795
The user global and local fields configurations must be merged
on pipeline setup. The DeepUpdate method did only keep references to the
globally shared fields, such that local fields settings do overwrite
the shared global state. If more then one beat.Client instance did set
local fields, the global state (and all fields added to an event) will
be given by the last client being instantiated.
The fix Clones the global fields before merging, such that the
beat.Client fields operate fully in isolation.