Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache msgpack packer/unpacker #2559

Merged
merged 3 commits into from
Aug 16, 2019
Merged

Conversation

ganmacs
Copy link
Member

@ganmacs ganmacs commented Aug 14, 2019

Which issue(s) this PR fixes:

What this PR does / why we need it:

cache msgpack packer/unpacker not to create unnecessary objects.
https://gist.github.com/ganmacs/a620be919851d49bff3bf2de58da2052 shows decreasing CPU time of Fluent::EventStream#to_msgpack_stream and Fluent::MessagePackFactory::Mixin#msgpack_packer.

Docs Changes:

no need

Release Note:

same as title

@ganmacs ganmacs added the enhancement Feature request or improve operations label Aug 14, 2019
@ganmacs ganmacs force-pushed the cache-msgpack branch 3 times, most recently from afc8b57 to 5e5dfb3 Compare August 15, 2019 02:20
@ganmacs ganmacs marked this pull request as ready for review August 16, 2019 06:55
@ganmacs ganmacs self-assigned this Aug 16, 2019
@@ -182,7 +184,7 @@ def filter_stream(tag, es)

def optimized_filter_stream(tag, es)
new_es = MultiEventStream.new
es.each do |time, record|
es.each(unpacker: @unpacker) do |time, record|
Copy link
Member

@repeatedly repeatedly Aug 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this safe? event_router and pipeline can be called from multiple inputs.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch!
b8021e9

Signed-off-by: Yuta Iwama <ganmacs@gmail.com>
Signed-off-by: Yuta Iwama <ganmacs@gmail.com>
Signed-off-by: Yuta Iwama <ganmacs@gmail.com>
@repeatedly repeatedly merged commit c6c6c03 into fluent:master Aug 16, 2019
@ganmacs ganmacs deleted the cache-msgpack branch August 19, 2019 02:32
daipom added a commit to daipom/fluentd that referenced this pull request May 15, 2023
Fixes bug in c6c6c03 (fluent#2559).

Received incomplete data must not affect data from other senders.

Signed-off-by: Daijiro Fukuda <fukuda@clear-code.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Feature request or improve operations
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants