Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

segment ebuf can be pre-allocated for better performance #23

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

yihuang
Copy link

@yihuang yihuang commented Aug 7, 2023

the number of allocations are clearly dropped, cpu time is also improved marginally.

Copy link

@semihbkgr semihbkgr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The size check, determining whether the buffer size exceeds the SegmentSize, is applied after appending the new entry data to the buffer.

var epos bpos
s.ebuf, epos = l.appendEntry(s.ebuf, b.entries[i].index, data)
s.epos = append(s.epos, epos)
if len(s.ebuf) >= l.opts.SegmentSize {
    // segment has reached capacity, cycle now

When the buffer size surpasses its capacity, it re-allocates memory to fit the underlying array.
Therefore, to fully leverage pre-allocation, I think we should ensure that the buffer's capacity is not exceeded.

@yihuang
Copy link
Author

yihuang commented Jan 2, 2024

the previous behavior allows the segment surpasses its capacity once with the last entry, and we don't have a limit of the size of an entry, it's not easy to preallocate for that I think.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants