Fix the incorrect size calculation logic of FileSink. #9429
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
✅ Deploy Preview for meta-velox canceled.
|
mbasmanova
reviewed
Apr 10, 2024
There was a problem hiding this comment.
@wypb Thank you for the fix.
@xiaoxmeng Meng, would you help review this fix?
wypb
force-pushed
the
fix_memory
branch
2 times, most recently
from
07944c7
to
cb25213
Compare
@mbasmanova sure, will do |
xiaoxmeng
reviewed
Apr 11, 2024
| buffers.back().append(chars[i]); | ||
| } | ||
|
|
||
| EXPECT_EQ(buffers.size(), 2); |
There was a problem hiding this comment.
FYI: ASSERT__EQ put expected value second while EXPECT_EQ put the expected value first. Thanks!
There was a problem hiding this comment.
I have modified it to ASSERT__EQ.
|
@xiaoxmeng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@xiaoxmeng merged this pull request in d4f8d85. |
|
Conbench analyzed the 1 benchmark run on commit There were no benchmark performance regressions. 🎉 The full Conbench report has more details. |
yanngyoung
pushed a commit
to yanngyoung/velox
that referenced
this pull request
Apr 12, 2024
…tor#9429) Summary: I was writing unit tests for the Textfile writer internally and found that data written to `MemorySink` may be overwritten. I investigated the reason and found that `FileSink::writeImpl` can accept multiple buffers. https://github.com/facebookincubator/velox/blob/main/velox/dwio/common/FileSink.cpp#L65-L80 Every time the data of a buffer is written, we should update `FileSink#size_` instead of updating after writing all buffers. Because `MemorySink::write` relies on `FileSink#size_` to write data to `MemorySink#data_`. https://github.com/facebookincubator/velox/blob/main/velox/dwio/common/FileSink.cpp#L184-L189 CC: mbasmanova xiaoxmeng Pull Request resolved: facebookincubator#9429 Reviewed By: amitkdutta, kewang1024 Differential Revision: D56004384 Pulled By: xiaoxmeng fbshipit-source-id: c824c5e481866abef12f99d570d41a852bb8b2e0
Joe-Abraham
pushed a commit
to Joe-Abraham/velox
that referenced
this pull request
Jun 7, 2024
…tor#9429) Summary: I was writing unit tests for the Textfile writer internally and found that data written to `MemorySink` may be overwritten. I investigated the reason and found that `FileSink::writeImpl` can accept multiple buffers. https://github.com/facebookincubator/velox/blob/main/velox/dwio/common/FileSink.cpp#L65-L80 Every time the data of a buffer is written, we should update `FileSink#size_` instead of updating after writing all buffers. Because `MemorySink::write` relies on `FileSink#size_` to write data to `MemorySink#data_`. https://github.com/facebookincubator/velox/blob/main/velox/dwio/common/FileSink.cpp#L184-L189 CC: mbasmanova xiaoxmeng Pull Request resolved: facebookincubator#9429 Reviewed By: amitkdutta, kewang1024 Differential Revision: D56004384 Pulled By: xiaoxmeng fbshipit-source-id: c824c5e481866abef12f99d570d41a852bb8b2e0
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I was writing unit tests for the Textfile writer internally and found
that data written to
MemorySinkmay be overwritten.I investigated the reason and found that
FileSink::writeImplcan accept multiple buffers.https://github.com/facebookincubator/velox/blob/main/velox/dwio/common/FileSink.cpp#L65-L80
Every time the data of a buffer is written, we should update
FileSink#size_instead of updating after writing all buffers. Because
MemorySink::writerelies on
FileSink#size_to write data toMemorySink#data_.https://github.com/facebookincubator/velox/blob/main/velox/dwio/common/FileSink.cpp#L184-L189
CC: @mbasmanova @xiaoxmeng