-
Notifications
You must be signed in to change notification settings - Fork 453
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use Seek() and Read() APIs for index/data files instead of mmap + share expensive seeker resources among seekers #1421
Conversation
|
||
var errorUnableToDetermineNumFieldsToSkip = errors.New("unable to determine num fields to skip") | ||
errorUnableToDetermineNumFieldsToSkip = errors.New("unable to determine num fields to skip") | ||
errorCalledDecodeBytesWithoutByteStreamDecoder = errors.New("called decodeBytes with out byte stream decoder") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: rename this to errThing... =
rather than errorThing... =
for consistency with other packages/etc.
src/dbnode/persist/fs/seek.go
Outdated
// Should never happen, either something is really wrong with the code or | ||
// the file on disk was corrupted | ||
for { | ||
currOffset, err := s.indexFd.Seek(0, os.SEEK_CUR) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this call necessary? Seems this is not needed because Seek(0, SEEK_CUR)
should cause no change as SEEK_CUR
means "seek relative to the current offset",
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can't we just keep track of currOffset
ourselves and avoid this syscall?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have looked at the different interfaces, looks difficult to actually call like .xmsgpackDecoder.Position()
or whatever which can call the reader underneath.
Maybe just add a comment here that in the future we can remove this Seek (and consequently syscall) call by tracking the position ourselves with some refactoring of interfaces/etc?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM but perhaps we add the comment about removing the Seek(0, SEEK_CUR)
syscall eventually?
5a7eb3b
to
ac4d608
Compare
Codecov Report
@@ Coverage Diff @@
## master #1421 +/- ##
========================================
+ Coverage 69.3% 70.9% +1.5%
========================================
Files 838 842 +4
Lines 71823 72006 +183
========================================
+ Hits 49838 51083 +1245
+ Misses 18687 17572 -1115
- Partials 3298 3351 +53
Continue to review full report at Codecov.
|
Had to create a new P.R for C.I reasons, see previous comments / discussion on #1299