-
Notifications
You must be signed in to change notification settings - Fork 2.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make datanode load statslog lazy if SkipBFStatsLog is true #23779
Conversation
@congqixia ut workflow job failed, comment |
Codecov Report
@@ Coverage Diff @@
## 2.2.0 #23779 +/- ##
==========================================
+ Coverage 80.63% 80.68% +0.04%
==========================================
Files 713 713
Lines 101662 101825 +163
==========================================
+ Hits 81979 82153 +174
+ Misses 16480 16457 -23
- Partials 3203 3215 +12
|
80fe40a
to
e578a90
Compare
/lgtm |
internal/datanode/channel_meta.go
Outdated
if Params.DataNodeCfg.SkipBFStatsLoad { | ||
// mark segment lazy loading | ||
s.setLoadingLazy(true) | ||
go func() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
didn't really make any sense to me
Why we go the function out but mean time we use a pool?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
removed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Recalled why add a goroutine to submit task into the work pool: if the worker pool is full, the Submit call will block here until there are some available worker.
e578a90
to
c36f26a
Compare
c36f26a
to
8f30eaf
Compare
@congqixia ut workflow job failed, comment |
1 similar comment
@congqixia ut workflow job failed, comment |
8f30eaf
to
6aa7e85
Compare
@congqixia E2e jenkins job failed, comment |
@congqixia E2e jenkins job failed, comment |
1 similar comment
@congqixia E2e jenkins job failed, comment |
if err != nil { | ||
// TODO if not retryable, add rebuild statslog logic | ||
log.Warn("failed to lazy load statslog for segment", zap.Error(err)) | ||
if c.retryableLoadError(err) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should we take of close here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The closed
check is at the beginning of this resubmit function
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
6aa7e85
to
e5eb246
Compare
/lgtm |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: congqixia, xiaofan-luan The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
See also #22994
Related to #23477
/kind improvement