-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[exporterhelper] Fix potential deadlock in the batch sender #10315
Conversation
@carsonip PTAL |
8c3311d
to
d658832
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #10315 +/- ##
==========================================
- Coverage 92.58% 92.57% -0.01%
==========================================
Files 387 387
Lines 18254 18250 -4
==========================================
- Hits 16900 16895 -5
- Misses 1008 1009 +1
Partials 346 346 ☔ View full report in Codecov by Sentry. |
d658832
to
09b2518
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
bugfix lgtm, I've managed to reproduce the deadlock with an updated test (see code suggestion)
Concurrent handling of the flush timeouts can run into a deadlock when a batch is simultaneously sent by reaching the minimum size and flush timeout. The deadlock can happen on the following lines: - https://github.com/open-telemetry/opentelemetry-collector/blob/115bc8e28e009ca93565dc4deb4cf6608fa63622/exporter/exporterhelper/batch_sender.go#L131 - https://github.com/open-telemetry/opentelemetry-collector/blob/115bc8e28e009ca93565dc4deb4cf6608fa63622/exporter/exporterhelper/batch_sender.go#L87 Co-authored-by: Carson Ip <carsonip@users.noreply.github.com>
db130ff
to
4ca2db6
Compare
Thanks @carsonip! |
Concurrent handling of the flush timeouts can run into a deadlock when a batch is simultaneously sent by reaching the minimum size and flush timeout. The deadlock can happen on the following lines:
opentelemetry-collector/exporter/exporterhelper/batch_sender.go
Line 131 in 115bc8e
opentelemetry-collector/exporter/exporterhelper/batch_sender.go
Line 87 in 115bc8e