Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Close ClientStreaming on context cancellation #152

Merged
merged 9 commits into from
Jan 5, 2024

Conversation

emcfarlane
Copy link
Collaborator

@emcfarlane emcfarlane commented Nov 16, 2023

CloseRequest and CloseResponse must both be called before the span can be ended. Remove onCloseCalled bool as it's easy to derive.

Updates

On CloseResponse we must have stopped the connection and it is now safe to close the span. To handle context cancellation and ensure the span is always closed context.AfterFunc is added with a simpler backport for !go1.21.

CloseRequest and CloseResponse must both be called before the span can
be ended. Remove onCalledClose bool as it's easy to derive.
@emcfarlane emcfarlane changed the title Remove onCalledClosed bool Remove onCloseCalled bool Nov 16, 2023
@jhump
Copy link
Member

jhump commented Dec 6, 2023

CloseRequest and CloseResponse must both be called before the span can be ended.

Is this really true? I think only CloseResponse must be called. If the response side is closed, the request side cannot be open, right? If that's not true, I think there's probably a bug (maybe even a potential deadlock issue) in the underlying connect-go library.

FWIW, I also think that cancelling the context used to create the RPC can be done, instead of calling CloseResponse. But I suppose the only way to catch that, w/out a call to CloseResponse, would be to add a goroutine that awaits ctx.Done() (which might be more resources than we should use for instrumentation...).

@emcfarlane
Copy link
Collaborator Author

@jhump my understanding was CloseRequest and CloseResponse are used as notifiers of the end of input/output and depending on the application use case may be called in any order.

context.AfterFunc would be a nice use case for this cleanup logic.

@jhump
Copy link
Member

jhump commented Dec 6, 2023

may be called in any order

This doesn't make sense to me. Once CloseResponse returns, the whole operation is done. I don't see how calling CloseRequest can possibly do anything other than be a no-op at that point. A client cannot half-close only the receive side of an HTTP/2 stream -- it can only half-close the send side.

context.AfterFunc would be a nice use case for this cleanup logic.

Ooh, nice. Yeah that's the way to go. (And a build tag could be used to provide a more heavy-weight implementation for folks not running Go 1.21.)

@emcfarlane
Copy link
Collaborator Author

emcfarlane commented Dec 13, 2023

@jhump updated with a context.AfterFunc impl. I think CloseRequest and CloseResponse will always be called but yep on CloseResponse return we should be closing and don't have to wait for CloseRequest as the ResponseBody close won't be able to continue sending requests.

Related issue around recording metrics on ctx cancel: open-telemetry/opentelemetry-go#4671

@emcfarlane emcfarlane changed the title Remove onCloseCalled bool Close ClientStreaming on context cancellation Dec 13, 2023
Copy link
Member

@jhump jhump left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nit about sync.Once vs atomic.Bool. Otherwise, LGTM.

context_legacy.go Outdated Show resolved Hide resolved
payloadinterceptor.go Outdated Show resolved Hide resolved
@jhump
Copy link
Member

jhump commented Dec 21, 2023

@emcfarlane, looks like there are merge conflicts from #153 that must be addressed

@emcfarlane emcfarlane requested a review from jhump December 22, 2023 09:42
@jhump jhump merged commit 0518f32 into connectrpc:main Jan 5, 2024
5 checks passed
@emcfarlane emcfarlane deleted the ed/remove-onclose branch January 5, 2024 19:01
akshayjshah pushed a commit that referenced this pull request Jun 24, 2024
Fix a data race for a streaming client on close when a the context is
cancelled. This was introduced in #152 where we changed the behaviour to
trigger a close on context cancellation. The state must be locked to
avoid a data race with Send/Receive calls.

Closes #170, #171.

---------

Signed-off-by: Edward McFarlane <emcfarlane@buf.build>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants