Do not store context in structs, and other fixes #84
Conversation
Stop storing context in structs, and instead pass a context as a function argument in the places where support for cancellation is needed. Much of the functionality does not need to support concelation, and even in some cases where a request may have been canceled, letting the function complete is preferable to avoid parital updates to MFS. Cancellation is only needed in places where a caller may be waiting for a signal that something has completed. - Handle additional TODO items in code - Fix potential race condition - Close() always stops republisher; fixes possible goroutine leak - Republisher can pick up new values during retry loop.
|
Thank you for submitting this PR!
Getting other community members to do a review would be great help too on complex PRs (you can ask in the chats/forums). If you are unsure about something, just leave us a comment.
We currently aim to provide initial feedback/triaging within two business days. Please keep an eye on any labelling actions, as these will indicate priorities and status of your contribution. |
There was a problem hiding this comment.
more generally, given that the dag service implementation may translate into a stall - either going out to network, or resulting in a datastore operation, it seems like having a context that can be canceled as the program exists is preferable to running those operations with context.background?
Agree that keeping a context on the struct isn't a good pattern, but is there a problem with keeping it as a first argument on methods and living for the duration of the method?
| @@ -117,13 +114,13 @@ func (d *Directory) localUpdate(c child) (*dag.ProtoNode, error) { | |||
| return nil, dag.ErrNotProtobuf | |||
| } | |||
|
|
|||
| err = d.dagService.Add(d.ctx, nd) | |||
There was a problem hiding this comment.
is it worth having context still an argument to constructor and threaded through to things like this, but not saved in the struct?
There was a problem hiding this comment.
I have been considering this, and where the use of context was removed and replaced with context.Background() are the places that I thought it was better to not support cancellation at all.
Suppose there is a single request that creates a new directory, puts some data into that directory, deletes the data from some other place. Do we really want cancellation in the middle of this? If a request is canceled or timed out due to a stall in a lower level (e.g. datastore), it may better to let the processing continue and either eventually succeed or error out. APIs like Close and Flush probably should not support cancellation at all. Thoughts?
Contexts spread like viruses, so I was hoping to limit the spread to where absolutely necessary.
@Stebalien Do you see there is a need to thread contexts through the APIs? If so I will make that change.
There was a problem hiding this comment.
This should take a context (from Flush). Flush should support cancellation as it might need to write a lot of data (and that update may need to be canceled).
|
I am concerned about is that this PR makes breaking API changes. If that is going to be a problem, then I can make a smaller PR with just the fixes and TODO items, and save the API changes for when this moves to another repo. |
| @@ -117,13 +114,13 @@ func (d *Directory) localUpdate(c child) (*dag.ProtoNode, error) { | |||
| return nil, dag.ErrNotProtobuf | |||
| } | |||
|
|
|||
| err = d.dagService.Add(d.ctx, nd) | |||
There was a problem hiding this comment.
This should take a context (from Flush). Flush should support cancellation as it might need to write a lot of data (and that update may need to be canceled).
| @@ -211,7 +208,7 @@ func (d *Directory) Uncache(name string) { | |||
| // childFromDag searches through this directories dag node for a child link | |||
| // with the given name | |||
| func (d *Directory) childFromDag(name string) (ipld.Node, error) { | |||
| return d.unixfsDir.Find(d.ctx, name) | |||
| return d.unixfsDir.Find(context.Background(), name) | |||
There was a problem hiding this comment.
Yeah, in general, never use context.Background() unless something really shouldn't be cancelable. This here will make canceling reads impossible.
|
Closing to save API changes for a later incarnation of mfs. |
Stop storing context in structs, and instead pass a context as a function argument in the places where support for cancellation is needed.
Much of the functionality does not need to support cancelation, and even in some cases where a request may have been canceled, letting the function complete is preferable to avoid partial updates to MFS. Cancellation only remains in places where a caller may be waiting for a signal that something has completed.
This PR will replace #58
Part of the motivation for this PR is in preparation for moving this package into a new repo shared by
go-ipfs-filesandgo-unixfs, so that fixes can be reviewed before placing files in a new repo. This PR can be canceled if a more complete rewrite is preferred to fixing the issues handled by this PR.