Wait to finish reading logs before calling Wait() on pipeline #1010
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This fixes errors like the following and chopped off
logs with the local backend:
The reason for this issue is that exec's Wait(), which is used
internally by the this backend, closes the stdout/stderr pipe, so we
must wait until all reading is done before calling it.
See https://pkg.go.dev/os/exec?utm_source=gopls#Cmd.StdoutPipe:
Instead of using a WaitGroup, one could call the reading synchronously,
but with the
step.Detached
case probably still needing it as a goroutine,it might become less readable.
An alternative solution could be to move this synchronization into the local backend
implementation itself, for example by constructing a new
io.Pipe()
, copying everythinginto it and waiting for this to be done before calling the exec
Wait()
.But I think it might be better in general to wait until all log has been read before considering
a pipeline finished.
I have also sometimes encountered errors like these:
Maybe it is related. In those cases, I did not get any logs at all in the webui. After the fix, I have not seen those anymore, but they were sporadic before too.