Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ipfs/kubo crashes docker desktop completely. #6650

Closed
3 tasks done
ChristianPavilonis opened this issue Dec 26, 2022 · 5 comments
Closed
3 tasks done

ipfs/kubo crashes docker desktop completely. #6650

ChristianPavilonis opened this issue Dec 26, 2022 · 5 comments

Comments

@ChristianPavilonis
Copy link

ChristianPavilonis commented Dec 26, 2022

  • I have tried with the latest version of Docker Desktop
  • I have tried disabling enabled experimental features
  • I have uploaded Diagnostics
  • Diagnostics ID: 9E96C75E-3E09-46EC-8B41-7CF8A54DBE1B/20221226201640

Expected behavior

Docker desktop to handle the error gracefully if a container fails.

Actual behavior

After ipfs/kubo runs for a few minutes it will completely crash docker desktop and I can't quit it or restart docker. I have to manually kill the processes.
The container crashes with ERRO[0108] error waiting for container: invalid character 'c' looking for beginning of value

Information

I've reset to factory settings, I've uninstalled and reinstalled. I've installed older docker versions (4.14.1).
This happens every time I run the kubo container.

  • macOS Version: 13.1 (22C65)
  • Intel chip or Apple chip: Apple M1 Max
  • Docker Desktop Version: 4.15.0

Output of /Applications/Docker.app/Contents/MacOS/com.docker.diagnose check

1 : The test: is the com.docker.vmnetd process responding?
Failed with: failed to ping vmnetd with error: failed to connect to /var/run/com.docker.vmnetd.sock: is vmnetd running?: dial unix /var/run/com.docker.vmnetd.sock: connect: no such file or directory

The com.docker.vmnetd process is needed to create symlinks for CLIs in your path.

Steps to reproduce the behavior

  1. run ipfs/kubo:latest
  2. wait
  3. error

Update

It seems to break with older versions of the image also (ipfs/kubo:v0.16.0) never used to have this problem. I haven't had a problem with any other images yet either.

@Neurone
Copy link

Neurone commented Jan 3, 2023

Same problem here with kubo 17.0.
I'm using a MacOS Ventura 13.1 (I tried also with 13.0, same problem) Apple M1.
I did some analysis and I think the error can be with vpnkit, because in my case sometime it crashes with "segmentation faul", sometimes with "bus error":

cd ~/Library/Containers/com.docker.docker/Data/log/
❯ grep -R "shutdown by" *
host/com.docker.backend.log:[2023-01-03T16:54:59.298080000Z][com.docker.backend][I] com.docker.vpnkit with pid: 3149 shutdown by signal: bus error
host/com.docker.backend.log:[2023-01-03T17:29:21.893046000Z][com.docker.backend][I] com.docker.vpnkit with pid: 1341 shutdown by signal: segmentation fault
host/com.docker.backend.log.0:[2023-01-03T16:28:08.827456000Z][com.docker.backend][I] com.docker.vpnkit with pid: 55584 shutdown by signal: bus error
host/com.docker.backend.log.2:[2022-12-28T00:05:50.146188000Z][com.docker.backend][I] com.docker.vpnkit with pid: 2267 shutdown by signal: bus error
host/com.docker.backend.log.2:[2022-12-28T00:29:42.978589000Z][com.docker.backend][I] com.docker.vpnkit with pid: 23840 shutdown by signal: segmentation fault
host/com.docker.backend.log.4:[2022-12-27T22:47:04.000551000Z][com.docker.backend][I] com.docker.vpnkit with pid: 2526 shutdown by signal: segmentation fault
host/com.docker.backend.log.4:[2022-12-27T23:34:15.190261000Z][com.docker.backend][I] electron pid 1686 shutdown by signal: killed
host/com.docker.backend.log.4:[2022-12-27T23:41:38.258816000Z][com.docker.backend][I] com.docker.vpnkit with pid: 3466 shutdown by signal: segmentation fault
host/com.docker.backend.log.4:[2022-12-27T23:42:17.019528000Z][com.docker.backend][I] electron pid 3459 shutdown by signal: killed
host/com.docker.backend.log.3:[2022-12-27T23:45:59.011219000Z][com.docker.backend][I] com.docker.vpnkit with pid: 6780 shutdown by signal: terminated
host/com.docker.backend.log.3:[2022-12-27T23:46:03.489579000Z][com.docker.backend][I] electron pid 6773 shutdown by signal: killed
host/com.docker.backend.log.3:[2022-12-27T23:55:26.105775000Z][com.docker.backend][I] com.docker.vpnkit with pid: 16461 shutdown by signal: bus error

@nicks
Copy link

nicks commented Jan 3, 2023

can you check if the debug build here resolves this issue for you? #6530 (comment)

@Neurone
Copy link

Neurone commented Jan 3, 2023

Yes, this test build by @djs55 solved the issue for me!

As a side note, I don't see networking log messages inside com.docker.vpnkit.log anymore, and now I see hundreds of continuously connection reset by peer in com.docker.backend.log:

❯ tail -f com.docker.backend.log
...
[2023-01-03T18:48:15.234685000Z][com.docker.backend][I] error copying: read tcp 192.168.2.219:60621->80.240.19.234:4001: read: connection reset by peer
[2023-01-03T18:48:15.337236000Z][com.docker.backend][I] error copying: read tcp 192.168.2.219:60590->45.76.100.131:4001: read: connection reset by peer
[2023-01-03T18:48:16.037641000Z][com.docker.backend][I] error copying: read tcp 192.168.2.219:60198->145.40.96.83:4001: read: connection reset by peer
[2023-01-03T18:48:16.381921000Z][com.docker.backend][I] error copying: read tcp 192.168.2.219:60202->64.31.25.87:4001: read: connection reset by peer
[2023-01-03T18:48:16.553505000Z][com.docker.backend][I] error copying: read tcp 192.168.2.219:60683->66.42.92.230:4001: read: connection reset by peer
[2023-01-03T18:48:16.598116000Z][com.docker.backend][I] error copying: read tcp 192.168.2.219:59908->118.45.210.183:4001: read: connection reset by peer
[2023-01-03T18:48:17.043140000Z][com.docker.backend][I] error copying: read tcp 192.168.2.219:60715->185.130.47.34:4001: read: connection reset by peer
[2023-01-03T18:48:17.043419000Z][com.docker.backend][I] error copying: read tcp 192.168.2.219:60714->185.130.47.226:4001: read: connection reset by peer
...

@ChristianPavilonis
Copy link
Author

I updated to 4.16.2 and it's working now.

@docker-robott
Copy link
Collaborator

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

/lifecycle locked

@docker docker locked and limited conversation to collaborators Mar 5, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants