error starting network: cannot use VNL

[rfay]

After the last batch of network stuff went in I wanted to try HEAD (macOS arm64) so tried brew unlink colima &&. brew install --HEAD --fetch-head colima then colima stop

But on colima start this is what I see:

rfay@rfay-tag1-m1:~/workspace/ddev$ colima start
INFO[0000] using docker runtime
INFO[0000] starting colima
INFO[0000] preparing network ...                         context=vm
INFO[0000] network dependencies missing                  context=vm
INFO[0000] sudo password may be required for setting up network dependencies  context=vm
Password:
WARN[0009] error starting network: error at 'preparing network': stat /Users/rfay/.colima/network/vmnet.ptp: no such file or directory  context=vm
INFO[0009] starting ...                                  context=vm
> msg="Using the existing instance \"colima\""
> msg="[hostagent] cannot use VNL \"/Users/rfay/.colima/network/vmnet.ptp\": sta
> msg="host agent process has exited: exit status 1"
FATA[0010] error starting vm: error at 'starting': exit status 1

[rfay]

I did a colima delete and started over, but same issue:

$ colima start --cpu 4 --memory 6 --disk 100
INFO[0000] starting colima
INFO[0000] preparing network ...                         context=vm
WARN[0005] error starting network: error at 'preparing network': stat /Users/rfay/.colima/network/vmnet.ptp: no such file or directory  context=vm
INFO[0005] creating and starting ...                     context=vm
> msg="Attempting to download the image from \"https://github.com/abiosoft/alpin
>
> 155.98 MiB / 167.88 MiB (92.91%) ? p/s
> 167.88 MiB / 167.88 MiB (100.00%) 293.26 MiB/stime="2022-03-19T12:56:58-06:00"
> msg="[hostagent] cannot use VNL \"/Users/rfay/.colima/network/vmnet.ptp\": sta
> msg="host agent process has exited: exit status 1"
FATA[0013] error starting vm: error at 'creating and starting': exit status 1

[abiosoft]

@rfay how recent was your install cos I pushed a fix about 30mins ago, can you kindly share the output of colima version?

Thanks.

[rfay]

I had HEAD-02932da (I see in backscroll)

Sorry, couldn't do anything so had to uninstall, delete, install stable, restart

[abiosoft]

It seems to work fine for me, I will await more reports. In the meantime, can you kindly share the output of /Users/rfay/.colima/network/vmnet.stderr?

Sorry, couldn't do anything so had to uninstall, delete, install stable, restart

As for this, it was an oversight from me and it has been fixed. It is now simply a warning and VM starts as normal, would only be missing reachable ip address.

[rfay]

With latest I was able to start up without deleting,

$ colima version
colima version HEAD-a5bb526
git commit: a5bb5262e58e91d3dc9ef1eb5176b4cca05e7846
rfay@rfay-tag1-m1:~/workspace/ddev$ colima start
INFO[0000] using docker runtime
INFO[0000] starting colima
INFO[0000] preparing network ...                         context=vm
WARN[0005] error starting network: error at 'preparing network': stat /Users/rfay/.colima/network/vmnet.ptp: no such file or directory  context=vm
WARN[0005] error setting up network, VM will not have a reachable IP address: stat /Users/rfay/.colima/network/vmnet.ptp: no such file or directory
INFO[0005] starting ...                                  context=vm
INFO[0027] provisioning ...                              context=docker
INFO[0027] starting ...                                  context=docker
INFO[0027] waiting for startup to complete ...           context=docker
INFO[0032] done

The error lines still seem alarming:

WARN[0005] error starting network: error at 'preparing network': stat /Users/rfay/.colima/network/vmnet.ptp: no such file or directory  context=vm
WARN[0005] error setting up network, VM will not have a reachable IP address: stat /Users/rfay/.colima/network/vmnet.ptp: no such file or directory

[rfay]

I'm running a full set of DDEV tests against colima HEAD, https://github.com/rfay/ddev/runs/5614470665?check_suite_focus=true ...

[rfay]

Full test suite completed successfully, with no port-already-occupied failures. It hasn't always failed, but 50% probably, so a clean run is a good sign.

[abiosoft]

There have been even more fixes and things should be fine now.

Kindly share the git commit hash of the version you test with. The version may differ based on how the binary is generated but the git commit is consistent.

$ colima version
colima version v0.3.4-4-g5e2e413
git commit: 5e2e41310e595553dcdc29ba45827d4030af37bb

[rfay]

I have been providing the version each time, see #229 (comment)

[rfay]

Thanks for the good work on this. I'm starting another full DDEV test suite on

$ colima version
colima version HEAD-5e2e413
git commit: 5e2e41310e595553dcdc29ba45827d4030af37bb

runtime: docker
arch: aarch64
client: v20.10.13
server: v20.10.11

[rfay]

With latest 5e2e413 I'm seeing failures on late-port-binding issues again. https://github.com/rfay/ddev/runs/5617784390?check_suite_focus=true

" Unable to listen on required ports, port 8037 is already in use,"

I hadn't seen that in the last couple of runs with earlier HEAD. But of course it's always been intermittent, so maybe that late-port-binding and late-port-release problem is just still active.

[abiosoft]

Oops, I think there has been some miscommunication or assumptions on my end. I was thinking you'd modify the test suite to try to reach the VM via the IP address rather than rely on port forwarding, based on the comments #189 (comment), #189 (comment) and #189 (comment)

The way port forwarding works, the intermittent issue will persist due to the delay, that hasn't changed. What has been added however is that the VM can be reached directly. e.g. rather than hope 127.0.0.1:8080 is forwarded/released, 192.168.106.2:8080 will have a predictable behaviour.

[rfay]

The ddev functionality and tests are the same tests that run on macOS with Docker Desktop, Windows, WSL2, and Linux. I don't anticipate changing ddev to work in a completely different way for colima. I'm sure you understand.

Thanks for explaining!

[rfay]

I've been thinking about your comment

I was thinking you'd modify the test suite to try to reach the VM via the IP address rather than rely on port forwarding, based on the comments #189 (comment), #189 (comment) and #189 (comment)

And I don't think I understand. What ddev does is run a container that uses network capabilities. Ports are used both within the docker network (containers talking to each other) and they are also bound to the host, so that a web browser, for example, can reach them.

What am I missing here as you talk about port forwarding?