Minikube start issues - Podman v2.0.2 - Fedora 32

[ghost]

Steps to reproduce the issue:

1. minikube start --driver=podman --container-runtime=cri-o

Full output of failed command:

minikubepodman2error.log

Full output of minikube start command used, if not already included:
Logs provided above.

Optional: Full output of minikube logs command:

~ ❯ minikube logs                                                                                                                                                                                                                           ⏎

💣  Unable to get machine status: state: unknown state "minikube": sudo -n podman container inspect minikube --format={{.State.Status}}: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube

Podman Version:

Version:      2.0.2
API Version:  1
Go Version:   go1.14.3
Built:        Thu Jan  1 01:00:00 1970
OS/Arch:      linux/amd64

Minikube version:

minikube version: v1.12.0
commit: c83e6c47124b71190e138dbc687d2556d31488d6

Most likely related to issues #8587 and #8674

[afbjorklund]

Most likely related to issues #8587 and #8674

Yeah, Fedora is not really a good environment for doing Kubernetes development these days... Too much have changed.

docker -> podman
(now varlink -> rest)
containerd -> cri-o
cgroups v1 -> cgroups v2

You might want to check out the supported OpenShift and CRC while we get things going again (which might take a while)

Also #7992 and #3552

[mazzystr]

Crap! Now I'm hitting problems with Fedora 32 and minikube ....

$ minikube start --driver=podman --container-runtime=cri-o --disk-size=50g --memory=8096m --apiserver-ips=10.88.0.1,10.88.0.2,10.88.0.3,10.88.0.4,10.88.0.5,10.88.0.6,10.88.0.7,10.88.0.8
😄  minikube v1.11.0 on Fedora 32
✨  Using the podman (experimental) driver based on existing profile

❗  'podman' driver reported an issue: Invalid character(s) found in patch number "2\n"
💡  Suggestion: Cant verify minimum required version for podman . See podman website for installation guide.
📘  Documentation: https://podman.io/getting-started/installation.html

💣  Failed to validate 'podman' driver

$ minikube start --driver=podman --container-runtime=cri-o --disk-size=50g --memory=8096m --apiserver-ips=10.88.0.1,10.88.0.2,10.88.0.3,10.88.0.4,10.88.0.5,10.88.0.6,10.88.0.7,10.88.0.8
😄  minikube v1.12.0 on Fedora 32
✨  Using the podman (experimental) driver based on user configuration
👍  Starting control plane node minikube in cluster minikube
🔥  Creating podman container (CPUs=2, Memory=8096MB) ...
✋  Stopping "minikube" in podman ...
🔥  Deleting "minikube" in podman ...
🤦  StartHost failed, but will try again: creating host: create: creating: create kic node: create container: sudo -n podman run --cgroup-manager cgroupfs -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --volume minikube:/var:exec --cpus=2 -e container=podman --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 gcr.io/k8s-minikube/kicbase:v0.0.10: exit status 126
stdout:

stderr:
time="2020-07-13T13:11:17-07:00" level=error msg="Error adding network: failed to add route '{0.0.0.0 00000000} via 10.244.0.1 dev eth0': network is unreachable"
time="2020-07-13T13:11:17-07:00" level=error msg="Error while adding pod to CNI network \"podman\": failed to add route '{0.0.0.0 00000000} via 10.244.0.1 dev eth0': network is unreachable"
Error: error configuring network namespace for container 64e5ed2f26d5deafd695ebc454912dc3a8996579d3d42bb87dad9df7787cca9d: failed to add route '{0.0.0.0 00000000} via 10.244.0.1 dev eth0': network is unreachable

🔥  Creating podman container (CPUs=2, Memory=8096MB) ...
😿  Failed to start podman container. "minikube start" may fix it: creating host: create: creating: setting up container node: creating volume for minikube container: sudo -n podman volume create minikube --label name.minikube.sigs.k8s.io=minikube --label created_by.minikube.sigs.k8s.io=true: exit status 125
stdout:

stderr:
Error: volume with name minikube already exists: volume already exists


💣  error provisioning host: Failed to start host: creating host: create: creating: setting up container node: creating volume for minikube container: sudo -n podman volume create minikube --label name.minikube.sigs.k8s.io=minikube --label created_by.minikube.sigs.k8s.io=true: exit status 125
stdout:

stderr:
Error: volume with name minikube already exists: volume already exists


😿  minikube is exiting due to an error. If the above message is not useful, open an issue:
👉  https://github.com/kubernetes/minikube/issues/new/choose

--driver=none works....

$ minikube start --driver=none --container-runtime=cri-o --disk-size=50g --memory=8096m
😄  minikube v1.12.0 on Fedora 32
✨  Using the none driver based on existing profile
❗  The 'none' driver does not respect the --memory flag
❗  Using the 'crio' runtime with the 'none' driver is an untested configuration!
❗  The 'none' driver does not respect the --memory flag
❗  Using the 'crio' runtime with the 'none' driver is an untested configuration!
👍  Starting control plane node minikube in cluster minikube
🔄  Restarting existing none bare metal machine for "minikube" ...
ℹ️  OS release is Fedora 32 (Thirty Two)
🎁  Preparing Kubernetes v1.18.3 on CRI-O  ...
🔗  Configuring bridge CNI (Container Networking Interface) ...
🤹  Configuring local host environment ...

❗  The 'none' driver is designed for experts who need to integrate with an existing VM
💡  Most users should use the newer 'docker' driver instead, which does not require root!
📘  For more information, see: https://minikube.sigs.k8s.io/docs/reference/drivers/none/

❗  kubectl and minikube configuration will be stored in /root
❗  To use kubectl or minikube commands as your own user, you may need to relocate them. For example, to overwrite your own settings, run:

    ▪ sudo mv /root/.kube /root/.minikube $HOME
    ▪ sudo chown -R $USER $HOME/.kube $HOME/.minikube

💡  This can also be done automatically by setting the env var CHANGE_MINIKUBE_NONE_USER=true
🔎  Verifying Kubernetes components...
🌟  Enabled addons: default-storageclass, storage-provisioner
🏄  Done! kubectl is now configured to use "minikube"

$ kubectl get nodes
NAME    STATUS   ROLES    AGE   VERSION
cube0   Ready    master   31s   v1.18.3

[afbjorklund]

The error on 1.11 is known (#8560) , it was trying to parse the newline (\n)

Root cause here is that podman v2 adds a newline to the version, where v1 didn't.

# podman version --format "{{.Version}}"
1.8.2#

# podman version --format "{{.Version}}"
2.0.2
#

There is not enough info from the first start failure, unfortunately exit status 126

[medyagh]

podman 2 is not supported by minikube due to a lot of podman 2 issues