Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minikube v1.20.0 not downloading images or starting on Pop!_OS 21.04 with Podman 3.0.1 Driver #12005

Closed
FilBot3 opened this issue Jul 19, 2021 · 12 comments
Labels
co/podman-driver podman driver issues kind/support Categorizes issue or PR as a support question.

Comments

@FilBot3
Copy link

FilBot3 commented Jul 19, 2021

Background

I had a working Podman setup with Minikube earlier this year, but recently recreated it after having issues with the awx-operator not working properly. Figured I would rebuild the minikube deployment and start from scratch. I pruned the Podman images, containers, and volumes using

sudo podman system prune -a -f

Then I ran minikube start --driver=podman and the following errors happened.

Troubleshooting

Steps to reproduce the issue:

  1. Install Podman from apt sources
sudo apt update && sudo apt install -y podman buildah skopeo
  1. Install minikube from .deb
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube_latest_amd64.deb
sudo dpkg -i minikube_latest_amd64.deb
  1. Attempt to start Minikube
minikube start --driver=podman

Full output of minikube logs command:

~ minikube logs

==> Audit <==
|--------------|-----------------|----------|--------|---------|-------------------------------|-------------------------------|
|   Command    |      Args       | Profile  |  User  | Version |          Start Time           |           End Time            |
|--------------|-----------------|----------|--------|---------|-------------------------------|-------------------------------|
| ip           |                 | minikube | filbot | v1.18.1 | Tue, 16 Mar 2021 20:54:14 CDT | Tue, 16 Mar 2021 20:54:15 CDT |
| ip           |                 | minikube | filbot | v1.18.1 | Tue, 16 Mar 2021 21:19:48 CDT | Tue, 16 Mar 2021 21:19:48 CDT |
| docker-env   |                 | minikube | filbot | v1.18.1 | Tue, 16 Mar 2021 21:35:17 CDT | Tue, 16 Mar 2021 21:35:17 CDT |
| docker-env   |                 | minikube | filbot | v1.18.1 | Tue, 16 Mar 2021 21:54:25 CDT | Tue, 16 Mar 2021 21:54:26 CDT |
| update-check |                 | minikube | filbot | v1.18.1 | Tue, 16 Mar 2021 22:24:20 CDT | Tue, 16 Mar 2021 22:24:20 CDT |
| ip           |                 | minikube | filbot | v1.18.1 | Tue, 16 Mar 2021 22:29:46 CDT | Tue, 16 Mar 2021 22:29:46 CDT |
| addons       | list            | minikube | filbot | v1.18.1 | Tue, 16 Mar 2021 22:40:48 CDT | Tue, 16 Mar 2021 22:40:48 CDT |
| addons       | -h              | minikube | filbot | v1.18.1 | Tue, 16 Mar 2021 22:41:39 CDT | Tue, 16 Mar 2021 22:41:39 CDT |
| stop         |                 | minikube | filbot | v1.18.1 | Tue, 16 Mar 2021 22:41:48 CDT | Tue, 16 Mar 2021 22:41:59 CDT |
| delete       |                 | minikube | filbot | v1.18.1 | Tue, 16 Mar 2021 22:42:07 CDT | Tue, 16 Mar 2021 22:42:10 CDT |
| update-check |                 | minikube | filbot | v1.18.1 | Wed, 17 Mar 2021 15:35:17 CDT | Wed, 17 Mar 2021 15:35:18 CDT |
| update-check |                 | minikube | filbot | v1.18.1 | Wed, 17 Mar 2021 15:48:13 CDT | Wed, 17 Mar 2021 15:48:14 CDT |
| update-check |                 | minikube | filbot | v1.18.1 | Wed, 17 Mar 2021 15:49:45 CDT | Wed, 17 Mar 2021 15:49:45 CDT |
| update-check |                 | minikube | filbot | v1.18.1 | Fri, 19 Mar 2021 21:51:43 CDT | Fri, 19 Mar 2021 21:51:44 CDT |
| update-check |                 | minikube | filbot | v1.18.1 | Mon, 22 Mar 2021 08:19:44 CDT | Mon, 22 Mar 2021 08:19:45 CDT |
| update-check |                 | minikube | filbot | v1.18.1 | Tue, 30 Mar 2021 10:52:24 CDT | Tue, 30 Mar 2021 10:52:34 CDT |
| update-check |                 | minikube | filbot | v1.18.1 | Tue, 30 Mar 2021 10:53:41 CDT | Tue, 30 Mar 2021 10:53:41 CDT |
| update-check |                 | minikube | filbot | v1.18.1 | Wed, 14 Apr 2021 14:20:39 CDT | Wed, 14 Apr 2021 14:20:39 CDT |
| update-check |                 | minikube | filbot | v1.18.1 | Thu, 15 Apr 2021 09:36:20 CDT | Thu, 15 Apr 2021 09:36:21 CDT |
| update-check |                 | minikube | filbot | v1.18.1 | Thu, 15 Apr 2021 11:12:18 CDT | Thu, 15 Apr 2021 11:12:18 CDT |
| update-check |                 | minikube | filbot | v1.18.1 | Thu, 15 Apr 2021 11:17:41 CDT | Thu, 15 Apr 2021 11:17:41 CDT |
| update-check |                 | minikube | filbot | v1.18.1 | Thu, 15 Apr 2021 11:41:03 CDT | Thu, 15 Apr 2021 11:41:04 CDT |
| update-check |                 | minikube | filbot | v1.18.1 | Thu, 15 Apr 2021 12:11:49 CDT | Thu, 15 Apr 2021 12:11:49 CDT |
| update-check |                 | minikube | filbot | v1.18.1 | Thu, 15 Apr 2021 12:26:16 CDT | Thu, 15 Apr 2021 12:26:17 CDT |
| update-check |                 | minikube | filbot | v1.18.1 | Mon, 26 Apr 2021 14:49:01 CDT | Mon, 26 Apr 2021 14:49:01 CDT |
| update-check |                 | minikube | filbot | v1.18.1 | Wed, 28 Apr 2021 17:06:01 CDT | Wed, 28 Apr 2021 17:06:01 CDT |
| update-check |                 | minikube | filbot | v1.18.1 | Thu, 29 Apr 2021 09:22:07 CDT | Thu, 29 Apr 2021 09:22:07 CDT |
| update-check |                 | minikube | filbot | v1.18.1 | Thu, 29 Apr 2021 10:19:24 CDT | Thu, 29 Apr 2021 10:19:24 CDT |
| update-check |                 | minikube | filbot | v1.18.1 | Wed, 19 May 2021 16:03:44 CDT | Wed, 19 May 2021 16:03:44 CDT |
| update-check |                 | minikube | filbot | v1.18.1 | Thu, 20 May 2021 08:17:11 CDT | Thu, 20 May 2021 08:17:11 CDT |
| update-check |                 | minikube | filbot | v1.18.1 | Fri, 21 May 2021 16:46:52 CDT | Fri, 21 May 2021 16:46:52 CDT |
| start        | --driver=podman | minikube | filbot | v1.20.0 | Thu, 27 May 2021 20:05:07 CDT | Thu, 27 May 2021 20:07:49 CDT |
| service      | list            | minikube | filbot | v1.20.0 | Thu, 27 May 2021 21:19:06 CDT | Thu, 27 May 2021 21:19:07 CDT |
| addons       | enable ingress  | minikube | filbot | v1.20.0 | Thu, 27 May 2021 21:40:24 CDT | Thu, 27 May 2021 21:40:46 CDT |
| ip           |                 | minikube | filbot | v1.20.0 | Thu, 27 May 2021 21:43:40 CDT | Thu, 27 May 2021 21:43:41 CDT |
| stop         |                 | minikube | filbot | v1.20.0 | Thu, 27 May 2021 22:02:36 CDT | Thu, 27 May 2021 22:02:49 CDT |
| delete       |                 | minikube | filbot | v1.20.0 | Thu, 27 May 2021 22:03:15 CDT | Thu, 27 May 2021 22:03:19 CDT |
| delete       |                 | minikube | filbot | v1.20.0 | Thu, 27 May 2021 22:04:41 CDT | Thu, 27 May 2021 22:04:42 CDT |
| start        | --driver=podman | minikube | filbot | v1.20.0 | Thu, 27 May 2021 22:06:29 CDT | Thu, 27 May 2021 22:07:53 CDT |
| addons       | enable ingress  | minikube | filbot | v1.20.0 | Thu, 27 May 2021 22:09:00 CDT | Thu, 27 May 2021 22:09:23 CDT |
| stop         |                 | minikube | filbot | v1.20.0 | Thu, 27 May 2021 23:02:18 CDT | Thu, 27 May 2021 23:02:30 CDT |
| start        | --driver=podman | minikube | filbot | v1.20.0 | Fri, 28 May 2021 15:35:43 CDT | Fri, 28 May 2021 15:36:15 CDT |
| stop         |                 | minikube | filbot | v1.20.0 | Fri, 28 May 2021 17:05:20 CDT | Fri, 28 May 2021 17:05:35 CDT |
| start        | --driver=podman | minikube | filbot | v1.20.0 | Fri, 04 Jun 2021 13:02:43 CDT | Fri, 04 Jun 2021 13:03:25 CDT |
| stop         |                 | minikube | filbot | v1.20.0 | Fri, 04 Jun 2021 13:49:07 CDT | Fri, 04 Jun 2021 13:49:19 CDT |
| start        | --driver=podman | minikube | filbot | v1.20.0 | Fri, 04 Jun 2021 14:16:50 CDT | Fri, 04 Jun 2021 14:17:16 CDT |
| stop         |                 | minikube | filbot | v1.20.0 | Fri, 04 Jun 2021 18:58:39 CDT | Fri, 04 Jun 2021 18:58:54 CDT |
| start        | --driver=podman | minikube | filbot | v1.20.0 | Mon, 14 Jun 2021 11:15:11 CDT | Mon, 14 Jun 2021 11:15:45 CDT |
| stop         |                 | minikube | filbot | v1.20.0 | Mon, 14 Jun 2021 14:16:06 CDT | Mon, 14 Jun 2021 14:16:21 CDT |
| update-check |                 | minikube | filbot | v1.20.0 | Wed, 23 Jun 2021 12:49:11 CDT | Wed, 23 Jun 2021 12:49:11 CDT |
| update-check |                 | minikube | filbot | v1.20.0 | Wed, 23 Jun 2021 12:51:02 CDT | Wed, 23 Jun 2021 12:51:02 CDT |
| start        | --driver=podman | minikube | filbot | v1.20.0 | Mon, 19 Jul 2021 11:19:18 CDT | Mon, 19 Jul 2021 11:19:46 CDT |
| stop         |                 | minikube | filbot | v1.20.0 | Mon, 19 Jul 2021 11:34:40 CDT | Mon, 19 Jul 2021 11:34:49 CDT |
| delete       |                 | minikube | filbot | v1.20.0 | Mon, 19 Jul 2021 11:35:07 CDT | Mon, 19 Jul 2021 11:35:13 CDT |
| start        | --driver=podman | minikube | filbot | v1.20.0 | Mon, 19 Jul 2021 11:35:21 CDT | Mon, 19 Jul 2021 11:36:02 CDT |
| addons       | -h              | minikube | filbot | v1.20.0 | Mon, 19 Jul 2021 11:36:29 CDT | Mon, 19 Jul 2021 11:36:29 CDT |
| addons       | list            | minikube | filbot | v1.20.0 | Mon, 19 Jul 2021 11:36:37 CDT | Mon, 19 Jul 2021 11:36:37 CDT |
| addons       | enable -h       | minikube | filbot | v1.20.0 | Mon, 19 Jul 2021 12:02:47 CDT | Mon, 19 Jul 2021 12:02:47 CDT |
| stop         |                 | minikube | filbot | v1.20.0 | Mon, 19 Jul 2021 12:11:28 CDT | Mon, 19 Jul 2021 12:11:36 CDT |
| delete       |                 | minikube | filbot | v1.20.0 | Mon, 19 Jul 2021 12:11:36 CDT | Mon, 19 Jul 2021 12:11:38 CDT |
|--------------|-----------------|----------|--------|---------|-------------------------------|-------------------------------|


==> Last Start <==
Log file created at: 2021/07/19 12:20:02
Running on machine: pop-os
Binary: Built with gc go1.16.1 for linux/amd64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0719 12:20:02.432459   70767 out.go:291] Setting OutFile to fd 1 ...
I0719 12:20:02.432647   70767 out.go:343] isatty.IsTerminal(1) = true
I0719 12:20:02.432661   70767 out.go:304] Setting ErrFile to fd 2...
I0719 12:20:02.432673   70767 out.go:343] isatty.IsTerminal(2) = true
I0719 12:20:02.432822   70767 root.go:316] Updating PATH: /home/filbot/.minikube/bin
I0719 12:20:02.433087   70767 out.go:298] Setting JSON to false
I0719 12:20:02.453649   70767 start.go:108] hostinfo: {"hostname":"pop-os","uptime":17197,"bootTime":1626698005,"procs":523,"os":"linux","platform":"debian","platformFamily":"debian","platformVersion":"bullseye/sid","kernelVersion":"5.11.0-7620-generic","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"host","hostId":"63c5b135-e8c7-4673-bd1c-91f260482952"}
I0719 12:20:02.453751   70767 start.go:118] virtualization: kvm host
I0719 12:20:02.455331   70767 out.go:170] 😄  minikube v1.20.0 on Debian bullseye/sid
I0719 12:20:02.456083   70767 driver.go:322] Setting default libvirt URI to qemu:///system
I0719 12:20:02.529039   70767 podman.go:121] podman version: 3.0.1
I0719 12:20:02.529992   70767 out.go:170] ✨  Using the podman driver based on existing profile
I0719 12:20:02.530018   70767 start.go:276] selected driver: podman
I0719 12:20:02.530027   70767 start.go:718] validating driver "podman" against &{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e Memory:7900 CPUs:2 DiskSize:20000 VMDriver: Driver:podman HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.2 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.2 ControlPlane:true Worker:true}] Addons:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: MultiNodeRequested:false}
I0719 12:20:02.530118   70767 start.go:729] status for podman: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc:}
I0719 12:20:02.530205   70767 cli_runner.go:115] Run: sudo -n podman system info --format json
I0719 12:20:02.934457   70767 info.go:281] podman info: {Host:{BuildahVersion:1.19.6 CgroupVersion:v1 Conmon:{Package:conmon: /usr/bin/conmon Path:/usr/bin/conmon Version:conmon version 2.0.25, commit: unknown} Distribution:{Distribution:pop Version:21.04} MemFree:16357789696 MemTotal:33532043264 OCIRuntime:{Name:crun Package:crun: /usr/bin/crun Path:/usr/bin/crun Version:crun version 0.17
commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL} SwapFree:4294434816 SwapTotal:4294434816 Arch:amd64 Cpus:16 Eventlogger:journald Hostname:pop-os Kernel:5.11.0-7620-generic Os:linux Rootless:false Uptime:4h 46m 37.2s (Approximately 0.17 days)} Registries:{Search:[]} Store:{ConfigFile:/etc/containers/storage.conf ContainerStore:{Number:0} GraphDriverName:overlay GraphOptions:{} GraphRoot:/var/lib/containers/storage GraphStatus:{BackingFilesystem:extfs NativeOverlayDiff:true SupportsDType:true UsingMetacopy:false} ImageStore:{Number:0} RunRoot:/var/run/containers/storage VolumePath:/var/lib/containers/storage/volumes}}
I0719 12:20:02.937161   70767 cni.go:93] Creating CNI manager for ""
I0719 12:20:02.937195   70767 cni.go:167] CNI unnecessary in this configuration, recommending no CNI
I0719 12:20:02.937219   70767 start_flags.go:273] config:
{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e Memory:7900 CPUs:2 DiskSize:20000 VMDriver: Driver:podman HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.2 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.2 ControlPlane:true Worker:true}] Addons:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: MultiNodeRequested:false}
I0719 12:20:02.938708   70767 out.go:170] 👍  Starting control plane node minikube in cluster minikube
I0719 12:20:02.938753   70767 cache.go:111] Beginning downloading kic base image for podman with docker
W0719 12:20:02.938768   70767 out.go:424] no arguments passed for "🚜  Pulling base image ...\n" - returning raw string
W0719 12:20:02.938796   70767 out.go:424] no arguments passed for "🚜  Pulling base image ...\n" - returning raw string
I0719 12:20:02.939559   70767 out.go:170] 🚜  Pulling base image ...
I0719 12:20:02.939592   70767 preload.go:98] Checking if preload exists for k8s version v1.20.2 and runtime docker
I0719 12:20:02.939633   70767 preload.go:106] Found local preload: /home/filbot/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v10-v1.20.2-docker-overlay2-amd64.tar.lz4
I0719 12:20:02.939637   70767 image.go:116] Checking for kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e in local cache directory
I0719 12:20:02.939647   70767 cache.go:54] Caching tarball of preloaded images
I0719 12:20:02.939680   70767 image.go:119] Found kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e in local cache directory, skipping pull
I0719 12:20:02.939683   70767 preload.go:132] Found /home/filbot/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v10-v1.20.2-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I0719 12:20:02.939693   70767 cache.go:131] kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e exists in cache, skipping pull
I0719 12:20:02.939705   70767 cache.go:57] Finished verifying existence of preloaded tar for  v1.20.2 on docker
I0719 12:20:02.939718   70767 cache.go:168] failed to download kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e, will try fallback image if available: <nil>
I0719 12:20:02.939738   70767 image.go:116] Checking for kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e in local cache directory
I0719 12:20:02.939756   70767 image.go:119] Found kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e in local cache directory, skipping pull
I0719 12:20:02.939767   70767 cache.go:131] kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e exists in cache, skipping pull
I0719 12:20:02.939778   70767 cache.go:168] failed to download kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e, will try fallback image if available: <nil>
I0719 12:20:02.939830   70767 profile.go:148] Saving config to /home/filbot/.minikube/profiles/minikube/config.json ...
E0719 12:20:02.940069   70767 cache.go:189] Error downloading kic artifacts:  failed to download kic base image or any fallback image
I0719 12:20:02.940088   70767 cache.go:194] Successfully downloaded all kic artifacts
I0719 12:20:02.940118   70767 start.go:313] acquiring machines lock for minikube: {Name:mkb45fe8b1deef9886cfd2b84df49d68ad57ae09 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0719 12:20:02.940321   70767 start.go:317] acquired machines lock for "minikube" in 164.066µs
I0719 12:20:02.940349   70767 start.go:93] Skipping create...Using existing machine configuration
I0719 12:20:02.940360   70767 fix.go:55] fixHost starting: 
I0719 12:20:02.940701   70767 cli_runner.go:115] Run: sudo -n podman container inspect minikube --format={{.State.Status}}
W0719 12:20:03.012786   70767 cli_runner.go:162] sudo -n podman container inspect minikube --format={{.State.Status}} returned with exit code 125
I0719 12:20:03.012828   70767 fix.go:108] recreateIfNeeded on minikube: state= err=unknown state "minikube": sudo -n podman container inspect minikube --format={{.State.Status}}: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 12:20:03.012841   70767 fix.go:113] machineExists: true. err=unknown state "minikube": sudo -n podman container inspect minikube --format={{.State.Status}}: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
W0719 12:20:03.012847   70767 fix.go:134] unexpected machine state, will restart: unknown state "minikube": sudo -n podman container inspect minikube --format={{.State.Status}}: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 12:20:03.013780   70767 out.go:170] 🔄  Restarting existing podman container for "minikube" ...
I0719 12:20:03.013831   70767 cli_runner.go:115] Run: sudo -n podman start --cgroup-manager cgroupfs minikube
W0719 12:20:03.068939   70767 cli_runner.go:162] sudo -n podman start --cgroup-manager cgroupfs minikube returned with exit code 125
I0719 12:20:03.069001   70767 cli_runner.go:115] Run: sudo -n podman inspect minikube
I0719 12:20:03.124913   70767 errors.go:78] Postmortem inspect ("sudo -n podman inspect minikube"): -- stdout --
[
    {
        "Name": "minikube",
        "Driver": "local",
        "Mountpoint": "/var/lib/containers/storage/volumes/minikube/_data",
        "CreatedAt": "2021-07-19T12:12:45.39965397-05:00",
        "Labels": {
            "created_by.minikube.sigs.k8s.io": "true",
            "name.minikube.sigs.k8s.io": "minikube"
        },
        "Scope": "local",
        "Options": {}
    }
]

-- /stdout --
I0719 12:20:03.124962   70767 cli_runner.go:115] Run: sudo -n podman logs --timestamps minikube
W0719 12:20:03.182403   70767 cli_runner.go:162] sudo -n podman logs --timestamps minikube returned with exit code 125
W0719 12:20:03.182470   70767 errors.go:83] Failed to get postmortem logs. sudo -n podman logs --timestamps minikube :sudo -n podman logs --timestamps minikube: exit status 125
stdout:

stderr:
Error: no container with name or ID minikube found: no such container
I0719 12:20:03.182618   70767 cli_runner.go:115] Run: sudo -n podman system info --format json
I0719 12:20:03.666363   70767 info.go:281] podman info: {Host:{BuildahVersion:1.19.6 CgroupVersion:v1 Conmon:{Package:conmon: /usr/bin/conmon Path:/usr/bin/conmon Version:conmon version 2.0.25, commit: unknown} Distribution:{Distribution:pop Version:21.04} MemFree:16351404032 MemTotal:33532043264 OCIRuntime:{Name:crun Package:crun: /usr/bin/crun Path:/usr/bin/crun Version:crun version 0.17
commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL} SwapFree:4294434816 SwapTotal:4294434816 Arch:amd64 Cpus:16 Eventlogger:journald Hostname:pop-os Kernel:5.11.0-7620-generic Os:linux Rootless:false Uptime:4h 46m 37.93s (Approximately 0.17 days)} Registries:{Search:[]} Store:{ConfigFile:/etc/containers/storage.conf ContainerStore:{Number:0} GraphDriverName:overlay GraphOptions:{} GraphRoot:/var/lib/containers/storage GraphStatus:{BackingFilesystem:extfs NativeOverlayDiff:true SupportsDType:true UsingMetacopy:false} ImageStore:{Number:0} RunRoot:/var/run/containers/storage VolumePath:/var/lib/containers/storage/volumes}}
I0719 12:20:03.666444   70767 errors.go:100] postmortem podman info: {Host:{BuildahVersion:1.19.6 CgroupVersion:v1 Conmon:{Package:conmon: /usr/bin/conmon Path:/usr/bin/conmon Version:conmon version 2.0.25, commit: unknown} Distribution:{Distribution:pop Version:21.04} MemFree:16351404032 MemTotal:33532043264 OCIRuntime:{Name:crun Package:crun: /usr/bin/crun Path:/usr/bin/crun Version:crun version 0.17
commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL} SwapFree:4294434816 SwapTotal:4294434816 Arch:amd64 Cpus:16 Eventlogger:journald Hostname:pop-os Kernel:5.11.0-7620-generic Os:linux Rootless:false Uptime:4h 46m 37.93s (Approximately 0.17 days)} Registries:{Search:[]} Store:{ConfigFile:/etc/containers/storage.conf ContainerStore:{Number:0} GraphDriverName:overlay GraphOptions:{} GraphRoot:/var/lib/containers/storage GraphStatus:{BackingFilesystem:extfs NativeOverlayDiff:true SupportsDType:true UsingMetacopy:false} ImageStore:{Number:0} RunRoot:/var/run/containers/storage VolumePath:/var/lib/containers/storage/volumes}}
I0719 12:20:03.666540   70767 network_create.go:249] running [podman network inspect minikube] to gather additional debugging logs...
I0719 12:20:03.666600   70767 cli_runner.go:115] Run: sudo -n podman network inspect minikube
I0719 12:20:03.794477   70767 network_create.go:254] output of [sudo -n podman network inspect minikube]: -- stdout --
[
    {
        "cniVersion": "0.4.0",
        "name": "minikube",
        "plugins": [
            {
                "bridge": "cni-podman0",
                "hairpinMode": true,
                "ipMasq": true,
                "ipam": {
                    "ranges": [
                        [
                            {
                                "gateway": "192.168.49.1",
                                "subnet": "192.168.49.0/24"
                            }
                        ]
                    ],
                    "routes": [
                        {
                            "dst": "0.0.0.0/0"
                        }
                    ],
                    "type": "host-local"
                },
                "isGateway": true,
                "type": "bridge"
            },
            {
                "capabilities": {
                    "portMappings": true
                },
                "type": "portmap"
            },
            {
                "backend": "",
                "type": "firewall"
            }
        ]
    }
]

-- /stdout --
I0719 12:20:03.794734   70767 cli_runner.go:115] Run: sudo -n podman system info --format json
I0719 12:20:04.290430   70767 info.go:281] podman info: {Host:{BuildahVersion:1.19.6 CgroupVersion:v1 Conmon:{Package:conmon: /usr/bin/conmon Path:/usr/bin/conmon Version:conmon version 2.0.25, commit: unknown} Distribution:{Distribution:pop Version:21.04} MemFree:16348712960 MemTotal:33532043264 OCIRuntime:{Name:crun Package:crun: /usr/bin/crun Path:/usr/bin/crun Version:crun version 0.17
commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL} SwapFree:4294434816 SwapTotal:4294434816 Arch:amd64 Cpus:16 Eventlogger:journald Hostname:pop-os Kernel:5.11.0-7620-generic Os:linux Rootless:false Uptime:4h 46m 38.55s (Approximately 0.17 days)} Registries:{Search:[]} Store:{ConfigFile:/etc/containers/storage.conf ContainerStore:{Number:0} GraphDriverName:overlay GraphOptions:{} GraphRoot:/var/lib/containers/storage GraphStatus:{BackingFilesystem:extfs NativeOverlayDiff:true SupportsDType:true UsingMetacopy:false} ImageStore:{Number:0} RunRoot:/var/run/containers/storage VolumePath:/var/lib/containers/storage/volumes}}
I0719 12:20:04.291060   70767 cli_runner.go:115] Run: sudo -n podman container inspect -f {{.NetworkSettings.IPAddress}} minikube
W0719 12:20:04.402655   70767 cli_runner.go:162] sudo -n podman container inspect -f {{.NetworkSettings.IPAddress}} minikube returned with exit code 125
I0719 12:20:04.402858   70767 ssh_runner.go:149] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0719 12:20:04.402988   70767 cli_runner.go:115] Run: sudo -n podman version --format {{.Version}}
I0719 12:20:04.525445   70767 cli_runner.go:115] Run: sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0719 12:20:04.622625   70767 cli_runner.go:162] sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 125
I0719 12:20:04.622843   70767 retry.go:31] will retry after 276.165072ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 12:20:04.899423   70767 cli_runner.go:115] Run: sudo -n podman version --format {{.Version}}
I0719 12:20:05.040888   70767 cli_runner.go:115] Run: sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0719 12:20:05.098430   70767 cli_runner.go:162] sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 125
I0719 12:20:05.098631   70767 retry.go:31] will retry after 540.190908ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 12:20:05.639630   70767 cli_runner.go:115] Run: sudo -n podman version --format {{.Version}}
I0719 12:20:05.786609   70767 cli_runner.go:115] Run: sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0719 12:20:05.900859   70767 cli_runner.go:162] sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 125
I0719 12:20:05.900940   70767 retry.go:31] will retry after 655.06503ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 12:20:06.556283   70767 cli_runner.go:115] Run: sudo -n podman version --format {{.Version}}
I0719 12:20:06.698682   70767 cli_runner.go:115] Run: sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0719 12:20:06.850653   70767 cli_runner.go:162] sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 125
W0719 12:20:06.850878   70767 start.go:257] error running df -h /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube

W0719 12:20:06.850932   70767 start.go:239] error getting percentage of /var that is free: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 12:20:06.850966   70767 fix.go:57] fixHost completed within 3.910603369s
I0719 12:20:06.850989   70767 start.go:80] releasing machines lock for "minikube", held for 3.910648102s
W0719 12:20:06.851043   70767 start.go:505] error starting host: podman inspect ip minikube: sudo -n podman container inspect -f {{.NetworkSettings.IPAddress}} minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
W0719 12:20:06.851305   70767 out.go:235] 🤦  StartHost failed, but will try again: podman inspect ip minikube: sudo -n podman container inspect -f {{.NetworkSettings.IPAddress}} minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube

I0719 12:20:06.851353   70767 start.go:520] Will try again in 5 seconds ...
I0719 12:20:11.852164   70767 start.go:313] acquiring machines lock for minikube: {Name:mkb45fe8b1deef9886cfd2b84df49d68ad57ae09 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0719 12:20:11.852394   70767 start.go:317] acquired machines lock for "minikube" in 151.151µs
I0719 12:20:11.852445   70767 start.go:93] Skipping create...Using existing machine configuration
I0719 12:20:11.852469   70767 fix.go:55] fixHost starting: 
I0719 12:20:11.853115   70767 cli_runner.go:115] Run: sudo -n podman container inspect minikube --format={{.State.Status}}
W0719 12:20:11.950567   70767 cli_runner.go:162] sudo -n podman container inspect minikube --format={{.State.Status}} returned with exit code 125
I0719 12:20:11.950699   70767 fix.go:108] recreateIfNeeded on minikube: state= err=unknown state "minikube": sudo -n podman container inspect minikube --format={{.State.Status}}: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 12:20:11.950737   70767 fix.go:113] machineExists: true. err=unknown state "minikube": sudo -n podman container inspect minikube --format={{.State.Status}}: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
W0719 12:20:11.950761   70767 fix.go:134] unexpected machine state, will restart: unknown state "minikube": sudo -n podman container inspect minikube --format={{.State.Status}}: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 12:20:11.952492   70767 out.go:170] 🔄  Restarting existing podman container for "minikube" ...
I0719 12:20:11.952671   70767 cli_runner.go:115] Run: sudo -n podman start --cgroup-manager cgroupfs minikube
W0719 12:20:12.069053   70767 cli_runner.go:162] sudo -n podman start --cgroup-manager cgroupfs minikube returned with exit code 125
I0719 12:20:12.069122   70767 cli_runner.go:115] Run: sudo -n podman inspect minikube
I0719 12:20:12.132850   70767 errors.go:78] Postmortem inspect ("sudo -n podman inspect minikube"): -- stdout --
[
    {
        "Name": "minikube",
        "Driver": "local",
        "Mountpoint": "/var/lib/containers/storage/volumes/minikube/_data",
        "CreatedAt": "2021-07-19T12:12:45.39965397-05:00",
        "Labels": {
            "created_by.minikube.sigs.k8s.io": "true",
            "name.minikube.sigs.k8s.io": "minikube"
        },
        "Scope": "local",
        "Options": {}
    }
]

-- /stdout --
I0719 12:20:12.132906   70767 cli_runner.go:115] Run: sudo -n podman logs --timestamps minikube
W0719 12:20:12.202675   70767 cli_runner.go:162] sudo -n podman logs --timestamps minikube returned with exit code 125
W0719 12:20:12.202745   70767 errors.go:83] Failed to get postmortem logs. sudo -n podman logs --timestamps minikube :sudo -n podman logs --timestamps minikube: exit status 125
stdout:

stderr:
Error: no container with name or ID minikube found: no such container
I0719 12:20:12.202876   70767 cli_runner.go:115] Run: sudo -n podman system info --format json
I0719 12:20:12.636926   70767 info.go:281] podman info: {Host:{BuildahVersion:1.19.6 CgroupVersion:v1 Conmon:{Package:conmon: /usr/bin/conmon Path:/usr/bin/conmon Version:conmon version 2.0.25, commit: unknown} Distribution:{Distribution:pop Version:21.04} MemFree:16312061952 MemTotal:33532043264 OCIRuntime:{Name:crun Package:crun: /usr/bin/crun Path:/usr/bin/crun Version:crun version 0.17
commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL} SwapFree:4294434816 SwapTotal:4294434816 Arch:amd64 Cpus:16 Eventlogger:journald Hostname:pop-os Kernel:5.11.0-7620-generic Os:linux Rootless:false Uptime:4h 46m 46.91s (Approximately 0.17 days)} Registries:{Search:[]} Store:{ConfigFile:/etc/containers/storage.conf ContainerStore:{Number:0} GraphDriverName:overlay GraphOptions:{} GraphRoot:/var/lib/containers/storage GraphStatus:{BackingFilesystem:extfs NativeOverlayDiff:true SupportsDType:true UsingMetacopy:false} ImageStore:{Number:0} RunRoot:/var/run/containers/storage VolumePath:/var/lib/containers/storage/volumes}}
I0719 12:20:12.636967   70767 errors.go:100] postmortem podman info: {Host:{BuildahVersion:1.19.6 CgroupVersion:v1 Conmon:{Package:conmon: /usr/bin/conmon Path:/usr/bin/conmon Version:conmon version 2.0.25, commit: unknown} Distribution:{Distribution:pop Version:21.04} MemFree:16312061952 MemTotal:33532043264 OCIRuntime:{Name:crun Package:crun: /usr/bin/crun Path:/usr/bin/crun Version:crun version 0.17
commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL} SwapFree:4294434816 SwapTotal:4294434816 Arch:amd64 Cpus:16 Eventlogger:journald Hostname:pop-os Kernel:5.11.0-7620-generic Os:linux Rootless:false Uptime:4h 46m 46.91s (Approximately 0.17 days)} Registries:{Search:[]} Store:{ConfigFile:/etc/containers/storage.conf ContainerStore:{Number:0} GraphDriverName:overlay GraphOptions:{} GraphRoot:/var/lib/containers/storage GraphStatus:{BackingFilesystem:extfs NativeOverlayDiff:true SupportsDType:true UsingMetacopy:false} ImageStore:{Number:0} RunRoot:/var/run/containers/storage VolumePath:/var/lib/containers/storage/volumes}}
I0719 12:20:12.637007   70767 network_create.go:249] running [podman network inspect minikube] to gather additional debugging logs...
I0719 12:20:12.637042   70767 cli_runner.go:115] Run: sudo -n podman network inspect minikube
I0719 12:20:12.696972   70767 network_create.go:254] output of [sudo -n podman network inspect minikube]: -- stdout --
[
    {
        "cniVersion": "0.4.0",
        "name": "minikube",
        "plugins": [
            {
                "bridge": "cni-podman0",
                "hairpinMode": true,
                "ipMasq": true,
                "ipam": {
                    "ranges": [
                        [
                            {
                                "gateway": "192.168.49.1",
                                "subnet": "192.168.49.0/24"
                            }
                        ]
                    ],
                    "routes": [
                        {
                            "dst": "0.0.0.0/0"
                        }
                    ],
                    "type": "host-local"
                },
                "isGateway": true,
                "type": "bridge"
            },
            {
                "capabilities": {
                    "portMappings": true
                },
                "type": "portmap"
            },
            {
                "backend": "",
                "type": "firewall"
            }
        ]
    }
]

-- /stdout --
I0719 12:20:12.697039   70767 cli_runner.go:115] Run: sudo -n podman system info --format json
I0719 12:20:13.105168   70767 info.go:281] podman info: {Host:{BuildahVersion:1.19.6 CgroupVersion:v1 Conmon:{Package:conmon: /usr/bin/conmon Path:/usr/bin/conmon Version:conmon version 2.0.25, commit: unknown} Distribution:{Distribution:pop Version:21.04} MemFree:16308281344 MemTotal:33532043264 OCIRuntime:{Name:crun Package:crun: /usr/bin/crun Path:/usr/bin/crun Version:crun version 0.17
commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL} SwapFree:4294434816 SwapTotal:4294434816 Arch:amd64 Cpus:16 Eventlogger:journald Hostname:pop-os Kernel:5.11.0-7620-generic Os:linux Rootless:false Uptime:4h 46m 47.36s (Approximately 0.17 days)} Registries:{Search:[]} Store:{ConfigFile:/etc/containers/storage.conf ContainerStore:{Number:0} GraphDriverName:overlay GraphOptions:{} GraphRoot:/var/lib/containers/storage GraphStatus:{BackingFilesystem:extfs NativeOverlayDiff:true SupportsDType:true UsingMetacopy:false} ImageStore:{Number:0} RunRoot:/var/run/containers/storage VolumePath:/var/lib/containers/storage/volumes}}
I0719 12:20:13.105439   70767 cli_runner.go:115] Run: sudo -n podman container inspect -f {{.NetworkSettings.IPAddress}} minikube
W0719 12:20:13.162511   70767 cli_runner.go:162] sudo -n podman container inspect -f {{.NetworkSettings.IPAddress}} minikube returned with exit code 125
I0719 12:20:13.162697   70767 ssh_runner.go:149] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0719 12:20:13.162819   70767 cli_runner.go:115] Run: sudo -n podman version --format {{.Version}}
I0719 12:20:13.277047   70767 cli_runner.go:115] Run: sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0719 12:20:13.340902   70767 cli_runner.go:162] sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 125
I0719 12:20:13.340980   70767 retry.go:31] will retry after 234.428547ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 12:20:13.576586   70767 cli_runner.go:115] Run: sudo -n podman version --format {{.Version}}
I0719 12:20:13.721131   70767 cli_runner.go:115] Run: sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0719 12:20:13.801085   70767 cli_runner.go:162] sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 125
I0719 12:20:13.801219   70767 retry.go:31] will retry after 346.739061ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 12:20:14.149092   70767 cli_runner.go:115] Run: sudo -n podman version --format {{.Version}}
I0719 12:20:14.252916   70767 cli_runner.go:115] Run: sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0719 12:20:14.305092   70767 cli_runner.go:162] sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 125
I0719 12:20:14.305149   70767 retry.go:31] will retry after 667.587979ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 12:20:14.973145   70767 cli_runner.go:115] Run: sudo -n podman version --format {{.Version}}
I0719 12:20:15.122670   70767 cli_runner.go:115] Run: sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0719 12:20:15.245504   70767 cli_runner.go:162] sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 125
W0719 12:20:15.245639   70767 start.go:257] error running df -h /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube

W0719 12:20:15.245664   70767 start.go:239] error getting percentage of /var that is free: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 12:20:15.245680   70767 fix.go:57] fixHost completed within 3.393211624s
I0719 12:20:15.245698   70767 start.go:80] releasing machines lock for "minikube", held for 3.39327336s
W0719 12:20:15.245884   70767 out.go:235] 😿  Failed to start podman container. Running "minikube delete" may fix it: podman inspect ip minikube: sudo -n podman container inspect -f {{.NetworkSettings.IPAddress}} minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube

I0719 12:20:15.247686   70767 out.go:170] 
W0719 12:20:15.247812   70767 out.go:235] ❌  Exiting due to GUEST_PROVISION: Failed to start host: podman inspect ip minikube: sudo -n podman container inspect -f  minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube

W0719 12:20:15.247845   70767 out.go:424] no arguments passed for "\n" - returning raw string
W0719 12:20:15.247859   70767 out.go:235] 
W0719 12:20:15.247877   70767 out.go:424] no arguments passed for "😿  If the above advice does not help, please let us know:\n" - returning raw string
W0719 12:20:15.247889   70767 out.go:424] no arguments passed for "👉  https://github.com/kubernetes/minikube/issues/new/choose\n\n" - returning raw string
W0719 12:20:15.247901   70767 out.go:424] no arguments passed for "Please attach the following file to the GitHub issue:\n" - returning raw string
W0719 12:20:15.247964   70767 out.go:424] no arguments passed for "😿  If the above advice does not help, please let us know:\n👉  https://github.com/kubernetes/minikube/issues/new/choose\n\nPlease attach the following file to the GitHub issue:\n- /home/filbot/.minikube/logs/lastStart.txt\n\n" - returning raw string
W0719 12:20:15.249188   70767 out.go:235] ╭────────────────────────────────────────────────────────────────────╮
W0719 12:20:15.249216   70767 out.go:235] │                                                                    │
W0719 12:20:15.249235   70767 out.go:235] │    😿  If the above advice does not help, please let us know:      │
W0719 12:20:15.249250   70767 out.go:235] │    👉  https://github.com/kubernetes/minikube/issues/new/choose    │
W0719 12:20:15.249265   70767 out.go:235] │                                                                    │
W0719 12:20:15.249280   70767 out.go:235] │    Please attach the following file to the GitHub issue:           │
W0719 12:20:15.249295   70767 out.go:235] │    - /home/filbot/.minikube/logs/lastStart.txt                     │
W0719 12:20:15.249310   70767 out.go:235] │                                                                    │
W0719 12:20:15.249324   70767 out.go:235] ╰────────────────────────────────────────────────────────────────────╯
W0719 12:20:15.249341   70767 out.go:235] 



❌  Exiting due to GUEST_STATUS: state: unknown state "minikube": sudo -n podman container inspect minikube --format=: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube


╭───────────────────────────────────────────────────────────────────────────╮
│                                                                           │
│    😿  If the above advice does not help, please let us know:             │
│    👉  https://github.com/kubernetes/minikube/issues/new/choose           │
│                                                                           │
│    Please attach the following file to the GitHub issue:                  │
│    - /tmp/minikube_logs_b1981aaac6eefac9aadc4a5fa2c0b218e1ceffb6_0.log    │
│                                                                           │
╰───────────────────────────────────────────────────────────────────────────╯

Full output of failed command:

~ minikube start --driver=podman --alsologtostderr
I0719 12:20:02.432459   70767 out.go:291] Setting OutFile to fd 1 ...
I0719 12:20:02.432647   70767 out.go:343] isatty.IsTerminal(1) = true
I0719 12:20:02.432661   70767 out.go:304] Setting ErrFile to fd 2...
I0719 12:20:02.432673   70767 out.go:343] isatty.IsTerminal(2) = true
I0719 12:20:02.432822   70767 root.go:316] Updating PATH: /home/filbot/.minikube/bin
I0719 12:20:02.433087   70767 out.go:298] Setting JSON to false
I0719 12:20:02.453649   70767 start.go:108] hostinfo: {"hostname":"pop-os","uptime":17197,"bootTime":1626698005,"procs":523,"os":"linux","platform":"debian","platformFamily":"debian","platformVersion":"bullseye/sid","kernelVersion":"5.11.0-7620-generic","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"host","hostId":"63c5b135-e8c7-4673-bd1c-91f260482952"}
I0719 12:20:02.453751   70767 start.go:118] virtualization: kvm host
I0719 12:20:02.455331   70767 out.go:170] 😄  minikube v1.20.0 on Debian bullseye/sid
😄  minikube v1.20.0 on Debian bullseye/sid
I0719 12:20:02.456083   70767 driver.go:322] Setting default libvirt URI to qemu:///system
I0719 12:20:02.529039   70767 podman.go:121] podman version: 3.0.1
I0719 12:20:02.529992   70767 out.go:170] ✨  Using the podman driver based on existing profile
✨  Using the podman driver based on existing profile
I0719 12:20:02.530018   70767 start.go:276] selected driver: podman
I0719 12:20:02.530027   70767 start.go:718] validating driver "podman" against &{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e Memory:7900 CPUs:2 DiskSize:20000 VMDriver: Driver:podman HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.2 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.2 ControlPlane:true Worker:true}] Addons:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: MultiNodeRequested:false}
I0719 12:20:02.530118   70767 start.go:729] status for podman: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc:}
I0719 12:20:02.530205   70767 cli_runner.go:115] Run: sudo -n podman system info --format json
I0719 12:20:02.934457   70767 info.go:281] podman info: {Host:{BuildahVersion:1.19.6 CgroupVersion:v1 Conmon:{Package:conmon: /usr/bin/conmon Path:/usr/bin/conmon Version:conmon version 2.0.25, commit: unknown} Distribution:{Distribution:pop Version:21.04} MemFree:16357789696 MemTotal:33532043264 OCIRuntime:{Name:crun Package:crun: /usr/bin/crun Path:/usr/bin/crun Version:crun version 0.17
commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL} SwapFree:4294434816 SwapTotal:4294434816 Arch:amd64 Cpus:16 Eventlogger:journald Hostname:pop-os Kernel:5.11.0-7620-generic Os:linux Rootless:false Uptime:4h 46m 37.2s (Approximately 0.17 days)} Registries:{Search:[]} Store:{ConfigFile:/etc/containers/storage.conf ContainerStore:{Number:0} GraphDriverName:overlay GraphOptions:{} GraphRoot:/var/lib/containers/storage GraphStatus:{BackingFilesystem:extfs NativeOverlayDiff:true SupportsDType:true UsingMetacopy:false} ImageStore:{Number:0} RunRoot:/var/run/containers/storage VolumePath:/var/lib/containers/storage/volumes}}
I0719 12:20:02.937161   70767 cni.go:93] Creating CNI manager for ""
I0719 12:20:02.937195   70767 cni.go:167] CNI unnecessary in this configuration, recommending no CNI
I0719 12:20:02.937219   70767 start_flags.go:273] config:
{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e Memory:7900 CPUs:2 DiskSize:20000 VMDriver: Driver:podman HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.2 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.2 ControlPlane:true Worker:true}] Addons:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: MultiNodeRequested:false}
I0719 12:20:02.938708   70767 out.go:170] 👍  Starting control plane node minikube in cluster minikube
👍  Starting control plane node minikube in cluster minikube
I0719 12:20:02.938753   70767 cache.go:111] Beginning downloading kic base image for podman with docker
W0719 12:20:02.938768   70767 out.go:424] no arguments passed for "🚜  Pulling base image ...\n" - returning raw string
W0719 12:20:02.938796   70767 out.go:424] no arguments passed for "🚜  Pulling base image ...\n" - returning raw string
I0719 12:20:02.939559   70767 out.go:170] 🚜  Pulling base image ...
🚜  Pulling base image ...
I0719 12:20:02.939592   70767 preload.go:98] Checking if preload exists for k8s version v1.20.2 and runtime docker
I0719 12:20:02.939633   70767 preload.go:106] Found local preload: /home/filbot/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v10-v1.20.2-docker-overlay2-amd64.tar.lz4
I0719 12:20:02.939637   70767 image.go:116] Checking for kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e in local cache directory
I0719 12:20:02.939647   70767 cache.go:54] Caching tarball of preloaded images
I0719 12:20:02.939680   70767 image.go:119] Found kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e in local cache directory, skipping pull
I0719 12:20:02.939683   70767 preload.go:132] Found /home/filbot/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v10-v1.20.2-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I0719 12:20:02.939693   70767 cache.go:131] kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e exists in cache, skipping pull
I0719 12:20:02.939705   70767 cache.go:57] Finished verifying existence of preloaded tar for  v1.20.2 on docker
I0719 12:20:02.939718   70767 cache.go:168] failed to download kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e, will try fallback image if available: <nil>
I0719 12:20:02.939738   70767 image.go:116] Checking for kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e in local cache directory
I0719 12:20:02.939756   70767 image.go:119] Found kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e in local cache directory, skipping pull
I0719 12:20:02.939767   70767 cache.go:131] kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e exists in cache, skipping pull
I0719 12:20:02.939778   70767 cache.go:168] failed to download kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e, will try fallback image if available: <nil>
I0719 12:20:02.939830   70767 profile.go:148] Saving config to /home/filbot/.minikube/profiles/minikube/config.json ...
E0719 12:20:02.940069   70767 cache.go:189] Error downloading kic artifacts:  failed to download kic base image or any fallback image
I0719 12:20:02.940088   70767 cache.go:194] Successfully downloaded all kic artifacts
I0719 12:20:02.940118   70767 start.go:313] acquiring machines lock for minikube: {Name:mkb45fe8b1deef9886cfd2b84df49d68ad57ae09 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0719 12:20:02.940321   70767 start.go:317] acquired machines lock for "minikube" in 164.066µs
I0719 12:20:02.940349   70767 start.go:93] Skipping create...Using existing machine configuration
I0719 12:20:02.940360   70767 fix.go:55] fixHost starting: 
I0719 12:20:02.940701   70767 cli_runner.go:115] Run: sudo -n podman container inspect minikube --format={{.State.Status}}
W0719 12:20:03.012786   70767 cli_runner.go:162] sudo -n podman container inspect minikube --format={{.State.Status}} returned with exit code 125
I0719 12:20:03.012828   70767 fix.go:108] recreateIfNeeded on minikube: state= err=unknown state "minikube": sudo -n podman container inspect minikube --format={{.State.Status}}: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 12:20:03.012841   70767 fix.go:113] machineExists: true. err=unknown state "minikube": sudo -n podman container inspect minikube --format={{.State.Status}}: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
W0719 12:20:03.012847   70767 fix.go:134] unexpected machine state, will restart: unknown state "minikube": sudo -n podman container inspect minikube --format={{.State.Status}}: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 12:20:03.013780   70767 out.go:170] 🔄  Restarting existing podman container for "minikube" ...
🔄  Restarting existing podman container for "minikube" ...
I0719 12:20:03.013831   70767 cli_runner.go:115] Run: sudo -n podman start --cgroup-manager cgroupfs minikube
W0719 12:20:03.068939   70767 cli_runner.go:162] sudo -n podman start --cgroup-manager cgroupfs minikube returned with exit code 125
I0719 12:20:03.069001   70767 cli_runner.go:115] Run: sudo -n podman inspect minikube
I0719 12:20:03.124913   70767 errors.go:78] Postmortem inspect ("sudo -n podman inspect minikube"): -- stdout --
[
    {
        "Name": "minikube",
        "Driver": "local",
        "Mountpoint": "/var/lib/containers/storage/volumes/minikube/_data",
        "CreatedAt": "2021-07-19T12:12:45.39965397-05:00",
        "Labels": {
            "created_by.minikube.sigs.k8s.io": "true",
            "name.minikube.sigs.k8s.io": "minikube"
        },
        "Scope": "local",
        "Options": {}
    }
]

-- /stdout --
I0719 12:20:03.124962   70767 cli_runner.go:115] Run: sudo -n podman logs --timestamps minikube
W0719 12:20:03.182403   70767 cli_runner.go:162] sudo -n podman logs --timestamps minikube returned with exit code 125
W0719 12:20:03.182470   70767 errors.go:83] Failed to get postmortem logs. sudo -n podman logs --timestamps minikube :sudo -n podman logs --timestamps minikube: exit status 125
stdout:

stderr:
Error: no container with name or ID minikube found: no such container
I0719 12:20:03.182618   70767 cli_runner.go:115] Run: sudo -n podman system info --format json
I0719 12:20:03.666363   70767 info.go:281] podman info: {Host:{BuildahVersion:1.19.6 CgroupVersion:v1 Conmon:{Package:conmon: /usr/bin/conmon Path:/usr/bin/conmon Version:conmon version 2.0.25, commit: unknown} Distribution:{Distribution:pop Version:21.04} MemFree:16351404032 MemTotal:33532043264 OCIRuntime:{Name:crun Package:crun: /usr/bin/crun Path:/usr/bin/crun Version:crun version 0.17
commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL} SwapFree:4294434816 SwapTotal:4294434816 Arch:amd64 Cpus:16 Eventlogger:journald Hostname:pop-os Kernel:5.11.0-7620-generic Os:linux Rootless:false Uptime:4h 46m 37.93s (Approximately 0.17 days)} Registries:{Search:[]} Store:{ConfigFile:/etc/containers/storage.conf ContainerStore:{Number:0} GraphDriverName:overlay GraphOptions:{} GraphRoot:/var/lib/containers/storage GraphStatus:{BackingFilesystem:extfs NativeOverlayDiff:true SupportsDType:true UsingMetacopy:false} ImageStore:{Number:0} RunRoot:/var/run/containers/storage VolumePath:/var/lib/containers/storage/volumes}}
I0719 12:20:03.666444   70767 errors.go:100] postmortem podman info: {Host:{BuildahVersion:1.19.6 CgroupVersion:v1 Conmon:{Package:conmon: /usr/bin/conmon Path:/usr/bin/conmon Version:conmon version 2.0.25, commit: unknown} Distribution:{Distribution:pop Version:21.04} MemFree:16351404032 MemTotal:33532043264 OCIRuntime:{Name:crun Package:crun: /usr/bin/crun Path:/usr/bin/crun Version:crun version 0.17
commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL} SwapFree:4294434816 SwapTotal:4294434816 Arch:amd64 Cpus:16 Eventlogger:journald Hostname:pop-os Kernel:5.11.0-7620-generic Os:linux Rootless:false Uptime:4h 46m 37.93s (Approximately 0.17 days)} Registries:{Search:[]} Store:{ConfigFile:/etc/containers/storage.conf ContainerStore:{Number:0} GraphDriverName:overlay GraphOptions:{} GraphRoot:/var/lib/containers/storage GraphStatus:{BackingFilesystem:extfs NativeOverlayDiff:true SupportsDType:true UsingMetacopy:false} ImageStore:{Number:0} RunRoot:/var/run/containers/storage VolumePath:/var/lib/containers/storage/volumes}}
I0719 12:20:03.666540   70767 network_create.go:249] running [podman network inspect minikube] to gather additional debugging logs...
I0719 12:20:03.666600   70767 cli_runner.go:115] Run: sudo -n podman network inspect minikube
I0719 12:20:03.794477   70767 network_create.go:254] output of [sudo -n podman network inspect minikube]: -- stdout --
[
    {
        "cniVersion": "0.4.0",
        "name": "minikube",
        "plugins": [
            {
                "bridge": "cni-podman0",
                "hairpinMode": true,
                "ipMasq": true,
                "ipam": {
                    "ranges": [
                        [
                            {
                                "gateway": "192.168.49.1",
                                "subnet": "192.168.49.0/24"
                            }
                        ]
                    ],
                    "routes": [
                        {
                            "dst": "0.0.0.0/0"
                        }
                    ],
                    "type": "host-local"
                },
                "isGateway": true,
                "type": "bridge"
            },
            {
                "capabilities": {
                    "portMappings": true
                },
                "type": "portmap"
            },
            {
                "backend": "",
                "type": "firewall"
            }
        ]
    }
]

-- /stdout --
I0719 12:20:03.794734   70767 cli_runner.go:115] Run: sudo -n podman system info --format json
I0719 12:20:04.290430   70767 info.go:281] podman info: {Host:{BuildahVersion:1.19.6 CgroupVersion:v1 Conmon:{Package:conmon: /usr/bin/conmon Path:/usr/bin/conmon Version:conmon version 2.0.25, commit: unknown} Distribution:{Distribution:pop Version:21.04} MemFree:16348712960 MemTotal:33532043264 OCIRuntime:{Name:crun Package:crun: /usr/bin/crun Path:/usr/bin/crun Version:crun version 0.17
commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL} SwapFree:4294434816 SwapTotal:4294434816 Arch:amd64 Cpus:16 Eventlogger:journald Hostname:pop-os Kernel:5.11.0-7620-generic Os:linux Rootless:false Uptime:4h 46m 38.55s (Approximately 0.17 days)} Registries:{Search:[]} Store:{ConfigFile:/etc/containers/storage.conf ContainerStore:{Number:0} GraphDriverName:overlay GraphOptions:{} GraphRoot:/var/lib/containers/storage GraphStatus:{BackingFilesystem:extfs NativeOverlayDiff:true SupportsDType:true UsingMetacopy:false} ImageStore:{Number:0} RunRoot:/var/run/containers/storage VolumePath:/var/lib/containers/storage/volumes}}
I0719 12:20:04.291060   70767 cli_runner.go:115] Run: sudo -n podman container inspect -f {{.NetworkSettings.IPAddress}} minikube
W0719 12:20:04.402655   70767 cli_runner.go:162] sudo -n podman container inspect -f {{.NetworkSettings.IPAddress}} minikube returned with exit code 125
I0719 12:20:04.402858   70767 ssh_runner.go:149] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0719 12:20:04.402988   70767 cli_runner.go:115] Run: sudo -n podman version --format {{.Version}}
I0719 12:20:04.525445   70767 cli_runner.go:115] Run: sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0719 12:20:04.622625   70767 cli_runner.go:162] sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 125
I0719 12:20:04.622843   70767 retry.go:31] will retry after 276.165072ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 12:20:04.899423   70767 cli_runner.go:115] Run: sudo -n podman version --format {{.Version}}
I0719 12:20:05.040888   70767 cli_runner.go:115] Run: sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0719 12:20:05.098430   70767 cli_runner.go:162] sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 125
I0719 12:20:05.098631   70767 retry.go:31] will retry after 540.190908ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 12:20:05.639630   70767 cli_runner.go:115] Run: sudo -n podman version --format {{.Version}}
I0719 12:20:05.786609   70767 cli_runner.go:115] Run: sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0719 12:20:05.900859   70767 cli_runner.go:162] sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 125
I0719 12:20:05.900940   70767 retry.go:31] will retry after 655.06503ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 12:20:06.556283   70767 cli_runner.go:115] Run: sudo -n podman version --format {{.Version}}
I0719 12:20:06.698682   70767 cli_runner.go:115] Run: sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0719 12:20:06.850653   70767 cli_runner.go:162] sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 125
W0719 12:20:06.850878   70767 start.go:257] error running df -h /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube

W0719 12:20:06.850932   70767 start.go:239] error getting percentage of /var that is free: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 12:20:06.850966   70767 fix.go:57] fixHost completed within 3.910603369s
I0719 12:20:06.850989   70767 start.go:80] releasing machines lock for "minikube", held for 3.910648102s
W0719 12:20:06.851043   70767 start.go:505] error starting host: podman inspect ip minikube: sudo -n podman container inspect -f {{.NetworkSettings.IPAddress}} minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
W0719 12:20:06.851305   70767 out.go:235] 🤦  StartHost failed, but will try again: podman inspect ip minikube: sudo -n podman container inspect -f {{.NetworkSettings.IPAddress}} minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube

🤦  StartHost failed, but will try again: podman inspect ip minikube: sudo -n podman container inspect -f {{.NetworkSettings.IPAddress}} minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube

I0719 12:20:06.851353   70767 start.go:520] Will try again in 5 seconds ...
I0719 12:20:11.852164   70767 start.go:313] acquiring machines lock for minikube: {Name:mkb45fe8b1deef9886cfd2b84df49d68ad57ae09 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0719 12:20:11.852394   70767 start.go:317] acquired machines lock for "minikube" in 151.151µs
I0719 12:20:11.852445   70767 start.go:93] Skipping create...Using existing machine configuration
I0719 12:20:11.852469   70767 fix.go:55] fixHost starting: 
I0719 12:20:11.853115   70767 cli_runner.go:115] Run: sudo -n podman container inspect minikube --format={{.State.Status}}
W0719 12:20:11.950567   70767 cli_runner.go:162] sudo -n podman container inspect minikube --format={{.State.Status}} returned with exit code 125
I0719 12:20:11.950699   70767 fix.go:108] recreateIfNeeded on minikube: state= err=unknown state "minikube": sudo -n podman container inspect minikube --format={{.State.Status}}: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 12:20:11.950737   70767 fix.go:113] machineExists: true. err=unknown state "minikube": sudo -n podman container inspect minikube --format={{.State.Status}}: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
W0719 12:20:11.950761   70767 fix.go:134] unexpected machine state, will restart: unknown state "minikube": sudo -n podman container inspect minikube --format={{.State.Status}}: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 12:20:11.952492   70767 out.go:170] 🔄  Restarting existing podman container for "minikube" ...
🔄  Restarting existing podman container for "minikube" ...
I0719 12:20:11.952671   70767 cli_runner.go:115] Run: sudo -n podman start --cgroup-manager cgroupfs minikube
W0719 12:20:12.069053   70767 cli_runner.go:162] sudo -n podman start --cgroup-manager cgroupfs minikube returned with exit code 125
I0719 12:20:12.069122   70767 cli_runner.go:115] Run: sudo -n podman inspect minikube
I0719 12:20:12.132850   70767 errors.go:78] Postmortem inspect ("sudo -n podman inspect minikube"): -- stdout --
[
    {
        "Name": "minikube",
        "Driver": "local",
        "Mountpoint": "/var/lib/containers/storage/volumes/minikube/_data",
        "CreatedAt": "2021-07-19T12:12:45.39965397-05:00",
        "Labels": {
            "created_by.minikube.sigs.k8s.io": "true",
            "name.minikube.sigs.k8s.io": "minikube"
        },
        "Scope": "local",
        "Options": {}
    }
]

-- /stdout --
I0719 12:20:12.132906   70767 cli_runner.go:115] Run: sudo -n podman logs --timestamps minikube
W0719 12:20:12.202675   70767 cli_runner.go:162] sudo -n podman logs --timestamps minikube returned with exit code 125
W0719 12:20:12.202745   70767 errors.go:83] Failed to get postmortem logs. sudo -n podman logs --timestamps minikube :sudo -n podman logs --timestamps minikube: exit status 125
stdout:

stderr:
Error: no container with name or ID minikube found: no such container
I0719 12:20:12.202876   70767 cli_runner.go:115] Run: sudo -n podman system info --format json
I0719 12:20:12.636926   70767 info.go:281] podman info: {Host:{BuildahVersion:1.19.6 CgroupVersion:v1 Conmon:{Package:conmon: /usr/bin/conmon Path:/usr/bin/conmon Version:conmon version 2.0.25, commit: unknown} Distribution:{Distribution:pop Version:21.04} MemFree:16312061952 MemTotal:33532043264 OCIRuntime:{Name:crun Package:crun: /usr/bin/crun Path:/usr/bin/crun Version:crun version 0.17
commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL} SwapFree:4294434816 SwapTotal:4294434816 Arch:amd64 Cpus:16 Eventlogger:journald Hostname:pop-os Kernel:5.11.0-7620-generic Os:linux Rootless:false Uptime:4h 46m 46.91s (Approximately 0.17 days)} Registries:{Search:[]} Store:{ConfigFile:/etc/containers/storage.conf ContainerStore:{Number:0} GraphDriverName:overlay GraphOptions:{} GraphRoot:/var/lib/containers/storage GraphStatus:{BackingFilesystem:extfs NativeOverlayDiff:true SupportsDType:true UsingMetacopy:false} ImageStore:{Number:0} RunRoot:/var/run/containers/storage VolumePath:/var/lib/containers/storage/volumes}}
I0719 12:20:12.636967   70767 errors.go:100] postmortem podman info: {Host:{BuildahVersion:1.19.6 CgroupVersion:v1 Conmon:{Package:conmon: /usr/bin/conmon Path:/usr/bin/conmon Version:conmon version 2.0.25, commit: unknown} Distribution:{Distribution:pop Version:21.04} MemFree:16312061952 MemTotal:33532043264 OCIRuntime:{Name:crun Package:crun: /usr/bin/crun Path:/usr/bin/crun Version:crun version 0.17
commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL} SwapFree:4294434816 SwapTotal:4294434816 Arch:amd64 Cpus:16 Eventlogger:journald Hostname:pop-os Kernel:5.11.0-7620-generic Os:linux Rootless:false Uptime:4h 46m 46.91s (Approximately 0.17 days)} Registries:{Search:[]} Store:{ConfigFile:/etc/containers/storage.conf ContainerStore:{Number:0} GraphDriverName:overlay GraphOptions:{} GraphRoot:/var/lib/containers/storage GraphStatus:{BackingFilesystem:extfs NativeOverlayDiff:true SupportsDType:true UsingMetacopy:false} ImageStore:{Number:0} RunRoot:/var/run/containers/storage VolumePath:/var/lib/containers/storage/volumes}}
I0719 12:20:12.637007   70767 network_create.go:249] running [podman network inspect minikube] to gather additional debugging logs...
I0719 12:20:12.637042   70767 cli_runner.go:115] Run: sudo -n podman network inspect minikube
I0719 12:20:12.696972   70767 network_create.go:254] output of [sudo -n podman network inspect minikube]: -- stdout --
[
    {
        "cniVersion": "0.4.0",
        "name": "minikube",
        "plugins": [
            {
                "bridge": "cni-podman0",
                "hairpinMode": true,
                "ipMasq": true,
                "ipam": {
                    "ranges": [
                        [
                            {
                                "gateway": "192.168.49.1",
                                "subnet": "192.168.49.0/24"
                            }
                        ]
                    ],
                    "routes": [
                        {
                            "dst": "0.0.0.0/0"
                        }
                    ],
                    "type": "host-local"
                },
                "isGateway": true,
                "type": "bridge"
            },
            {
                "capabilities": {
                    "portMappings": true
                },
                "type": "portmap"
            },
            {
                "backend": "",
                "type": "firewall"
            }
        ]
    }
]

-- /stdout --
I0719 12:20:12.697039   70767 cli_runner.go:115] Run: sudo -n podman system info --format json
I0719 12:20:13.105168   70767 info.go:281] podman info: {Host:{BuildahVersion:1.19.6 CgroupVersion:v1 Conmon:{Package:conmon: /usr/bin/conmon Path:/usr/bin/conmon Version:conmon version 2.0.25, commit: unknown} Distribution:{Distribution:pop Version:21.04} MemFree:16308281344 MemTotal:33532043264 OCIRuntime:{Name:crun Package:crun: /usr/bin/crun Path:/usr/bin/crun Version:crun version 0.17
commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL} SwapFree:4294434816 SwapTotal:4294434816 Arch:amd64 Cpus:16 Eventlogger:journald Hostname:pop-os Kernel:5.11.0-7620-generic Os:linux Rootless:false Uptime:4h 46m 47.36s (Approximately 0.17 days)} Registries:{Search:[]} Store:{ConfigFile:/etc/containers/storage.conf ContainerStore:{Number:0} GraphDriverName:overlay GraphOptions:{} GraphRoot:/var/lib/containers/storage GraphStatus:{BackingFilesystem:extfs NativeOverlayDiff:true SupportsDType:true UsingMetacopy:false} ImageStore:{Number:0} RunRoot:/var/run/containers/storage VolumePath:/var/lib/containers/storage/volumes}}
I0719 12:20:13.105439   70767 cli_runner.go:115] Run: sudo -n podman container inspect -f {{.NetworkSettings.IPAddress}} minikube
W0719 12:20:13.162511   70767 cli_runner.go:162] sudo -n podman container inspect -f {{.NetworkSettings.IPAddress}} minikube returned with exit code 125
I0719 12:20:13.162697   70767 ssh_runner.go:149] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0719 12:20:13.162819   70767 cli_runner.go:115] Run: sudo -n podman version --format {{.Version}}
I0719 12:20:13.277047   70767 cli_runner.go:115] Run: sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0719 12:20:13.340902   70767 cli_runner.go:162] sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 125
I0719 12:20:13.340980   70767 retry.go:31] will retry after 234.428547ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 12:20:13.576586   70767 cli_runner.go:115] Run: sudo -n podman version --format {{.Version}}
I0719 12:20:13.721131   70767 cli_runner.go:115] Run: sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0719 12:20:13.801085   70767 cli_runner.go:162] sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 125
I0719 12:20:13.801219   70767 retry.go:31] will retry after 346.739061ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 12:20:14.149092   70767 cli_runner.go:115] Run: sudo -n podman version --format {{.Version}}
I0719 12:20:14.252916   70767 cli_runner.go:115] Run: sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0719 12:20:14.305092   70767 cli_runner.go:162] sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 125
I0719 12:20:14.305149   70767 retry.go:31] will retry after 667.587979ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 12:20:14.973145   70767 cli_runner.go:115] Run: sudo -n podman version --format {{.Version}}
I0719 12:20:15.122670   70767 cli_runner.go:115] Run: sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0719 12:20:15.245504   70767 cli_runner.go:162] sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 125
W0719 12:20:15.245639   70767 start.go:257] error running df -h /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube

W0719 12:20:15.245664   70767 start.go:239] error getting percentage of /var that is free: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 12:20:15.245680   70767 fix.go:57] fixHost completed within 3.393211624s
I0719 12:20:15.245698   70767 start.go:80] releasing machines lock for "minikube", held for 3.39327336s
W0719 12:20:15.245884   70767 out.go:235] 😿  Failed to start podman container. Running "minikube delete" may fix it: podman inspect ip minikube: sudo -n podman container inspect -f {{.NetworkSettings.IPAddress}} minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube

😿  Failed to start podman container. Running "minikube delete" may fix it: podman inspect ip minikube: sudo -n podman container inspect -f {{.NetworkSettings.IPAddress}} minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube

I0719 12:20:15.247686   70767 out.go:170] 

W0719 12:20:15.247812   70767 out.go:235] ❌  Exiting due to GUEST_PROVISION: Failed to start host: podman inspect ip minikube: sudo -n podman container inspect -f  minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube

❌  Exiting due to GUEST_PROVISION: Failed to start host: podman inspect ip minikube: sudo -n podman container inspect -f  minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube

W0719 12:20:15.247845   70767 out.go:424] no arguments passed for "\n" - returning raw string
W0719 12:20:15.247859   70767 out.go:235] 

W0719 12:20:15.247877   70767 out.go:424] no arguments passed for "😿  If the above advice does not help, please let us know:\n" - returning raw string
W0719 12:20:15.247889   70767 out.go:424] no arguments passed for "👉  https://github.com/kubernetes/minikube/issues/new/choose\n\n" - returning raw string
W0719 12:20:15.247901   70767 out.go:424] no arguments passed for "Please attach the following file to the GitHub issue:\n" - returning raw string
W0719 12:20:15.247964   70767 out.go:424] no arguments passed for "😿  If the above advice does not help, please let us know:\n👉  https://github.com/kubernetes/minikube/issues/new/choose\n\nPlease attach the following file to the GitHub issue:\n- /home/filbot/.minikube/logs/lastStart.txt\n\n" - returning raw string
W0719 12:20:15.249188   70767 out.go:235] ╭────────────────────────────────────────────────────────────────────╮
╭────────────────────────────────────────────────────────────────────╮
W0719 12:20:15.249216   70767 out.go:235] │                                                                    │
│                                                                    │
W0719 12:20:15.249235   70767 out.go:235] │    😿  If the above advice does not help, please let us know:      │
│    😿  If the above advice does not help, please let us know:      │
W0719 12:20:15.249250   70767 out.go:235] │    👉  https://github.com/kubernetes/minikube/issues/new/choose    │
│    👉  https://github.com/kubernetes/minikube/issues/new/choose    │
W0719 12:20:15.249265   70767 out.go:235] │                                                                    │
│                                                                    │
W0719 12:20:15.249280   70767 out.go:235] │    Please attach the following file to the GitHub issue:           │
│    Please attach the following file to the GitHub issue:           │
W0719 12:20:15.249295   70767 out.go:235] │    - /home/filbot/.minikube/logs/lastStart.txt                     │
│    - /home/filbot/.minikube/logs/lastStart.txt                     │
W0719 12:20:15.249310   70767 out.go:235] │                                                                    │
│                                                                    │
W0719 12:20:15.249324   70767 out.go:235] ╰────────────────────────────────────────────────────────────────────╯
╰────────────────────────────────────────────────────────────────────╯
W0719 12:20:15.249341   70767 out.go:235] 

I0719 12:20:15.251052   70767 out.go:170] 

Shortened output, without --alsologtostderr and system information including minikube, podman, and Pop!_OS.

~ id
uid=1000(filbot) gid=1000(filbot) groups=1000(filbot),27(sudo),108(kvm),132(libvirt),998(docker)

➜  ~ sudo cat /etc/sudoers.d/filbot
[sudo] password for filbot: 
filbot ALL=(ALL) NOPASSWD: /usr/bin/podman

➜  ~ neofetch                                                          
             /////////////                filbot@pop-os 
         /////////////////////            ------------- 
      ///////*767////////////////         OS: Pop!_OS 21.04 x86_64 
    //////7676767676*//////////////       Host: Oryx Pro oryp6 
   /////76767//7676767//////////////      Kernel: 5.11.0-7620-generic 
  /////767676///*76767///////////////     Uptime: 4 hours, 42 mins 
 ///////767676///76767.///7676*///////    Packages: 3772 (dpkg), 60 (flatpak) 
/////////767676//76767///767676////////   Shell: zsh 5.8 
//////////76767676767////76767/////////   Resolution: 1920x1080, 1680x1050 
///////////76767676//////7676//////////   DE: GNOME 3.38.4 
////////////,7676,///////767///////////   WM: Mutter 
/////////////*7676///////76////////////   WM Theme: Pop 
///////////////7676////////////////////   Theme: Pop-dark [GTK2/3] 
 ///////////////7676///767////////////    Icons: breeze-dark [GTK2/3] 
  //////////////////////'////////////     Terminal: gnome-terminal 
   //////.7676767676767676767,//////      CPU: Intel i7-10875H (16) @ 5.100GHz 
    /////767676767676767676767/////       GPU: NVIDIA GeForce RTX 2060 Mobile 
      ///////////////////////////         GPU: Intel CometLake-H GT2 [UHD Graphics] 
         /////////////////////            Memory: 8799MiB / 31978MiB 
             /////////////

➜  ~ podman --version
podman version 3.0.1

➜  ~ minikube version
minikube version: v1.20.0
commit: c61663e942ec43b20e8e70839dcca52e44cd85ae

➜  ~ minikube start --driver=podman                                    
😄  minikube v1.20.0 on Debian bullseye/sid
✨  Using the podman driver based on existing profile
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
E0719 12:14:57.371904   67682 cache.go:189] Error downloading kic artifacts:  failed to download kic base image or any fallback image
🔄  Restarting existing podman container for "minikube" ...
🤦  StartHost failed, but will try again: podman inspect ip minikube: sudo -n podman container inspect -f {{.NetworkSettings.IPAddress}} minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube

🔄  Restarting existing podman container for "minikube" ...
😿  Failed to start podman container. Running "minikube delete" may fix it: podman inspect ip minikube: sudo -n podman container inspect -f {{.NetworkSettings.IPAddress}} minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube


❌  Exiting due to GUEST_PROVISION: Failed to start host: podman inspect ip minikube: sudo -n podman container inspect -f  minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube


╭────────────────────────────────────────────────────────────────────╮
│                                                                    │
│    😿  If the above advice does not help, please let us know:      │
│    👉  https://github.com/kubernetes/minikube/issues/new/choose    │
│                                                                    │
│    Please attach the following file to the GitHub issue:           │
│    - /home/filbot/.minikube/logs/lastStart.txt                     │
│                                                                    │
╰────────────────────────────────────────────────────────────────────╯
@afbjorklund
Copy link
Collaborator

afbjorklund commented Jul 19, 2021

Will need the logs from the first run, not from the second (since it will just say that it can't find the container, etc)

You should not need to delete the containers/network/volumes, just the minikube profile (which will do it for you)

@afbjorklund afbjorklund added co/podman-driver podman driver issues kind/support Categorizes issue or PR as a support question. labels Jul 19, 2021
@FilBot3
Copy link
Author

FilBot3 commented Jul 19, 2021

after doing the initial minikube stop && minikube delete the containers were left in my Podman images. That's why I pruned them. I do not have logs from the first run, the terminal rolled over before I thought of saving it.

@afbjorklund
Copy link
Collaborator

There should be some log files in /tmp, otherwise maybe you could do another start ?

@FilBot3
Copy link
Author

FilBot3 commented Jul 19, 2021

This is what I see in my /tmp

~ ll /tmp
[...]
-rw-rw-r-- 1 filbot filbot  183 Jul 19 11:36 minikube_addon_a916fd27217bb88fab01e1bfb1ac8674d545b941_0.log
-rw-rw-r-- 1 filbot filbot  98K Jul 19 12:09 minikube_addons_7ed8c84e4ce93cbcc83e463eeed18a2c4adfac94_0.log
-rw-rw-r-- 1 filbot filbot  747 Jul 19 11:36 minikube_addons_89ff1ee9c8cbbf522b4b6f1ee582e05f15571c5e_0.log
-rw-rw-r-- 1 filbot filbot  547 Jul 19 12:02 minikube_addons_b1ea12cdc153f60a2e7eb241fe4ae51495ef2953_0.log
-rw-rw-r-- 1 filbot filbot  547 Jul 19 11:36 minikube_addons_cce829a0f7696ef46bd1c21e186832e4edccdfcf_0.log
-rw-rw-r-- 1 filbot filbot 208K Jul 19 12:33 minikube_addons_f908c94c424316e407fb8fcb48af06d63b073a1d_0.log
-rw-rw-r-- 1 filbot filbot  16K Jul 19 12:30 minikube_delete_0270979c635fa6787b7a65d8f242ea269e079c43_0.log
-rw-rw-r-- 1 filbot filbot  96K Jul 19 12:28 minikube_logs_b1981aaac6eefac9aadc4a5fa2c0b218e1ceffb6_0.log
-rw-rw-r-- 1 filbot filbot  366 Jul 19 12:13 minikube_status_9ccdd6f7f52a5e52cbca336c59b6ea55aba8c2da_0.log
-rw-rw-r-- 1 filbot filbot  36K Jul 19 12:30 minikube_stop_3f14e2ff40adb89347d5b858038244f6a04f46a0_0.log
-rw-rw-r-- 1 filbot filbot 1.4K Jul 19 12:17 minikube_version_1382b4e92568a7c769568fc59968f434ceb987f4_0.log
[...]

I'll do another start this time. I'll do one with the --alsologtostderr.

@FilBot3
Copy link
Author

FilBot3 commented Jul 19, 2021

Here is: minikube start --driver=podman --alsologtostderr.

~ minikube start --driver=podman --alsologtostderr                                                       
I0719 13:36:33.110956   79076 out.go:291] Setting OutFile to fd 1 ...
I0719 13:36:33.111041   79076 out.go:343] isatty.IsTerminal(1) = true
I0719 13:36:33.111051   79076 out.go:304] Setting ErrFile to fd 2...
I0719 13:36:33.111058   79076 out.go:343] isatty.IsTerminal(2) = true
I0719 13:36:33.111168   79076 root.go:316] Updating PATH: /home/filbot/.minikube/bin
I0719 13:36:33.111423   79076 out.go:298] Setting JSON to false
I0719 13:36:33.129564   79076 start.go:108] hostinfo: {"hostname":"pop-os","uptime":21788,"bootTime":1626698005,"procs":522,"os":"linux","platform":"debian","platformFamily":"debian","platformVersion":"bullseye/sid","kernelVersion":"5.11.0-7620-generic","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"host","hostId":"63c5b135-e8c7-4673-bd1c-91f260482952"}
I0719 13:36:33.129647   79076 start.go:118] virtualization: kvm host
I0719 13:36:33.141886   79076 out.go:170] 😄  minikube v1.20.0 on Debian bullseye/sid
😄  minikube v1.20.0 on Debian bullseye/sid
I0719 13:36:33.142089   79076 driver.go:322] Setting default libvirt URI to qemu:///system
I0719 13:36:33.213146   79076 podman.go:121] podman version: 3.0.1
I0719 13:36:33.214347   79076 out.go:170] ✨  Using the podman driver based on user configuration
✨  Using the podman driver based on user configuration
I0719 13:36:33.214374   79076 start.go:276] selected driver: podman
I0719 13:36:33.214386   79076 start.go:718] validating driver "podman" against <nil>
I0719 13:36:33.214410   79076 start.go:729] status for podman: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc:}
I0719 13:36:33.214530   79076 cli_runner.go:115] Run: sudo -n podman system info --format json
I0719 13:36:33.629152   79076 info.go:281] podman info: {Host:{BuildahVersion:1.19.6 CgroupVersion:v1 Conmon:{Package:conmon: /usr/bin/conmon Path:/usr/bin/conmon Version:conmon version 2.0.25, commit: unknown} Distribution:{Distribution:pop Version:21.04} MemFree:15354544128 MemTotal:33532043264 OCIRuntime:{Name:crun Package:crun: /usr/bin/crun Path:/usr/bin/crun Version:crun version 0.17
commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL} SwapFree:4294434816 SwapTotal:4294434816 Arch:amd64 Cpus:16 Eventlogger:journald Hostname:pop-os Kernel:5.11.0-7620-generic Os:linux Rootless:false Uptime:6h 3m 7.89s (Approximately 0.25 days)} Registries:{Search:[]} Store:{ConfigFile:/etc/containers/storage.conf ContainerStore:{Number:0} GraphDriverName:overlay GraphOptions:{} GraphRoot:/var/lib/containers/storage GraphStatus:{BackingFilesystem:extfs NativeOverlayDiff:true SupportsDType:true UsingMetacopy:false} ImageStore:{Number:0} RunRoot:/var/run/containers/storage VolumePath:/var/lib/containers/storage/volumes}}
I0719 13:36:33.629414   79076 start_flags.go:259] no existing cluster config was found, will generate one from the flags 
I0719 13:36:33.655375   79076 start_flags.go:314] Using suggested 7900MB memory alloc based on sys=31978MB, container=31978MB
I0719 13:36:33.655853   79076 start_flags.go:715] Wait components to verify : map[apiserver:true system_pods:true]
I0719 13:36:33.655925   79076 cni.go:93] Creating CNI manager for ""
I0719 13:36:33.655979   79076 cni.go:167] CNI unnecessary in this configuration, recommending no CNI
I0719 13:36:33.656009   79076 start_flags.go:273] config:
{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e Memory:7900 CPUs:2 DiskSize:20000 VMDriver: Driver:podman HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.2 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: MultiNodeRequested:false}
I0719 13:36:33.658819   79076 out.go:170] 👍  Starting control plane node minikube in cluster minikube
👍  Starting control plane node minikube in cluster minikube
I0719 13:36:33.658921   79076 cache.go:111] Beginning downloading kic base image for podman with docker
W0719 13:36:33.658962   79076 out.go:424] no arguments passed for "🚜  Pulling base image ...\n" - returning raw string
W0719 13:36:33.659018   79076 out.go:424] no arguments passed for "🚜  Pulling base image ...\n" - returning raw string
I0719 13:36:33.660242   79076 out.go:170] 🚜  Pulling base image ...
🚜  Pulling base image ...
I0719 13:36:33.660305   79076 preload.go:98] Checking if preload exists for k8s version v1.20.2 and runtime docker
I0719 13:36:33.660407   79076 preload.go:106] Found local preload: /home/filbot/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v10-v1.20.2-docker-overlay2-amd64.tar.lz4
I0719 13:36:33.660442   79076 cache.go:54] Caching tarball of preloaded images
I0719 13:36:33.660446   79076 image.go:116] Checking for gcr.io/k8s-minikube/kicbase:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e in local cache directory
I0719 13:36:33.660501   79076 preload.go:132] Found /home/filbot/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v10-v1.20.2-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I0719 13:36:33.660534   79076 image.go:119] Found gcr.io/k8s-minikube/kicbase:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e in local cache directory, skipping pull
I0719 13:36:33.660541   79076 cache.go:57] Finished verifying existence of preloaded tar for  v1.20.2 on docker
I0719 13:36:33.660566   79076 cache.go:131] gcr.io/k8s-minikube/kicbase:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e exists in cache, skipping pull
I0719 13:36:33.660602   79076 cache.go:168] failed to download gcr.io/k8s-minikube/kicbase:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e, will try fallback image if available: <nil>
I0719 13:36:33.660628   79076 image.go:116] Checking for kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e in local cache directory
I0719 13:36:33.660656   79076 image.go:119] Found kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e in local cache directory, skipping pull
I0719 13:36:33.660670   79076 cache.go:131] kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e exists in cache, skipping pull
I0719 13:36:33.660692   79076 cache.go:168] failed to download kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e, will try fallback image if available: <nil>
W0719 13:36:33.660721   79076 out.go:424] no arguments passed for "❗  minikube was unable to download gcr.io/k8s-minikube/kicbase:v0.0.22, but successfully downloaded kicbase/stable:v0.0.22 as a fallback image\n" - returning raw string
W0719 13:36:33.660786   79076 out.go:235] ❗  minikube was unable to download gcr.io/k8s-minikube/kicbase:v0.0.22, but successfully downloaded kicbase/stable:v0.0.22 as a fallback image
❗  minikube was unable to download gcr.io/k8s-minikube/kicbase:v0.0.22, but successfully downloaded kicbase/stable:v0.0.22 as a fallback image
I0719 13:36:33.661409   79076 profile.go:148] Saving config to /home/filbot/.minikube/profiles/minikube/config.json ...
I0719 13:36:33.661495   79076 lock.go:36] WriteFile acquiring /home/filbot/.minikube/profiles/minikube/config.json: {Name:mkde552573cc4fe111badcbccdf8dc701af1839b Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
E0719 13:36:33.661908   79076 cache.go:189] Error downloading kic artifacts:  failed to download kic base image or any fallback image
I0719 13:36:33.661934   79076 cache.go:194] Successfully downloaded all kic artifacts
I0719 13:36:33.661970   79076 start.go:313] acquiring machines lock for minikube: {Name:mkb45fe8b1deef9886cfd2b84df49d68ad57ae09 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0719 13:36:33.662104   79076 start.go:317] acquired machines lock for "minikube" in 95.01µs
I0719 13:36:33.662139   79076 start.go:89] Provisioning new machine with config: &{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:kicbase/stable:v0.0.22@sha256:7cc3a3cb6e51c628d8ede157ad9e1f797e8d22a1b3cedc12d3f1999cb52f962e Memory:7900 CPUs:2 DiskSize:20000 VMDriver: Driver:podman HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.2 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.2 ControlPlane:true Worker:true}] Addons:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: MultiNodeRequested:false} &{Name: IP: Port:8443 KubernetesVersion:v1.20.2 ControlPlane:true Worker:true}
I0719 13:36:33.662317   79076 start.go:126] createHost starting for "" (driver="podman")
I0719 13:36:33.663839   79076 out.go:197] 🔥  Creating podman container (CPUs=2, Memory=7900MB) ...
🔥  Creating podman container (CPUs=2, Memory=7900MB) ...| I0719 13:36:33.664393   79076 start.go:160] libmachine.API.Create for "minikube" (driver="podman")
I0719 13:36:33.664468   79076 client.go:168] LocalClient.Create starting
I0719 13:36:33.664603   79076 main.go:128] libmachine: Reading certificate data from /home/filbot/.minikube/certs/ca.pem
I0719 13:36:33.664693   79076 main.go:128] libmachine: Decoding PEM data...
I0719 13:36:33.664769   79076 main.go:128] libmachine: Parsing certificate...
I0719 13:36:33.665148   79076 main.go:128] libmachine: Reading certificate data from /home/filbot/.minikube/certs/cert.pem
I0719 13:36:33.665242   79076 main.go:128] libmachine: Decoding PEM data...
I0719 13:36:33.665309   79076 main.go:128] libmachine: Parsing certificate...
I0719 13:36:33.666243   79076 cli_runner.go:115] Run: sudo -n podman network inspect minikube --format "{{range .plugins}}{{if eq .type "bridge"}}{{(index (index .ipam.ranges 0) 0).subnet}},{{(index (index .ipam.ranges 0) 0).gateway}}{{end}}{{end}}"
/ I0719 13:36:33.805538   79076 network_create.go:61] Found existing network {name:minikube subnet:0xc001124c00 gateway:[0 0 0 0 0 0 0 0 0 0 255 255 192 168 49 1] mtu:0}
I0719 13:36:33.805582   79076 kic.go:106] calculated static IP "192.168.49.2" for the "minikube" container
I0719 13:36:33.805681   79076 cli_runner.go:115] Run: sudo -n podman ps -a --format {{.Names}}
- I0719 13:36:33.913299   79076 cli_runner.go:115] Run: sudo -n podman volume create minikube --label name.minikube.sigs.k8s.io=minikube --label created_by.minikube.sigs.k8s.io=true
\ I0719 13:36:34.001267   79076 oci.go:102] Successfully created a podman volume minikube
I0719 13:36:34.001356   79076 cli_runner.go:115] Run: sudo -n podman run --rm --name minikube-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --entrypoint /usr/bin/test -v minikube:/var kicbase/stable:v0.0.22 -d /var/lib
| W0719 13:36:34.081049   79076 cli_runner.go:162] sudo -n podman run --rm --name minikube-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --entrypoint /usr/bin/test -v minikube:/var kicbase/stable:v0.0.22 -d /var/lib returned with exit code 125
I0719 13:36:34.081127   79076 client.go:171] LocalClient.Create took 416.639352ms
| I0719 13:36:36.081536   79076 ssh_runner.go:149] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0719 13:36:36.081947   79076 cli_runner.go:115] Run: sudo -n podman version --format {{.Version}}
/ I0719 13:36:36.185145   79076 cli_runner.go:115] Run: sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0719 13:36:36.258744   79076 cli_runner.go:162] sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 125
I0719 13:36:36.258936   79076 retry.go:31] will retry after 276.165072ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
| I0719 13:36:36.535787   79076 cli_runner.go:115] Run: sudo -n podman version --format {{.Version}}
/ I0719 13:36:36.629210   79076 cli_runner.go:115] Run: sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
- W0719 13:36:36.696800   79076 cli_runner.go:162] sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 125
I0719 13:36:36.696879   79076 retry.go:31] will retry after 540.190908ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
\ I0719 13:36:37.237702   79076 cli_runner.go:115] Run: sudo -n podman version --format {{.Version}}
| I0719 13:36:37.378826   79076 cli_runner.go:115] Run: sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
- W0719 13:36:37.513497   79076 cli_runner.go:162] sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 125
I0719 13:36:37.513616   79076 retry.go:31] will retry after 655.06503ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
| I0719 13:36:38.169221   79076 cli_runner.go:115] Run: sudo -n podman version --format {{.Version}}
- I0719 13:36:38.321149   79076 cli_runner.go:115] Run: sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
\ W0719 13:36:38.394492   79076 cli_runner.go:162] sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 125
W0719 13:36:38.394704   79076 start.go:257] error running df -h /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube

W0719 13:36:38.394750   79076 start.go:239] error getting percentage of /var that is free: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 13:36:38.394787   79076 start.go:129] duration metric: createHost completed in 4.732452925s
I0719 13:36:38.394813   79076 start.go:80] releasing machines lock for "minikube", held for 4.732684259s
W0719 13:36:38.394873   79076 start.go:505] error starting host: creating host: create: creating: setting up container node: preparing volume for minikube container: sudo -n podman run --rm --name minikube-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --entrypoint /usr/bin/test -v minikube:/var kicbase/stable:v0.0.22 -d /var/lib: exit status 125
stdout:

stderr:
Error: error getting default registries to try: short-name "kicbase/stable:v0.0.22" did not resolve to an alias and no unqualified-search registries are defined in "/etc/containers/registries.conf"
I0719 13:36:38.396249   79076 cli_runner.go:115] Run: sudo -n podman container inspect minikube --format={{.State.Status}}
| W0719 13:36:38.541010   79076 cli_runner.go:162] sudo -n podman container inspect minikube --format={{.State.Status}} returned with exit code 125
I0719 13:36:38.541049   79076 delete.go:82] Unable to get host status for minikube, assuming it has already been deleted: state: unknown state "minikube": sudo -n podman container inspect minikube --format={{.State.Status}}: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
W0719 13:36:38.541144   79076 out.go:235] 🤦  StartHost failed, but will try again: creating host: create: creating: setting up container node: preparing volume for minikube container: sudo -n podman run --rm --name minikube-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --entrypoint /usr/bin/test -v minikube:/var kicbase/stable:v0.0.22 -d /var/lib: exit status 125
stdout:

stderr:
Error: error getting default registries to try: short-name "kicbase/stable:v0.0.22" did not resolve to an alias and no unqualified-search registries are defined in "/etc/containers/registries.conf"


🤦  StartHost failed, but will try again: creating host: create: creating: setting up container node: preparing volume for minikube container: sudo -n podman run --rm --name minikube-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --entrypoint /usr/bin/test -v minikube:/var kicbase/stable:v0.0.22 -d /var/lib: exit status 125
stdout:

stderr:
Error: error getting default registries to try: short-name "kicbase/stable:v0.0.22" did not resolve to an alias and no unqualified-search registries are defined in "/etc/containers/registries.conf"

I0719 13:36:38.541164   79076 start.go:520] Will try again in 5 seconds ...
I0719 13:36:43.542373   79076 start.go:313] acquiring machines lock for minikube: {Name:mkb45fe8b1deef9886cfd2b84df49d68ad57ae09 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0719 13:36:43.542615   79076 start.go:317] acquired machines lock for "minikube" in 163.797µs
I0719 13:36:43.542670   79076 start.go:93] Skipping create...Using existing machine configuration
I0719 13:36:43.542700   79076 fix.go:55] fixHost starting: 
I0719 13:36:43.543479   79076 cli_runner.go:115] Run: sudo -n podman container inspect minikube --format={{.State.Status}}
W0719 13:36:43.684989   79076 cli_runner.go:162] sudo -n podman container inspect minikube --format={{.State.Status}} returned with exit code 125
I0719 13:36:43.685031   79076 fix.go:108] recreateIfNeeded on minikube: state= err=unknown state "minikube": sudo -n podman container inspect minikube --format={{.State.Status}}: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 13:36:43.685050   79076 fix.go:113] machineExists: true. err=unknown state "minikube": sudo -n podman container inspect minikube --format={{.State.Status}}: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
W0719 13:36:43.685062   79076 fix.go:134] unexpected machine state, will restart: unknown state "minikube": sudo -n podman container inspect minikube --format={{.State.Status}}: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 13:36:43.686290   79076 out.go:170] 🔄  Restarting existing podman container for "minikube" ...
🔄  Restarting existing podman container for "minikube" ...
I0719 13:36:43.686347   79076 cli_runner.go:115] Run: sudo -n podman start --cgroup-manager cgroupfs minikube
W0719 13:36:43.758570   79076 cli_runner.go:162] sudo -n podman start --cgroup-manager cgroupfs minikube returned with exit code 125
I0719 13:36:43.758746   79076 cli_runner.go:115] Run: sudo -n podman inspect minikube
I0719 13:36:43.918490   79076 errors.go:78] Postmortem inspect ("sudo -n podman inspect minikube"): -- stdout --
[
    {
        "Name": "minikube",
        "Driver": "local",
        "Mountpoint": "/var/lib/containers/storage/volumes/minikube/_data",
        "CreatedAt": "2021-07-19T13:36:33.987291638-05:00",
        "Labels": {
            "created_by.minikube.sigs.k8s.io": "true",
            "name.minikube.sigs.k8s.io": "minikube"
        },
        "Scope": "local",
        "Options": {}
    }
]

-- /stdout --
I0719 13:36:43.918673   79076 cli_runner.go:115] Run: sudo -n podman logs --timestamps minikube
W0719 13:36:44.070581   79076 cli_runner.go:162] sudo -n podman logs --timestamps minikube returned with exit code 125
W0719 13:36:44.070686   79076 errors.go:83] Failed to get postmortem logs. sudo -n podman logs --timestamps minikube :sudo -n podman logs --timestamps minikube: exit status 125
stdout:

stderr:
Error: no container with name or ID minikube found: no such container
I0719 13:36:44.070843   79076 cli_runner.go:115] Run: sudo -n podman system info --format json
I0719 13:36:44.501119   79076 info.go:281] podman info: {Host:{BuildahVersion:1.19.6 CgroupVersion:v1 Conmon:{Package:conmon: /usr/bin/conmon Path:/usr/bin/conmon Version:conmon version 2.0.25, commit: unknown} Distribution:{Distribution:pop Version:21.04} MemFree:15354499072 MemTotal:33532043264 OCIRuntime:{Name:crun Package:crun: /usr/bin/crun Path:/usr/bin/crun Version:crun version 0.17
commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL} SwapFree:4294434816 SwapTotal:4294434816 Arch:amd64 Cpus:16 Eventlogger:journald Hostname:pop-os Kernel:5.11.0-7620-generic Os:linux Rootless:false Uptime:6h 3m 18.77s (Approximately 0.25 days)} Registries:{Search:[]} Store:{ConfigFile:/etc/containers/storage.conf ContainerStore:{Number:0} GraphDriverName:overlay GraphOptions:{} GraphRoot:/var/lib/containers/storage GraphStatus:{BackingFilesystem:extfs NativeOverlayDiff:true SupportsDType:true UsingMetacopy:false} ImageStore:{Number:0} RunRoot:/var/run/containers/storage VolumePath:/var/lib/containers/storage/volumes}}
I0719 13:36:44.501155   79076 errors.go:100] postmortem podman info: {Host:{BuildahVersion:1.19.6 CgroupVersion:v1 Conmon:{Package:conmon: /usr/bin/conmon Path:/usr/bin/conmon Version:conmon version 2.0.25, commit: unknown} Distribution:{Distribution:pop Version:21.04} MemFree:15354499072 MemTotal:33532043264 OCIRuntime:{Name:crun Package:crun: /usr/bin/crun Path:/usr/bin/crun Version:crun version 0.17
commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL} SwapFree:4294434816 SwapTotal:4294434816 Arch:amd64 Cpus:16 Eventlogger:journald Hostname:pop-os Kernel:5.11.0-7620-generic Os:linux Rootless:false Uptime:6h 3m 18.77s (Approximately 0.25 days)} Registries:{Search:[]} Store:{ConfigFile:/etc/containers/storage.conf ContainerStore:{Number:0} GraphDriverName:overlay GraphOptions:{} GraphRoot:/var/lib/containers/storage GraphStatus:{BackingFilesystem:extfs NativeOverlayDiff:true SupportsDType:true UsingMetacopy:false} ImageStore:{Number:0} RunRoot:/var/run/containers/storage VolumePath:/var/lib/containers/storage/volumes}}
I0719 13:36:44.501190   79076 network_create.go:249] running [podman network inspect minikube] to gather additional debugging logs...
I0719 13:36:44.501215   79076 cli_runner.go:115] Run: sudo -n podman network inspect minikube
I0719 13:36:44.556889   79076 network_create.go:254] output of [sudo -n podman network inspect minikube]: -- stdout --
[
    {
        "cniVersion": "0.4.0",
        "name": "minikube",
        "plugins": [
            {
                "bridge": "cni-podman0",
                "hairpinMode": true,
                "ipMasq": true,
                "ipam": {
                    "ranges": [
                        [
                            {
                                "gateway": "192.168.49.1",
                                "subnet": "192.168.49.0/24"
                            }
                        ]
                    ],
                    "routes": [
                        {
                            "dst": "0.0.0.0/0"
                        }
                    ],
                    "type": "host-local"
                },
                "isGateway": true,
                "type": "bridge"
            },
            {
                "capabilities": {
                    "portMappings": true
                },
                "type": "portmap"
            },
            {
                "backend": "",
                "type": "firewall"
            }
        ]
    }
]

-- /stdout --
I0719 13:36:44.556955   79076 cli_runner.go:115] Run: sudo -n podman system info --format json
I0719 13:36:44.953073   79076 info.go:281] podman info: {Host:{BuildahVersion:1.19.6 CgroupVersion:v1 Conmon:{Package:conmon: /usr/bin/conmon Path:/usr/bin/conmon Version:conmon version 2.0.25, commit: unknown} Distribution:{Distribution:pop Version:21.04} MemFree:15356416000 MemTotal:33532043264 OCIRuntime:{Name:crun Package:crun: /usr/bin/crun Path:/usr/bin/crun Version:crun version 0.17
commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL} SwapFree:4294434816 SwapTotal:4294434816 Arch:amd64 Cpus:16 Eventlogger:journald Hostname:pop-os Kernel:5.11.0-7620-generic Os:linux Rootless:false Uptime:6h 3m 19.21s (Approximately 0.25 days)} Registries:{Search:[]} Store:{ConfigFile:/etc/containers/storage.conf ContainerStore:{Number:0} GraphDriverName:overlay GraphOptions:{} GraphRoot:/var/lib/containers/storage GraphStatus:{BackingFilesystem:extfs NativeOverlayDiff:true SupportsDType:true UsingMetacopy:false} ImageStore:{Number:0} RunRoot:/var/run/containers/storage VolumePath:/var/lib/containers/storage/volumes}}
I0719 13:36:44.953329   79076 cli_runner.go:115] Run: sudo -n podman container inspect -f {{.NetworkSettings.IPAddress}} minikube
W0719 13:36:45.012891   79076 cli_runner.go:162] sudo -n podman container inspect -f {{.NetworkSettings.IPAddress}} minikube returned with exit code 125
I0719 13:36:45.012951   79076 ssh_runner.go:149] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0719 13:36:45.012996   79076 cli_runner.go:115] Run: sudo -n podman version --format {{.Version}}
I0719 13:36:45.068948   79076 cli_runner.go:115] Run: sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0719 13:36:45.125034   79076 cli_runner.go:162] sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 125
I0719 13:36:45.125094   79076 retry.go:31] will retry after 234.428547ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 13:36:45.360485   79076 cli_runner.go:115] Run: sudo -n podman version --format {{.Version}}
I0719 13:36:45.456919   79076 cli_runner.go:115] Run: sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0719 13:36:45.512966   79076 cli_runner.go:162] sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 125
I0719 13:36:45.513030   79076 retry.go:31] will retry after 346.739061ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 13:36:45.860766   79076 cli_runner.go:115] Run: sudo -n podman version --format {{.Version}}
I0719 13:36:45.986612   79076 cli_runner.go:115] Run: sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0719 13:36:46.125635   79076 cli_runner.go:162] sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 125
I0719 13:36:46.125750   79076 retry.go:31] will retry after 667.587979ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 13:36:46.793722   79076 cli_runner.go:115] Run: sudo -n podman version --format {{.Version}}
I0719 13:36:46.944853   79076 cli_runner.go:115] Run: sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0719 13:36:46.998604   79076 cli_runner.go:162] sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 125
I0719 13:36:46.998807   79076 retry.go:31] will retry after 477.256235ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 13:36:47.476680   79076 cli_runner.go:115] Run: sudo -n podman version --format {{.Version}}
I0719 13:36:47.552817   79076 cli_runner.go:115] Run: sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0719 13:36:47.608928   79076 cli_runner.go:162] sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 125
W0719 13:36:47.609024   79076 start.go:257] error running df -h /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube

W0719 13:36:47.609047   79076 start.go:239] error getting percentage of /var that is free: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": sudo -n podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube
I0719 13:36:47.609057   79076 fix.go:57] fixHost completed within 4.066364651s
I0719 13:36:47.609064   79076 start.go:80] releasing machines lock for "minikube", held for 4.06641774s
W0719 13:36:47.609186   79076 out.go:235] 😿  Failed to start podman container. Running "minikube delete" may fix it: podman inspect ip minikube: sudo -n podman container inspect -f {{.NetworkSettings.IPAddress}} minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube

😿  Failed to start podman container. Running "minikube delete" may fix it: podman inspect ip minikube: sudo -n podman container inspect -f {{.NetworkSettings.IPAddress}} minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube

I0719 13:36:47.610820   79076 out.go:170] 

W0719 13:36:47.610908   79076 out.go:235] ❌  Exiting due to GUEST_PROVISION: Failed to start host: podman inspect ip minikube: sudo -n podman container inspect -f  minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube

❌  Exiting due to GUEST_PROVISION: Failed to start host: podman inspect ip minikube: sudo -n podman container inspect -f  minikube: exit status 125
stdout:

stderr:
Error: error inspecting object: no such container minikube

W0719 13:36:47.610931   79076 out.go:424] no arguments passed for "\n" - returning raw string
W0719 13:36:47.610940   79076 out.go:235] 

W0719 13:36:47.610952   79076 out.go:424] no arguments passed for "😿  If the above advice does not help, please let us know:\n" - returning raw string
W0719 13:36:47.610960   79076 out.go:424] no arguments passed for "👉  https://github.com/kubernetes/minikube/issues/new/choose\n\n" - returning raw string
W0719 13:36:47.610968   79076 out.go:424] no arguments passed for "Please attach the following file to the GitHub issue:\n" - returning raw string
W0719 13:36:47.611021   79076 out.go:424] no arguments passed for "😿  If the above advice does not help, please let us know:\n👉  https://github.com/kubernetes/minikube/issues/new/choose\n\nPlease attach the following file to the GitHub issue:\n- /home/filbot/.minikube/logs/lastStart.txt\n\n" - returning raw string
W0719 13:36:47.612008   79076 out.go:235] ╭────────────────────────────────────────────────────────────────────╮
╭────────────────────────────────────────────────────────────────────╮
W0719 13:36:47.612032   79076 out.go:235] │                                                                    │
│                                                                    │
W0719 13:36:47.612046   79076 out.go:235] │    😿  If the above advice does not help, please let us know:      │
│    😿  If the above advice does not help, please let us know:      │
W0719 13:36:47.612054   79076 out.go:235] │    👉  https://github.com/kubernetes/minikube/issues/new/choose    │
│    👉  https://github.com/kubernetes/minikube/issues/new/choose    │
W0719 13:36:47.612064   79076 out.go:235] │                                                                    │
│                                                                    │
W0719 13:36:47.612072   79076 out.go:235] │    Please attach the following file to the GitHub issue:           │
│    Please attach the following file to the GitHub issue:           │
W0719 13:36:47.612082   79076 out.go:235] │    - /home/filbot/.minikube/logs/lastStart.txt                     │
│    - /home/filbot/.minikube/logs/lastStart.txt                     │
W0719 13:36:47.612094   79076 out.go:235] │                                                                    │
│                                                                    │
W0719 13:36:47.612105   79076 out.go:235] ╰────────────────────────────────────────────────────────────────────╯
╰────────────────────────────────────────────────────────────────────╯
W0719 13:36:47.612117   79076 out.go:235] 

I0719 13:36:47.613147   79076 out.go:170] 

➜  ~ 

Seeing that the images are unabele to be resolved, could it be that Podman generally requires the docker.io/ pre-bits such as when I use Podman to pull images or build images, I have to do

podman pull docker.io/library/node:14-buster-slim

I've had spotty luck without qualifying the image names like that. This also works for Docker as well in some of the tests I've used.

@FilBot3
Copy link
Author

FilBot3 commented Jul 19, 2021

Upgrading to Minikube v1.22.0-0 seems to have made thing work. I can attempt to replicate this again in a virtual machine and get the same errors.

~ curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube_latest_amd64.deb
sudo dpkg -i minikube_latest_amd64.deb

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 21.8M  100 21.8M    0     0  8312k      0  0:00:02  0:00:02 --:--:-- 8309k
[sudo] password for filbot: 
(Reading database ... 389420 files and directories currently installed.)
Preparing to unpack minikube_latest_amd64.deb ...
Unpacking minikube (1.22.0-0) over (1.20.0-0) ...
Setting up minikube (1.22.0-0) ...

➜  ~ minikube start --driver=podman
😄  minikube v1.22.0 on Debian bullseye/sid
✨  Using the podman driver based on user configuration
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
💾  Downloading Kubernetes v1.21.2 preload ...
    > preloaded-images-k8s-v11-v1...: 502.14 MiB / 502.14 MiB  100.00% 5.18 MiB
    > gcr.io/k8s-minikube/kicbase...: 361.08 MiB / 361.09 MiB  100.00% 3.74 MiB
E0719 16:21:50.969400   92601 cache.go:200] Error downloading kic artifacts:  not yet implemented, see issue #8426
🔥  Creating podman container (CPUs=2, Memory=7900MB) ...
🐳  Preparing Kubernetes v1.21.2 on Docker 20.10.7 ...
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: storage-provisioner, default-storageclass
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
➜  ~ minikube addons enable ingress
    ▪ Using image docker.io/jettech/kube-webhook-certgen:v1.5.1
    ▪ Using image k8s.gcr.io/ingress-nginx/controller:v0.44.0
    ▪ Using image docker.io/jettech/kube-webhook-certgen:v1.5.1
🔎  Verifying ingress addon...
🌟  The 'ingress' addon is enabled

➜  ~ podman --version
podman version 3.0.1

➜  ~ neofetch 
             /////////////                filbot@pop-os 
         /////////////////////            ------------- 
      ///////*767////////////////         OS: Pop!_OS 21.04 x86_64 
    //////7676767676*//////////////       Host: Oryx Pro oryp6 
   /////76767//7676767//////////////      Kernel: 5.11.0-7620-generic 
  /////767676///*76767///////////////     Uptime: 8 hours, 53 mins 
 ///////767676///76767.///7676*///////    Packages: 3772 (dpkg), 60 (flatpak) 
/////////767676//76767///767676////////   Shell: zsh 5.8 
//////////76767676767////76767/////////   Resolution: 1920x1080, 1680x1050 
///////////76767676//////7676//////////   DE: GNOME 3.38.4 
////////////,7676,///////767///////////   WM: Mutter 
/////////////*7676///////76////////////   WM Theme: Pop 
///////////////7676////////////////////   Theme: Pop-dark [GTK2/3] 
 ///////////////7676///767////////////    Icons: breeze-dark [GTK2/3] 
  //////////////////////'////////////     Terminal: gnome-terminal 
   //////.7676767676767676767,//////      CPU: Intel i7-10875H (16) @ 5.100GHz 
    /////767676767676767676767/////       GPU: NVIDIA GeForce RTX 2060 Mobile 
      ///////////////////////////         GPU: Intel CometLake-H GT2 [UHD Graphics] 
         /////////////////////            Memory: 9569MiB / 31978MiB 
             /////////////

➜  ~

Although, the Container-Runtime section says docker://20.10.7, I think I'm on actually using Podman?

~ kubectl get nodes -o wide
NAME       STATUS   ROLES                  AGE     VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION        CONTAINER-RUNTIME
minikube   Ready    control-plane,master   8m17s   v1.21.2   192.168.49.2   <none>        Ubuntu 20.04.2 LTS   5.11.0-7620-generic   docker://20.10.7

@afbjorklund
Copy link
Collaborator

Although, the Container-Runtime section says docker://20.10.7, I think I'm on actually using Podman?

You are using docker-in-podman, you could also run crio-in-podman if you prefer (see --container-runtime)

@FilBot3
Copy link
Author

FilBot3 commented Jul 20, 2021

I don't believe I have docker-in-podman, unless Ubuntu (Pop!_OS) calls that out as a dependency. I need to get CRI-O setup with my laptop, so that will be a little bit.

@FilBot3
Copy link
Author

FilBot3 commented Jul 20, 2021

However, I believe that this issue is no longer relevant as upgrading Minikube to work with the updated Podman version is currently working.

@afbjorklund
Copy link
Collaborator

afbjorklund commented Jul 20, 2021

Seeing that the images are unabele to be resolved, could it be that Podman generally requires the docker.io/

This is a setting in Podman, if you want everything to break from history you can use: short-name-mode="enforcing"

See https://www.redhat.com/sysadmin/container-image-short-names

@afbjorklund
Copy link
Collaborator

afbjorklund commented Jul 20, 2021

I need to get CRI-O setup with my laptop, so that will be a little bit.

You don't need CRI-O on your laptop, only on the control-plane node.

minikube start --driver=podman --container-runtime=cri-o

It can be seen (as e.g. "minikube") when running sudo podman ps

You can access the node with minikube ssh (or with podman exec)

Currently the KIC base image used for it is based on Ubuntu LTS


Podman here is used instead of a hypervisor, like for instance KVM

@spowelljr
Copy link
Member

However, I believe that this issue is no longer relevant as upgrading Minikube to work with the updated Podman version is currently working.

Based on this comment I'm going to close this issue, but if you have any further questions feel free to reopen the issue, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
co/podman-driver podman driver issues kind/support Categorizes issue or PR as a support question.
Projects
None yet
Development

No branches or pull requests

3 participants