Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minikube failing to start on Ubuntu 22.04.3 LTS #17170

Closed
udoyen opened this issue Sep 3, 2023 · 6 comments
Closed

Minikube failing to start on Ubuntu 22.04.3 LTS #17170

udoyen opened this issue Sep 3, 2023 · 6 comments
Labels
kind/support Categorizes issue or PR as a support question.

Comments

@udoyen
Copy link

udoyen commented Sep 3, 2023

What Happened?

I can start minikube on my OS when I run the command minikube start

Attach the log file

  • ==> container status <==

  • CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID

  • ==> describe nodes <==

  • ==> dmesg <==

  • [Sep 3 15:35] Support mounting host directories into pods #2
    [ +0.004027] Support kubernetes dashboard. #3
    [ +0.358864] Hangcheck: starting hangcheck timer 0.9.1 (tick is 180 seconds, margin is 60 seconds).
    [ +0.019853] the cryptoloop driver has been deprecated and will be removed in in Linux 5.16
    [ +0.051275] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.
    [Sep 3 15:36] grpcfuse: loading out-of-tree module taints kernel.

  • ==> kernel <==

  • 15:54:44 up 18 min, 0 users, load average: 0.43, 0.34, 0.20
    Linux minikube 5.15.49-linuxkit Need a reliable and low latency local cluster setup for Kubernetes  #1 SMP Tue Sep 13 07:51:46 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
    PRETTY_NAME="Ubuntu 20.04.5 LTS"

  • ==> kubelet <==

  • -- Logs begin at Sun 2023-09-03 15:37:35 UTC, end at Sun 2023-09-03 15:54:44 UTC. --
    Sep 03 15:54:42 minikube kubelet[25959]: E0903 15:54:42.118101 25959 run.go:74] "command failed" err="failed to run Kubelet: validate service connection: validate CRI v1 runtime API for endpoint "unix:///var/run/cri-dockerd.sock": rpc error: code = Unimplemented desc = unknown service runtime.v1.RuntimeService"
    Sep 03 15:54:42 minikube systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE
    Sep 03 15:54:42 minikube systemd[1]: kubelet.service: Failed with result 'exit-code'.
    Sep 03 15:54:42 minikube systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 860.
    Sep 03 15:54:42 minikube systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
    Sep 03 15:54:42 minikube systemd[1]: Started kubelet: The Kubernetes Node Agent.
    Sep 03 15:54:42 minikube kubelet[25969]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
    Sep 03 15:54:42 minikube kubelet[25969]: I0903 15:54:42.869062 25969 server.go:415] "Kubelet version" kubeletVersion="v1.27.3"
    Sep 03 15:54:42 minikube kubelet[25969]: I0903 15:54:42.869199 25969 server.go:417] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
    Sep 03 15:54:42 minikube kubelet[25969]: I0903 15:54:42.869605 25969 server.go:837] "Client rotation is on, will bootstrap in background"
    Sep 03 15:54:42 minikube kubelet[25969]: I0903 15:54:42.871405 25969 certificate_store.go:130] Loading cert/key pair from "/var/lib/kubelet/pki/kubelet-client-current.pem".
    Sep 03 15:54:42 minikube kubelet[25969]: I0903 15:54:42.872834 25969 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/var/lib/minikube/certs/ca.crt"
    Sep 03 15:54:42 minikube kubelet[25969]: W0903 15:54:42.875643 25969 sysinfo.go:203] Nodes topology is not available, providing CPU topology
    Sep 03 15:54:42 minikube kubelet[25969]: I0903 15:54:42.878890 25969 server.go:662] "--cgroups-per-qos enabled, but --cgroup-root was not specified. defaulting to /"
    Sep 03 15:54:42 minikube kubelet[25969]: I0903 15:54:42.879233 25969 container_manager_linux.go:266] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
    Sep 03 15:54:42 minikube kubelet[25969]: I0903 15:54:42.879332 25969 container_manager_linux.go:271] "Creating Container Manager object based on Node Config" nodeConfig={RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: KubeletOOMScoreAdj:-999 ContainerRuntime: CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:systemd KubeletRootDir:/var/lib/kubelet ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: ReservedSystemCPUs: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[] HardEvictionThresholds:[]} QOSReserved:map[] CPUManagerPolicy:none CPUManagerPolicyOptions:map[] TopologyManagerScope:container CPUManagerReconcilePeriod:10s ExperimentalMemoryManagerPolicy:None ExperimentalMemoryManagerReservedMemory:[] PodPidsLimit:-1 EnforceCPULimits:true CPUCFSQuotaPeriod:100ms TopologyManagerPolicy:none ExperimentalTopologyManagerPolicyOptions:map[]}
    Sep 03 15:54:42 minikube kubelet[25969]: I0903 15:54:42.879355 25969 topology_manager.go:136] "Creating topology manager with policy per scope" topologyPolicyName="none" topologyScopeName="container"
    Sep 03 15:54:42 minikube kubelet[25969]: I0903 15:54:42.879371 25969 container_manager_linux.go:302] "Creating device plugin manager"
    Sep 03 15:54:42 minikube kubelet[25969]: I0903 15:54:42.879410 25969 state_mem.go:36] "Initialized new in-memory state store"
    Sep 03 15:54:42 minikube kubelet[25969]: E0903 15:54:42.882907 25969 run.go:74] "command failed" err="failed to run Kubelet: validate service connection: validate CRI v1 runtime API for endpoint "unix:///var/run/cri-dockerd.sock": rpc error: code = Unimplemented desc = unknown service runtime.v1.RuntimeService"
    Sep 03 15:54:42 minikube systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE
    Sep 03 15:54:42 minikube systemd[1]: kubelet.service: Failed with result 'exit-code'.
    Sep 03 15:54:43 minikube systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 861.
    Sep 03 15:54:43 minikube systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
    Sep 03 15:54:43 minikube systemd[1]: Started kubelet: The Kubernetes Node Agent.
    Sep 03 15:54:43 minikube kubelet[25979]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
    Sep 03 15:54:43 minikube kubelet[25979]: I0903 15:54:43.595452 25979 server.go:415] "Kubelet version" kubeletVersion="v1.27.3"
    Sep 03 15:54:43 minikube kubelet[25979]: I0903 15:54:43.595501 25979 server.go:417] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
    Sep 03 15:54:43 minikube kubelet[25979]: I0903 15:54:43.595685 25979 server.go:837] "Client rotation is on, will bootstrap in background"
    Sep 03 15:54:43 minikube kubelet[25979]: I0903 15:54:43.597027 25979 certificate_store.go:130] Loading cert/key pair from "/var/lib/kubelet/pki/kubelet-client-current.pem".
    Sep 03 15:54:43 minikube kubelet[25979]: I0903 15:54:43.597772 25979 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/var/lib/minikube/certs/ca.crt"
    Sep 03 15:54:43 minikube kubelet[25979]: W0903 15:54:43.600018 25979 sysinfo.go:203] Nodes topology is not available, providing CPU topology
    Sep 03 15:54:43 minikube kubelet[25979]: I0903 15:54:43.603067 25979 server.go:662] "--cgroups-per-qos enabled, but --cgroup-root was not specified. defaulting to /"
    Sep 03 15:54:43 minikube kubelet[25979]: I0903 15:54:43.603288 25979 container_manager_linux.go:266] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
    Sep 03 15:54:43 minikube kubelet[25979]: I0903 15:54:43.603350 25979 container_manager_linux.go:271] "Creating Container Manager object based on Node Config" nodeConfig={RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: KubeletOOMScoreAdj:-999 ContainerRuntime: CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:systemd KubeletRootDir:/var/lib/kubelet ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: ReservedSystemCPUs: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[] HardEvictionThresholds:[]} QOSReserved:map[] CPUManagerPolicy:none CPUManagerPolicyOptions:map[] TopologyManagerScope:container CPUManagerReconcilePeriod:10s ExperimentalMemoryManagerPolicy:None ExperimentalMemoryManagerReservedMemory:[] PodPidsLimit:-1 EnforceCPULimits:true CPUCFSQuotaPeriod:100ms TopologyManagerPolicy:none ExperimentalTopologyManagerPolicyOptions:map[]}
    Sep 03 15:54:43 minikube kubelet[25979]: I0903 15:54:43.603381 25979 topology_manager.go:136] "Creating topology manager with policy per scope" topologyPolicyName="none" topologyScopeName="container"
    Sep 03 15:54:43 minikube kubelet[25979]: I0903 15:54:43.603395 25979 container_manager_linux.go:302] "Creating device plugin manager"
    Sep 03 15:54:43 minikube kubelet[25979]: I0903 15:54:43.603420 25979 state_mem.go:36] "Initialized new in-memory state store"
    Sep 03 15:54:43 minikube kubelet[25979]: E0903 15:54:43.604816 25979 run.go:74] "command failed" err="failed to run Kubelet: validate service connection: validate CRI v1 runtime API for endpoint "unix:///var/run/cri-dockerd.sock": rpc error: code = Unimplemented desc = unknown service runtime.v1.RuntimeService"
    Sep 03 15:54:43 minikube systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE
    Sep 03 15:54:43 minikube systemd[1]: kubelet.service: Failed with result 'exit-code'.
    Sep 03 15:54:44 minikube systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 862.
    Sep 03 15:54:44 minikube systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
    Sep 03 15:54:44 minikube systemd[1]: Started kubelet: The Kubernetes Node Agent.
    Sep 03 15:54:44 minikube kubelet[26109]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
    Sep 03 15:54:44 minikube kubelet[26109]: I0903 15:54:44.343754 26109 server.go:415] "Kubelet version" kubeletVersion="v1.27.3"
    Sep 03 15:54:44 minikube kubelet[26109]: I0903 15:54:44.343795 26109 server.go:417] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
    Sep 03 15:54:44 minikube kubelet[26109]: I0903 15:54:44.343919 26109 server.go:837] "Client rotation is on, will bootstrap in background"
    Sep 03 15:54:44 minikube kubelet[26109]: I0903 15:54:44.345059 26109 certificate_store.go:130] Loading cert/key pair from "/var/lib/kubelet/pki/kubelet-client-current.pem".
    Sep 03 15:54:44 minikube kubelet[26109]: I0903 15:54:44.345754 26109 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/var/lib/minikube/certs/ca.crt"
    Sep 03 15:54:44 minikube kubelet[26109]: W0903 15:54:44.347573 26109 sysinfo.go:203] Nodes topology is not available, providing CPU topology
    Sep 03 15:54:44 minikube kubelet[26109]: I0903 15:54:44.350361 26109 server.go:662] "--cgroups-per-qos enabled, but --cgroup-root was not specified. defaulting to /"
    Sep 03 15:54:44 minikube kubelet[26109]: I0903 15:54:44.350666 26109 container_manager_linux.go:266] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
    Sep 03 15:54:44 minikube kubelet[26109]: I0903 15:54:44.350740 26109 container_manager_linux.go:271] "Creating Container Manager object based on Node Config" nodeConfig={RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: KubeletOOMScoreAdj:-999 ContainerRuntime: CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:systemd KubeletRootDir:/var/lib/kubelet ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: ReservedSystemCPUs: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[] HardEvictionThresholds:[]} QOSReserved:map[] CPUManagerPolicy:none CPUManagerPolicyOptions:map[] TopologyManagerScope:container CPUManagerReconcilePeriod:10s ExperimentalMemoryManagerPolicy:None ExperimentalMemoryManagerReservedMemory:[] PodPidsLimit:-1 EnforceCPULimits:true CPUCFSQuotaPeriod:100ms TopologyManagerPolicy:none ExperimentalTopologyManagerPolicyOptions:map[]}
    Sep 03 15:54:44 minikube kubelet[26109]: I0903 15:54:44.350763 26109 topology_manager.go:136] "Creating topology manager with policy per scope" topologyPolicyName="none" topologyScopeName="container"
    Sep 03 15:54:44 minikube kubelet[26109]: I0903 15:54:44.350776 26109 container_manager_linux.go:302] "Creating device plugin manager"
    Sep 03 15:54:44 minikube kubelet[26109]: I0903 15:54:44.350808 26109 state_mem.go:36] "Initialized new in-memory state store"
    Sep 03 15:54:44 minikube kubelet[26109]: E0903 15:54:44.352901 26109 run.go:74] "command failed" err="failed to run Kubelet: validate service connection: validate CRI v1 runtime API for endpoint "unix:///var/run/cri-dockerd.sock": rpc error: code = Unimplemented desc = unknown service runtime.v1.RuntimeService"
    Sep 03 15:54:44 minikube systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE
    Sep 03 15:54:44 minikube systemd[1]: kubelet.service: Failed with result 'exit-code'.

Operating System

Ubuntu

Driver

Docker

@rmsilva1973
Copy link
Contributor

@udoyen Please start with "--logs=logs.txt" and attach (please, don't paste) the log file to the issue.

@rmsilva1973
Copy link
Contributor

/kind support

@k8s-ci-robot k8s-ci-robot added the kind/support Categorizes issue or PR as a support question. label Sep 4, 2023
@spowelljr
Copy link
Member

Hi @udoyen, we haven't heard back from you, do you still have this issue?
There isn't enough information in this issue to make it actionable, and a long enough duration has passed, so this issue is likely difficult to replicate.

I will close this issue for now but feel free to reopen when you feel ready to provide more information.

@mtovmassian
Copy link

I encountered the same error while upgrading minikube (docker driver) from v1.30.1 to v1.32.0.

The only solution I found was the brutal one:

  • Delete the current local minikube/kicbase image: docker rmi gcr.io/k8s-minikube/kicbase:v0.0.34
  • Delete the local minikube directory content: rm -r ~/.minikube/*
  • Delete the minikube volume: docker volume rm minikube

@spowelljr
Copy link
Member

@mtovmassian Did you try the minikube delete command?

@mtovmassian
Copy link

mtovmassian commented Dec 12, 2023

@spowelljr I did not. But thank you for mentioning minikube delete it would do the job as well for sure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/support Categorizes issue or PR as a support question.
Projects
None yet
Development

No branches or pull requests

5 participants