Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

k8s v1.15-beta1: kubelet: unknown flag: --allow-privileged #4371

Closed
tstromberg opened this issue May 29, 2019 · 12 comments
Closed

k8s v1.15-beta1: kubelet: unknown flag: --allow-privileged #4371

tstromberg opened this issue May 29, 2019 · 12 comments
Labels
area/kubernetes-versions Improving support for versions of Kubernetes good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.

Comments

@tstromberg
Copy link
Contributor

If minikube start --kubernetes-version=v1.15.0-beta.1 is used, a failure occurs:


Unfortunately, an error has occurred:
	timed out waiting for the condition

This error is likely caused by:
	- The kubelet is not running
	- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
	- 'systemctl status kubelet'
	- 'journalctl -xeu kubelet'

Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
	- 'docker ps -a | grep kube | grep -v pause'
	Once you have found the failing container, you can inspect its logs with:
	- 'docker logs CONTAINERID'

: Process exited with status 1

😿  Sorry that minikube crashed. If this was unexpected, we would love to hear from you:
👉  https://github.com/kubernetes/minikube/issues/new
❌  Problems detected in "kubelet":
    May 29 18:23:32 k8s115 kubelet[2944]: F0529 18:23:32.984077    2944 server.go:156] unknown flag: --allow-privileged
@tstromberg tstromberg added good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. labels May 29, 2019
@tstromberg
Copy link
Contributor Author

tstromberg commented May 29, 2019

Deprecation notice for this flag: https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.15.md#action-required

If anyone wants to fix support for v1.15, here is where to get started:

NewUnversionedOption(Kubelet, "allow-privileged", "true"),

This will need to be changed to a NewUnversionedOption() so that it isn't set for v1.15+. If Kubernetes v1.15.0-beta1 is working properly, the PR should also update NewestKubernetesVersion so that this release is tested in our integration tests:

var NewestKubernetesVersion = "v1.14.2"

@tstromberg tstromberg added priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. area/kubernetes-versions Improving support for versions of Kubernetes labels May 29, 2019
@blueelvis
Copy link
Contributor

Am I correct in understanding that I would have to remove the above line 245 which you have mentioned and add code similar to this

so that this is set only for versions which support it? (Basically setting the LessThanOrEqual

@fenglixa
Copy link
Contributor

fenglixa commented Jun 3, 2019

I would like to provide PR for this ticket. Thanks @blueelvis @tstromberg

oz123 added a commit to oz123/python that referenced this issue Jun 21, 2019
Sometimes, minikube lags behind kubernetes which can lead
to a breakage of the test suite, since minikube will fail to start
with the latest kubernetes version.
See for example [this issue](kubernetes/minikube#4371).

This change defaults to leave the decision of which k8s version to use,
to minikube itself. This is defined in:

https://github.com/kubernetes/minikube/master/pkg/minikube/constants/constants.go

However, if one really desires it is still possible to pass

`--kubernetes-version=X.Y.Z`

to minikube initialization start command via an environment variable
before invoking the test suite:

export $MINIKUBE_ARGS="--kubernetes-version=X.Y.Z"

This allows ofcourse passing other flags to minikube also.
@the-redback
Copy link

Kubernetes-1.15.0 unable to run on minikube 1.1.1 as this error still exists on last release.
Is there any early plan of release? @tstromberg

@fenglixa
Copy link
Contributor

@the-redback Let me check.

@fenglixa
Copy link
Contributor

fenglixa commented Jun 24, 2019

With latest code, minikube could start kubenetes-1.15.0 successfully, as below my test result:
image

@the-redback I am worried the fixes was not in the build you are using, wish you could help double check. Thanks.

@tstromberg
Copy link
Contributor Author

tstromberg commented Jun 24, 2019

@the-redback - minikube v1.2.0 with v1.15.0 support will be published this week.

If you build from source code, v1.15.0 will work fine.

@the-redback
Copy link

Thanks @tstromberg .

@the-redback
Copy link

the-redback commented Jun 24, 2019

@fenglixa , It's strange see the log I am getting below:

minikube start --kubernetes-version=v1.15.0                                        
😄  minikube v1.1.1 on linux (amd64)
🔥  Creating virtualbox VM (CPUs=2, Memory=4096MB, Disk=20000MB) ...
🐳  Configuring environment for Kubernetes v1.15.0 on Docker 18.09.6
🚜  Pulling images ...
🚀  Launching Kubernetes ... 

💣  Error starting cluster: cmd failed: sudo /usr/bin/kubeadm init --config /var/lib/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--data-minikube,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap
[init] Using Kubernetes version: v1.15.0
[preflight] Running pre-flight checks
	[WARNING Service-Docker]: docker service is not enabled, please run 'systemctl enable docker.service'
	[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
	[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs/"
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [minikube localhost] and IPs [192.168.99.132 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [minikube localhost] and IPs [192.168.99.132 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

Unfortunately, an error has occurred:
	timed out waiting for the condition

This error is likely caused by:
	- The kubelet is not running
	- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
	- 'systemctl status kubelet'
	- 'journalctl -xeu kubelet'

Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
	- 'docker ps -a | grep kube | grep -v pause'
	Once you have found the failing container, you can inspect its logs with:
	- 'docker logs CONTAINERID'

: Process exited with status 1

😿  Sorry that minikube crashed. If this was unexpected, we would love to hear from you:
👉  https://github.com/kubernetes/minikube/issues/new
❌  Problems detected in "kubelet":
    Jun 24 03:42:49 minikube kubelet[3337]: F0624 03:42:49.281947    3337 server.go:156] unknown flag: --allow-privileged
$ minikube logs
==> dmesg <==
[  +5.001997] hpet1: lost 318 rtc interrupts
[  +4.999172] hpet1: lost 318 rtc interrupts
[  +5.007680] hpet1: lost 319 rtc interrupts
[  +4.999116] hpet1: lost 318 rtc interrupts
[  +5.003335] hpet1: lost 318 rtc interrupts
[  +5.003097] hpet1: lost 318 rtc interrupts
[Jun24 03:43] hpet1: lost 318 rtc interrupts
[  +5.008778] hpet1: lost 319 rtc interrupts
[  +5.007643] hpet1: lost 318 rtc interrupts
[  +5.007168] hpet1: lost 319 rtc interrupts
[  +5.006272] hpet1: lost 318 rtc interrupts
[  +5.008691] hpet1: lost 319 rtc interrupts
[  +5.002578] hpet1: lost 319 rtc interrupts
[  +5.003898] hpet1: lost 318 rtc interrupts
[  +5.007827] hpet1: lost 319 rtc interrupts
[  +5.009710] hpet1: lost 318 rtc interrupts
[  +5.004324] hpet1: lost 320 rtc interrupts
[  +5.006799] hpet1: lost 318 rtc interrupts
[Jun24 03:44] hpet1: lost 319 rtc interrupts
[  +5.007505] hpet1: lost 318 rtc interrupts
[  +5.004414] hpet1: lost 319 rtc interrupts
[  +5.003668] hpet1: lost 318 rtc interrupts
[  +5.002914] hpet1: lost 318 rtc interrupts
[  +5.001436] hpet1: lost 318 rtc interrupts
[  +5.001063] hpet1: lost 318 rtc interrupts
[  +5.002960] hpet1: lost 318 rtc interrupts
[  +4.999915] hpet1: lost 318 rtc interrupts
[  +5.007372] hpet1: lost 320 rtc interrupts
[  +5.001900] hpet1: lost 318 rtc interrupts
[  +5.002593] hpet1: lost 318 rtc interrupts
[Jun24 03:45] hpet1: lost 318 rtc interrupts
[  +5.001272] hpet1: lost 319 rtc interrupts
[  +5.000045] hpet1: lost 318 rtc interrupts
[  +5.003967] hpet1: lost 318 rtc interrupts
[  +4.998959] hpet1: lost 318 rtc interrupts
[  +5.008547] hpet1: lost 318 rtc interrupts
[  +5.000154] hpet1: lost 318 rtc interrupts
[  +5.007697] hpet1: lost 319 rtc interrupts
[  +4.999678] hpet1: lost 318 rtc interrupts
[  +5.008492] hpet1: lost 318 rtc interrupts
[  +5.002611] hpet1: lost 320 rtc interrupts
[  +5.008974] hpet1: lost 318 rtc interrupts
[Jun24 03:46] hpet1: lost 318 rtc interrupts
[  +5.003711] hpet1: lost 318 rtc interrupts
[  +5.000451] hpet1: lost 318 rtc interrupts
[  +5.001327] hpet1: lost 319 rtc interrupts
[  +5.001412] hpet1: lost 318 rtc interrupts
[  +5.006476] hpet1: lost 319 rtc interrupts
[  +5.001029] hpet1: lost 319 rtc interrupts
[  +5.009116] hpet1: lost 319 rtc interrupts

==> kernel <==
 03:46:37 up 7 min,  1 user,  load average: 0.06, 0.32, 0.22
Linux minikube 4.15.0 #1 SMP Thu Jun 6 15:07:18 PDT 2019 x86_64 GNU/Linux

==> kubelet <==
-- Logs begin at Mon 2019-06-24 03:40:05 UTC, end at Mon 2019-06-24 03:46:37 UTC. --
Jun 24 03:46:14 minikube kubelet[3549]:       --registry-burst int32                                                                                      Maximum size of a bursty pulls, temporarily allows pulls to burst to this number, while still not exceeding registry-qps. Only used if --registry-qps > 0 (default 10) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 24 03:46:14 minikube kubelet[3549]:       --registry-qps int32                                                                                        If > 0, limit registry pull QPS to this value.  If 0, unlimited. (default 5) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 24 03:46:14 minikube kubelet[3549]:       --resolv-conf string                                                                                        Resolver configuration file used as the basis for the container DNS resolution configuration. (default "/etc/resolv.conf") (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 24 03:46:14 minikube kubelet[3549]:       --root-dir string                                                                                           Directory path for managing kubelet files (volume mounts,etc). (default "/var/lib/kubelet")
Jun 24 03:46:14 minikube kubelet[3549]:       --rotate-certificates                                                                                       <Warning: Beta feature> Auto rotate the kubelet client certificates by requesting new certificates from the kube-apiserver when the certificate expiration approaches. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 24 03:46:14 minikube kubelet[3549]:       --rotate-server-certificates                                                                                Auto-request and rotate the kubelet serving certificates by requesting new certificates from the kube-apiserver when the certificate expiration approaches. Requires the RotateKubeletServerCertificate feature gate to be enabled, and approval of the submitted CertificateSigningRequest objects. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 24 03:46:14 minikube kubelet[3549]:       --runonce                                                                                                   If true, exit after spawning pods from static pod files or remote urls. Exclusive with --enable-server
Jun 24 03:46:14 minikube kubelet[3549]:       --runtime-cgroups string                                                                                    Optional absolute name of cgroups to create and run the runtime in.
Jun 24 03:46:14 minikube kubelet[3549]:       --runtime-request-timeout duration                                                                          Timeout of all runtime requests except long running request - pull, logs, exec and attach. When timeout exceeded, kubelet will cancel the request, throw out an error and retry later. (default 2m0s) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 24 03:46:14 minikube kubelet[3549]:       --seccomp-profile-root string                                                                               <Warning: Alpha feature> Directory path for seccomp profiles. (default "/var/lib/kubelet/seccomp")
Jun 24 03:46:14 minikube kubelet[3549]:       --serialize-image-pulls                                                                                     Pull images one at a time. We recommend *not* changing the default value on nodes that run docker daemon with version < 1.9 or an Aufs storage backend. Issue #10959 has more details. (default true) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 24 03:46:14 minikube kubelet[3549]:       --skip-headers                                                                                              If true, avoid header prefixes in the log messages
Jun 24 03:46:14 minikube kubelet[3549]:       --skip-log-headers                                                                                          If true, avoid headers when opening log files
Jun 24 03:46:14 minikube kubelet[3549]:       --stderrthreshold severity                                                                                  logs at or above this threshold go to stderr (default 2)
Jun 24 03:46:14 minikube kubelet[3549]:       --storage-driver-buffer-duration duration                                                                   Writes in the storage driver will be buffered for this duration, and committed to the non memory backends as a single transaction (default 1m0s) (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.)
Jun 24 03:46:14 minikube kubelet[3549]:       --storage-driver-db string                                                                                  database name (default "cadvisor") (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.)
Jun 24 03:46:14 minikube kubelet[3549]:       --storage-driver-host string                                                                                database host:port (default "localhost:8086") (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.)
Jun 24 03:46:14 minikube kubelet[3549]:       --storage-driver-password string                                                                            database password (default "root") (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.)
Jun 24 03:46:14 minikube kubelet[3549]:       --storage-driver-secure                                                                                     use secure connection with database (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.)
Jun 24 03:46:14 minikube kubelet[3549]:       --storage-driver-table string                                                                               table name (default "stats") (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.)
Jun 24 03:46:14 minikube kubelet[3549]:       --storage-driver-user string                                                                                database username (default "root") (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.)
Jun 24 03:46:14 minikube kubelet[3549]:       --streaming-connection-idle-timeout duration                                                                Maximum time a streaming connection can be idle before the connection is automatically closed. 0 indicates no timeout. Example: '5m' (default 4h0m0s) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 24 03:46:14 minikube kubelet[3549]:       --sync-frequency duration                                                                                   Max period between synchronizing running containers and config (default 1m0s) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 24 03:46:14 minikube kubelet[3549]:       --system-cgroups /                                                                                          Optional absolute name of cgroups in which to place all non-kernel processes that are not already inside a cgroup under /. Empty for no container. Rolling back the flag requires a reboot. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 24 03:46:14 minikube kubelet[3549]:       --system-reserved mapStringString                                                                           A set of ResourceName=ResourceQuantity (e.g. cpu=200m,memory=500Mi,ephemeral-storage=1Gi) pairs that describe resources reserved for non-kubernetes components. Currently only cpu and memory are supported. See http://kubernetes.io/docs/user-guide/compute-resources for more detail. [default=none] (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 24 03:46:14 minikube kubelet[3549]:       --system-reserved-cgroup string                                                                             Absolute name of the top level cgroup that is used to manage non-kubernetes components for which compute resources were reserved via '--system-reserved' flag. Ex. '/system-reserved'. [default=''] (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 24 03:46:14 minikube kubelet[3549]:       --tls-cert-file string                                                                                      File containing x509 Certificate used for serving HTTPS (with intermediate certs, if any, concatenated after server cert). If --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory passed to --cert-dir. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 24 03:46:14 minikube kubelet[3549]:       --tls-cipher-suites strings                                                                                 Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be used. Possible values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_RC4_128_SHA,TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_RC4_128_SHA,TLS_RSA_WITH_3DES_EDE_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA256,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_RC4_128_SHA (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 24 03:46:14 minikube kubelet[3549]:       --tls-min-version string                                                                                    Minimum TLS version supported. Possible values: VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13 (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 24 03:46:14 minikube kubelet[3549]:       --tls-private-key-file string                                                                               File containing x509 private key matching --tls-cert-file. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 24 03:46:14 minikube kubelet[3549]:   -v, --v Level                                                                                                   number for the log level verbosity
Jun 24 03:46:14 minikube kubelet[3549]:       --version version[=true]                                                                                    Print version information and quit
Jun 24 03:46:14 minikube kubelet[3549]:       --vmodule moduleSpec                                                                                        comma-separated list of pattern=N settings for file-filtered logging
Jun 24 03:46:14 minikube kubelet[3549]:       --volume-plugin-dir string                                                                                  The full path of the directory in which to search for additional third party volume plugins (default "/usr/libexec/kubernetes/kubelet-plugins/volume/exec/")
Jun 24 03:46:14 minikube kubelet[3549]:       --volume-stats-agg-period duration                                                                          Specifies interval for kubelet to calculate and cache the volume disk usage for all pods and volumes.  To disable volume calculations, set to 0. (default 1m0s) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 24 03:46:14 minikube kubelet[3549]: F0624 03:46:14.291998    3549 server.go:156] unknown flag: --allow-privileged
Jun 24 03:46:14 minikube systemd[1]: kubelet.service: Main process exited, code=exited, status=255/n/a
Jun 24 03:46:14 minikube systemd[1]: kubelet.service: Failed with result 'exit-code'.
Jun 24 03:46:24 minikube systemd[1]: kubelet.service: Service hold-off time over, scheduling restart.
Jun 24 03:46:24 minikube systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 31.
Jun 24 03:46:24 minikube systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
Jun 24 03:46:24 minikube systemd[1]: Started kubelet: The Kubernetes Node Agent.
Jun 24 03:46:24 minikube systemd[1]: kubelet.service: Main process exited, code=exited, status=255/n/a
Jun 24 03:46:24 minikube systemd[1]: kubelet.service: Failed with result 'exit-code'.
Jun 24 03:46:34 minikube systemd[1]: kubelet.service: Service hold-off time over, scheduling restart.
Jun 24 03:46:34 minikube systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 32.
Jun 24 03:46:34 minikube systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
Jun 24 03:46:34 minikube systemd[1]: Started kubelet: The Kubernetes Node Agent.
Jun 24 03:46:34 minikube systemd[1]: kubelet.service: Main process exited, code=exited, status=255/n/a
Jun 24 03:46:34 minikube systemd[1]: kubelet.service: Failed with result 'exit-code'.
$ minikube version
minikube version: v1.1.1

$ virtualbox --help
Oracle VM VirtualBox VM Selector v6.0.6_Ubuntu

I sshed into minikube to see the kubelet version:

$ kubelet --version
Kubernetes v1.15.0

$ systemctl status kubelet
● kubelet.service - kubelet: The Kubernetes Node Agent
   Loaded: loaded (/usr/lib/systemd/system/kubelet.service; disabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/kubelet.service.d
           └─10-kubeadm.conf
   Active: activating (auto-restart) (Result: exit-code) since Mon 2019-06-24 03:47:26 UTC; 6s ago
     Docs: http://kubernetes.io/docs/
  Process: 3668 ExecStart=/usr/bin/kubelet --allow-privileged=true --authorization-mode=Webhook --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --cgroup-driver=cgroupfs --client-ca-file=/var/lib/minikube/certs/ca.crt --cluster-dns=10.96.0.10 --cluster-domain=cluster.local --container-runtime=docker --fail-swap-on=false --hostname-override=minikube --kubeconfig=/etc/kubernetes/kubelet.conf --pod-manifest-path=/etc/kubernetes/manifests (code=exited, status=255)
 Main PID: 3668 (code=exited, status=255)

Jun 24 03:47:26 minikube systemd[1]: kubelet.service: Main process exited, code=exited, status=255/n/a
Jun 24 03:47:26 minikube systemd[1]: kubelet.service: Failed with result 'exit-code'.

@the-redback
Copy link

@fenglixa, I see. you are using custom build instead of last stable release, right? That's what I was reporting in the first place. Then I got confused seeing your log. :)

anyway, end of misunderstanding. I am okay to wait one more week for next release. :D

@fenglixa
Copy link
Contributor

Yes, I am using source code build, so from @tstromberg, this fix will be included in v1.2.0.

@tstromberg
Copy link
Contributor Author

minikube v1.2.0 is released, and defaults to Kubernetes v1.1.15. Thank you for your patience!

oz123 added a commit to oz123/python that referenced this issue Jun 25, 2019
Sometimes, minikube lags behind kubernetes which can lead
to a breakage of the test suite, since minikube will fail to start
with the latest kubernetes version.
See for example [this issue](kubernetes/minikube#4371).

This change defaults to leave the decision of which k8s version to use,
to minikube itself. This is defined in:

https://github.com/kubernetes/minikube/master/pkg/minikube/constants/constants.go

However, if one really desires it is still possible to pass

`--kubernetes-version=X.Y.Z`

to minikube initialization start command via an environment variable
before invoking the test suite:

export $MINIKUBE_ARGS="--kubernetes-version=X.Y.Z"

This allows ofcourse passing other flags to minikube also.
oz123 added a commit to oz123/python that referenced this issue Jun 25, 2019
Sometimes, minikube lags behind kubernetes which can lead
to a breakage of the test suite, since minikube will fail to start
with the latest kubernetes version.
See for example [this issue](kubernetes/minikube#4371).

This change defaults to leave the decision of which k8s version to use,
to minikube itself. This is defined in:

https://github.com/kubernetes/minikube/master/pkg/minikube/constants/constants.go

However, if one really desires it is still possible to pass

`--kubernetes-version=X.Y.Z`

to minikube initialization start command via an environment variable
before invoking the test suite:

export $MINIKUBE_ARGS="--kubernetes-version=X.Y.Z"

This allows ofcourse passing other flags to minikube also.
bh717 pushed a commit to bh717/python-dapp that referenced this issue Apr 1, 2024
Sometimes, minikube lags behind kubernetes which can lead
to a breakage of the test suite, since minikube will fail to start
with the latest kubernetes version.
See for example [this issue](kubernetes/minikube#4371).

This change defaults to leave the decision of which k8s version to use,
to minikube itself. This is defined in:

https://github.com/kubernetes/minikube/master/pkg/minikube/constants/constants.go

However, if one really desires it is still possible to pass

`--kubernetes-version=X.Y.Z`

to minikube initialization start command via an environment variable
before invoking the test suite:

export $MINIKUBE_ARGS="--kubernetes-version=X.Y.Z"

This allows ofcourse passing other flags to minikube also.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/kubernetes-versions Improving support for versions of Kubernetes good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
Projects
None yet
Development

No branches or pull requests

4 participants