none: Fix 'minikube delete' issues when the apiserver is down #8664

tstromberg · 2020-07-07T22:14:10Z

While testing the none driver, I noticed that APIServerStatus could get into an infinite retry loop, which prevented minikube delete from working properly:

I0707 15:08:05.442332 2165558 retry.go:30] will retry after 7.91826225s: state is "Stopped"                                                                                         
I0707 15:08:13.360813 2165558 api_server.go:215] Checking apiserver healthz at https://172.31.120.49:8443/healthz ...                                                               
I0707 15:08:13.361324 2165558 api_server.go:225] stopped: https://172.31.120.49:8443/healthz: Get https://172.31.120.49:8443/healthz: dial tcp 172.31.120.49:8443: connect: connecti
on refused                                                                                                                                                                          
I0707 15:08:13.361395 2165558 retry.go:30] will retry after 9.953714808s: state is "Stopped"

This PR fixes 3 issues which prevented minikube delete from working properly with the none driver:

retry.Local did not respect maxTime!
apiServerHealthz propagated upwards the error for non-running clusters. machine/delete.go interpreted this error as an inability to get state, which allowed it to assume that Kubernetes had already been deleted.
The timeout was too danged long for deleting a stopped cluster.

Here is an example of the 3 changes working together to allow minikube delete work well for the none driver when the apiserver isn't available:

I0707 15:25:12.925721 2188998 api_server.go:231] stopped: https://172.31.120.49:8443/healthz: Get https://172.31.120.49:8443/healthz: dial tcp 172.31.120.49:8443: connect: connection refused
I0707 15:25:12.925779 2188998 system_pods.go:161] Checking kubelet status ...
I0707 15:25:12.925859 2188998 exec_runner.go:49] Run: sudo systemctl is-active --quiet service kubelet
🔥  Deleting "minikube" in none ...
I0707 15:25:12.968473 2188998 exec_runner.go:49] Run: sudo systemctl stop -f kubelet
I0707 15:25:13.004981 2188998 exec_runner.go:49] Run: docker ps -a --filter=name=k8s_ --format={{.ID}}
I0707 15:25:13.051234 2188998 none.go:185] Removing: [/var/tmp/minikube /etc/kubernetes/manifests /var/lib/minikube]
I0707 15:25:13.051330 2188998 exec_runner.go:49] Run: sudo rm -rf /var/tmp/minikube /etc/kubernetes/manifests /var/lib/minikube
I0707 15:25:13.102587 2188998 lock.go:35] WriteFile acquiring /root/.kube/config: {Name:mk72a1487fd2da23da9e8181e16f352a6105bd56 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
💀  Removed all traces of the "minikube" cluster.

tstromberg · 2020-07-07T22:14:17Z

/ok-to-test

k8s-ci-robot · 2020-07-07T22:14:43Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tstromberg

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [tstromberg]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

TravisBuddy · 2020-07-07T22:20:06Z

Travis tests have failed

Hey @tstromberg,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

1st Build

View build log

make test

which go-bindata || GO111MODULE=off GOBIN="/home/travis/gopath/bin" go get github.com/jteeuwen/go-bindata/...
PATH="/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.13.9.linux.amd64/bin:/home/travis/bin:/home/travis/bin:/home/travis/.local/bin:/usr/local/lib/jvm/openjdk11/bin:/opt/pyenv/shims:/home/travis/.phpenv/shims:/home/travis/perl5/perlbrew/bin:/home/travis/.nvm/versions/node/v8.12.0/bin:/home/travis/.rvm/gems/ruby-2.5.3/bin:/home/travis/.rvm/gems/ruby-2.5.3@global/bin:/home/travis/.rvm/rubies/ruby-2.5.3/bin:/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.11.1.linux.amd64/bin:/usr/local/maven-3.6.0/bin:/usr/local/cmake-3.12.4/bin:/usr/local/clang-7.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/home/travis/.rvm/bin:/home/travis/.phpenv/bin:/opt/pyenv/bin:/home/travis/.yarn/bin:/home/travis/gopath/bin" go-bindata -nomemcopy -o pkg/minikube/assets/assets.go -pkg assets deploy/addons/...
gofmt -s -w pkg/minikube/assets/assets.go
which go-bindata || GO111MODULE=off GOBIN="/home/travis/gopath/bin" go get github.com/jteeuwen/go-bindata/...
/home/travis/gopath/bin/go-bindata
PATH="/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.13.9.linux.amd64/bin:/home/travis/bin:/home/travis/bin:/home/travis/.local/bin:/usr/local/lib/jvm/openjdk11/bin:/opt/pyenv/shims:/home/travis/.phpenv/shims:/home/travis/perl5/perlbrew/bin:/home/travis/.nvm/versions/node/v8.12.0/bin:/home/travis/.rvm/gems/ruby-2.5.3/bin:/home/travis/.rvm/gems/ruby-2.5.3@global/bin:/home/travis/.rvm/rubies/ruby-2.5.3/bin:/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.11.1.linux.amd64/bin:/usr/local/maven-3.6.0/bin:/usr/local/cmake-3.12.4/bin:/usr/local/clang-7.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/home/travis/.rvm/bin:/home/travis/.phpenv/bin:/opt/pyenv/bin:/home/travis/.yarn/bin:/home/travis/gopath/bin" go-bindata -nomemcopy -o pkg/minikube/translate/translations.go -pkg translate translations/...
gofmt -s -w pkg/minikube/translate/translations.go
MINIKUBE_LDFLAGS="-X k8s.io/minikube/pkg/version.version=v1.12.0-beta.1 -X k8s.io/minikube/pkg/version.isoVersion=v1.11.0 -X k8s.io/minikube/pkg/version.isoPath=minikube/iso -X k8s.io/minikube/pkg/version.gitCommitID="0369a1e24e4e3f70e4aa537c28f6d74c3564edcf"" ./test.sh
= schema_check ==========================================================
go: downloading github.com/xeipuuv/gojsonschema v0.0.0-20180618132009-1d523034197f
go: extracting github.com/xeipuuv/gojsonschema v0.0.0-20180618132009-1d523034197f
go: downloading github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415
go: extracting github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415
go: downloading github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f
go: extracting github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f
go: finding github.com/xeipuuv/gojsonschema v0.0.0-20180618132009-1d523034197f
go: finding github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415
go: finding github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f
ok
= go test ===============================================================
go: downloading github.com/machine-drivers/machine v0.7.1-0.20200323212942-41eb826190d8
go: downloading github.com/spf13/cobra v1.0.0
go: downloading github.com/blang/semver v3.5.0+incompatible
go: downloading github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: downloading k8s.io/client-go v0.17.3
go: downloading k8s.io/api v0.17.3
go: extracting github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: extracting github.com/blang/semver v3.5.0+incompatible
go: extracting github.com/spf13/cobra v1.0.0
go: downloading k8s.io/apimachinery v0.17.3
go: extracting github.com/machine-drivers/machine v0.7.1-0.20200323212942-41eb826190d8
go: downloading github.com/phayes/freeport v0.0.0-20180830031419-95f893ade6f2
go: downloading github.com/cloudfoundry-attic/jibber_jabber v0.0.0-20151120183258-bcc4c8345a21
go: downloading github.com/otiai10/copy v1.0.2
go: downloading github.com/afbjorklund/go-containerregistry v0.0.0-20200602203322-347d93793dc9
go: extracting github.com/phayes/freeport v0.0.0-20180830031419-95f893ade6f2
go: downloading github.com/opencontainers/go-digest v1.0.0-rc1
go: extracting k8s.io/apimachinery v0.17.3
go: extracting github.com/opencontainers/go-digest v1.0.0-rc1
go: extracting github.com/otiai10/copy v1.0.2
go: downloading github.com/google/slowjam v0.0.0-20200530021616-df27e642fe7b
go: extracting k8s.io/client-go v0.17.3
go: extracting github.com/cloudfoundry-attic/jibber_jabber v0.0.0-20151120183258-bcc4c8345a21
go: downloading github.com/shirou/gopsutil v2.18.12+incompatible
go: extracting github.com/google/slowjam v0.0.0-20200530021616-df27e642fe7b
go: extracting github.com/afbjorklund/go-containerregistry v0.0.0-20200602203322-347d93793dc9
go: downloading github.com/intel-go/cpuid v0.0.0-20181003105527-1a4a6f06a1c6
go: extracting github.com/shirou/gopsutil v2.18.12+incompatible
go: extracting github.com/intel-go/cpuid v0.0.0-20181003105527-1a4a6f06a1c6
go: downloading golang.org/x/crypto v0.0.0-20200510223506-06a226fb4e37
go: downloading golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a
go: downloading github.com/machine-drivers/docker-machine-driver-vmware v0.1.1
go: downloading github.com/mitchellh/go-ps v0.0.0-20190716172923-621e5597135b
go: downloading gopkg.in/yaml.v2 v2.3.0
go: downloading golang.org/x/sys v0.0.0-20200523222454-059865788121
go: extracting github.com/machine-drivers/docker-machine-driver-vmware v0.1.1
go: extracting golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a
go: downloading github.com/cenkalti/backoff v2.2.1+incompatible
go: downloading github.com/spf13/viper v1.7.0
go: extracting github.com/mitchellh/go-ps v0.0.0-20190716172923-621e5597135b
go: extracting github.com/spf13/viper v1.7.0
go: downloading github.com/pkg/profile v0.0.0-20161223203901-3a8809bd8a80
go: extracting golang.org/x/crypto v0.0.0-20200510223506-06a226fb4e37
go: extracting k8s.io/api v0.17.3
go: extracting gopkg.in/yaml.v2 v2.3.0
go: downloading github.com/spf13/jwalterweatherman v1.1.0
go: downloading github.com/spf13/cast v1.3.1
go: downloading github.com/subosito/gotenv v1.2.0
go: extracting github.com/cenkalti/backoff v2.2.1+incompatible
go: downloading github.com/mitchellh/mapstructure v1.3.1
go: extracting github.com/spf13/jwalterweatherman v1.1.0
go: extracting github.com/subosito/gotenv v1.2.0
go: downloading gopkg.in/ini.v1 v1.56.0
go: extracting github.com/pkg/profile v0.0.0-20161223203901-3a8809bd8a80
go: extracting github.com/spf13/cast v1.3.1
go: downloading github.com/pkg/errors v0.9.1
go: extracting github.com/mitchellh/mapstructure v1.3.1
go: downloading github.com/fsnotify/fsnotify v1.4.9
go: extracting golang.org/x/sys v0.0.0-20200523222454-059865788121
go: extracting github.com/pkg/errors v0.9.1
go: downloading github.com/docker/go-units v0.4.0
go: downloading sigs.k8s.io/sig-storage-lib-external-provisioner v4.0.0+incompatible
go: extracting github.com/fsnotify/fsnotify v1.4.9
go: extracting gopkg.in/ini.v1 v1.56.0
go: downloading github.com/hashicorp/hcl v1.0.0
go: downloading github.com/docker/docker v1.4.2-0.20190924003213-a8608b5b67c7
go: extracting sigs.k8s.io/sig-storage-lib-external-provisioner v4.0.0+incompatible
go: downloading github.com/docker/cli v0.0.0-20200303162255-7d407207c304
go: extracting github.com/docker/go-units v0.4.0
go: downloading github.com/gogo/protobuf v1.3.1
go: extracting github.com/hashicorp/hcl v1.0.0
go: downloading golang.org/x/time v0.0.0-20200416051211-89c76fbcd5d1
go: downloading github.com/juju/clock v0.0.0-20190205081909-9c5c9712527c
go: extracting golang.org/x/time v0.0.0-20200416051211-89c76fbcd5d1
go: extracting github.com/juju/clock v0.0.0-20190205081909-9c5c9712527c
go: downloading github.com/imdario/mergo v0.3.9
go: downloading github.com/pkg/browser v0.0.0-20160118053552-9302be274faa
go: extracting github.com/imdario/mergo v0.3.9
go: extracting github.com/pkg/browser v0.0.0-20160118053552-9302be274faa
go: downloading k8s.io/kubernetes v1.17.3
go: extracting github.com/docker/cli v0.0.0-20200303162255-7d407207c304
go: extracting github.com/docker/docker v1.4.2-0.20190924003213-a8608b5b67c7
go: extracting github.com/gogo/protobuf v1.3.1
go: downloading github.com/juju/mutex v0.0.0-20180619145857-d21b13acf4bf
go: downloading github.com/json-iterator/go v1.1.8
go: downloading github.com/cpuguy83/go-md2man v1.0.10
go: downloading github.com/spf13/afero v1.2.2
go: extracting github.com/cpuguy83/go-md2man v1.0.10
go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.0
go: extracting github.com/spf13/afero v1.2.2
go: extracting github.com/juju/mutex v0.0.0-20180619145857-d21b13acf4bf
go: downloading github.com/prometheus/client_golang v1.1.0
go: downloading k8s.io/utils v0.0.0-20200229041039-0a110f9eb7ab
go: extracting github.com/json-iterator/go v1.1.8
go: extracting github.com/cpuguy83/go-md2man/v2 v2.0.0
go: downloading github.com/pelletier/go-toml v1.8.0
go: downloading github.com/hooklift/iso9660 v0.0.0-20170318115843-1cf07e5970d8
go: downloading golang.org/x/text v0.3.2
go: downloading github.com/sayboras/dockerclient v0.0.0-20191231050035-015626177a97
go: extracting k8s.io/utils v0.0.0-20200229041039-0a110f9eb7ab
go: downloading github.com/golang-collections/collections v0.0.0-20130729185459-604e922904d3
go: extracting github.com/prometheus/client_golang v1.1.0
go: downloading github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
go: downloading github.com/containerd/containerd v1.3.1-0.20191213020239-082f7e3aed57
go: downloading github.com/davecgh/go-spew v1.1.1
go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: extracting github.com/hooklift/iso9660 v0.0.0-20170318115843-1cf07e5970d8
go: downloading golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2
go: extracting github.com/pelletier/go-toml v1.8.0
go: extracting github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: downloading github.com/mattn/go-isatty v0.0.12
go: downloading github.com/golang/protobuf v1.4.2
go: extracting github.com/sayboras/dockerclient v0.0.0-20191231050035-015626177a97
go: downloading k8s.io/kubectl v0.17.3
go: extracting github.com/mattn/go-isatty v0.0.12
go: downloading github.com/docker/go-connections v0.4.0
go: extracting github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
go: downloading github.com/olekukonko/tablewriter v0.0.4
go: extracting github.com/docker/go-connections v0.4.0
go: downloading github.com/docker/distribution v2.7.1+incompatible
go: extracting github.com/golang-collections/collections v0.0.0-20130729185459-604e922904d3
go: downloading github.com/juju/errors v0.0.0-20190806202954-0232dcc7464d
go: extracting github.com/davecgh/go-spew v1.1.1
go: downloading github.com/google/uuid v1.1.1
go: extracting github.com/containerd/containerd v1.3.1-0.20191213020239-082f7e3aed57
go: extracting github.com/google/uuid v1.1.1
go: downloading github.com/prometheus/common v0.6.0
go: extracting github.com/olekukonko/tablewriter v0.0.4
go: downloading github.com/magiconair/properties v1.8.1
go: extracting github.com/juju/errors v0.0.0-20190806202954-0232dcc7464d
go: downloading github.com/prometheus/procfs v0.0.5
go: extracting github.com/golang/protobuf v1.4.2
go: extracting golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2
go: downloading google.golang.org/grpc v1.29.1
go: extracting github.com/prometheus/common v0.6.0
go: extracting github.com/magiconair/properties v1.8.1
go: downloading github.com/sirupsen/logrus v1.6.0
go: extracting github.com/prometheus/procfs v0.0.5
go: downloading github.com/google/go-cmp v0.4.1
go: downloading k8s.io/klog v1.0.0
go: downloading github.com/beorn7/perks v1.0.1
go: extracting k8s.io/kubernetes v1.17.3
go: extracting k8s.io/kubectl v0.17.3
go: extracting github.com/docker/distribution v2.7.1+incompatible
go: extracting google.golang.org/grpc v1.29.1
go: extracting github.com/beorn7/perks v1.0.1
go: extracting github.com/google/go-cmp v0.4.1
go: downloading github.com/russross/blackfriday v1.5.3-0.20200218234912-41c5fccfd6f6
go: extracting golang.org/x/text v0.3.2
go: downloading google.golang.org/protobuf v1.24.0
go: extracting github.com/sirupsen/logrus v1.6.0
go: extracting k8s.io/klog v1.0.0
go: downloading github.com/spf13/pflag v1.0.5
go: downloading gopkg.in/inf.v0 v0.9.1
go: downloading golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d
go: extracting github.com/russross/blackfriday v1.5.3-0.20200218234912-41c5fccfd6f6
go: downloading github.com/russross/blackfriday/v2 v2.0.1
go: downloading sigs.k8s.io/yaml v1.1.0
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.1
go: downloading github.com/mattn/go-runewidth v0.0.7
go: downloading google.golang.org/genproto v0.0.0-20200527145253-8367513e4ece
go: downloading github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51
go: extracting gopkg.in/inf.v0 v0.9.1
go: extracting github.com/spf13/pflag v1.0.5
go: extracting google.golang.org/protobuf v1.24.0
go: downloading k8s.io/cluster-bootstrap v0.17.3
go: downloading github.com/miekg/dns v1.1.4
go: downloading github.com/docker/docker-credential-helpers v0.6.3
go: extracting sigs.k8s.io/yaml v1.1.0
go: extracting github.com/matttproud/golang_protobuf_extensions v1.0.1
go: extracting github.com/russross/blackfriday/v2 v2.0.1
go: extracting golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d
go: extracting github.com/mattn/go-runewidth v0.0.7
go: extracting github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51
go: downloading github.com/libvirt/libvirt-go v3.4.0+incompatible
go: downloading github.com/shurcooL/sanitized_anchor_name v1.0.0
go: downloading k8s.io/kube-proxy v0.17.3
go: downloading github.com/hashicorp/golang-lru v0.5.3
go: extracting google.golang.org/genproto v0.0.0-20200527145253-8367513e4ece
go: downloading github.com/MakeNowJust/heredoc v0.0.0-20170808103936-bb23615498cd
go: downloading k8s.io/kube-openapi v0.0.0-20191107075043-30be4d16710a
go: extracting github.com/docker/docker-credential-helpers v0.6.3
go: extracting k8s.io/cluster-bootstrap v0.17.3
go: downloading github.com/googleapis/gnostic v0.3.0
go: downloading github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e
go: downloading k8s.io/kubelet v0.17.3
go: extracting github.com/miekg/dns v1.1.4
go: downloading github.com/modern-go/reflect2 v1.0.1
go: extracting github.com/libvirt/libvirt-go v3.4.0+incompatible
go: extracting github.com/hashicorp/golang-lru v0.5.3
go: extracting github.com/shurcooL/sanitized_anchor_name v1.0.0
go: downloading github.com/opencontainers/image-spec v1.0.1
go: extracting k8s.io/kube-openapi v0.0.0-20191107075043-30be4d16710a
go: downloading google.golang.org/api v0.25.0
go: extracting github.com/MakeNowJust/heredoc v0.0.0-20170808103936-bb23615498cd
go: downloading cloud.google.com/go v0.57.0
go: extracting k8s.io/kube-proxy v0.17.3
go: downloading github.com/c4milo/gotoolkit v0.0.0-20170318115440-bcc06269efa9
go: downloading github.com/mitchellh/go-wordwrap v1.0.0
go: downloading k8s.io/component-base v0.17.3
go: extracting k8s.io/kubelet v0.17.3
go: extracting github.com/googleapis/gnostic v0.3.0
go: downloading github.com/afbjorklund/go-getter v1.4.1-0.20190910175809-eb9f6c26742c
go: extracting github.com/modern-go/reflect2 v1.0.1
go: extracting github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e
go: downloading github.com/google/gofuzz v1.0.0
go: downloading github.com/docker/spdystream v0.0.0-20160310174837-449fdfce4d96
go: extracting k8s.io/component-base v0.17.3
go: extracting github.com/mitchellh/go-wordwrap v1.0.0
go: extracting cloud.google.com/go v0.57.0
go: extracting github.com/opencontainers/image-spec v1.0.1
go: extracting github.com/c4milo/gotoolkit v0.0.0-20170318115440-bcc06269efa9
go: downloading github.com/cheggaaa/pb v1.0.27
go: downloading cloud.google.com/go/storage v1.8.0
go: extracting google.golang.org/api v0.25.0
go: extracting github.com/docker/spdystream v0.0.0-20160310174837-449fdfce4d96
go: extracting github.com/afbjorklund/go-getter v1.4.1-0.20190910175809-eb9f6c26742c
go: extracting github.com/google/gofuzz v1.0.0
go: downloading github.com/aws/aws-sdk-go v1.31.6
go: downloading github.com/ulikunitz/xz v0.5.7
go: downloading github.com/mitchellh/go-homedir v1.1.0
go: downloading github.com/hashicorp/go-cleanhttp v0.5.1
go: downloading github.com/hashicorp/go-safetemp v1.0.0
go: downloading github.com/hashicorp/go-version v1.2.0
go: extracting cloud.google.com/go/storage v1.8.0
go: extracting github.com/cheggaaa/pb v1.0.27
go: downloading github.com/cheggaaa/pb/v3 v3.0.1
go: downloading github.com/mitchellh/go-testing-interface v1.0.0
go: extracting github.com/ulikunitz/xz v0.5.7
go: extracting github.com/mitchellh/go-homedir v1.1.0
go: extracting github.com/hashicorp/go-cleanhttp v0.5.1
go: extracting github.com/hashicorp/go-safetemp v1.0.0
go: downloading github.com/bgentry/go-netrc v0.0.0-20140422174119-9fd32a8b3d3d
go: downloading github.com/googleapis/gax-go v2.0.2+incompatible
go: extracting github.com/hashicorp/go-version v1.2.0
go: downloading go.opencensus.io v0.22.3
go: extracting github.com/aws/aws-sdk-go v1.31.6
go: extracting github.com/cheggaaa/pb/v3 v3.0.1
go: downloading github.com/fatih/color v1.9.0
go: extracting github.com/mitchellh/go-testing-interface v1.0.0
go: downloading github.com/VividCortex/ewma v1.1.1
go: downloading github.com/mattn/go-colorable v0.1.6
go: downloading github.com/jmespath/go-jmespath v0.3.0
go: extracting github.com/bgentry/go-netrc v0.0.0-20140422174119-9fd32a8b3d3d
go: extracting github.com/googleapis/gax-go v2.0.2+incompatible
go: downloading github.com/googleapis/gax-go/v2 v2.0.5
go: extracting github.com/fatih/color v1.9.0
go: extracting github.com/VividCortex/ewma v1.1.1
go: extracting github.com/mattn/go-colorable v0.1.6
go: extracting go.opencensus.io v0.22.3
go: extracting github.com/jmespath/go-jmespath v0.3.0
go: extracting github.com/googleapis/gax-go/v2 v2.0.5
go: finding github.com/machine-drivers/machine v0.7.1-0.20200323212942-41eb826190d8
go: finding github.com/golang-collections/collections v0.0.0-20130729185459-604e922904d3
go: finding github.com/pkg/errors v0.9.1
go: finding github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: finding github.com/google/slowjam v0.0.0-20200530021616-df27e642fe7b
go: finding github.com/pkg/profile v0.0.0-20161223203901-3a8809bd8a80
go: finding github.com/docker/go-units v0.4.0
go: finding github.com/spf13/cobra v1.0.0
go: finding github.com/olekukonko/tablewriter v0.0.4
go: finding k8s.io/client-go v0.17.3
go: finding github.com/blang/semver v3.5.0+incompatible
go: finding github.com/spf13/viper v1.7.0
go: finding github.com/hooklift/iso9660 v0.0.0-20170318115843-1cf07e5970d8
go: finding github.com/libvirt/libvirt-go v3.4.0+incompatible
go: finding golang.org/x/crypto v0.0.0-20200510223506-06a226fb4e37
go: finding github.com/mattn/go-runewidth v0.0.7
go: finding github.com/imdario/mergo v0.3.9
go: finding github.com/juju/clock v0.0.0-20190205081909-9c5c9712527c
go: finding k8s.io/api v0.17.3
go: finding github.com/spf13/pflag v1.0.5
go: finding github.com/juju/mutex v0.0.0-20180619145857-d21b13acf4bf
go: finding k8s.io/apimachinery v0.17.3
go: finding github.com/c4milo/gotoolkit v0.0.0-20170318115440-bcc06269efa9
go: finding github.com/mattn/go-isatty v0.0.12
go: finding golang.org/x/sys v0.0.0-20200523222454-059865788121
go: finding github.com/juju/errors v0.0.0-20190806202954-0232dcc7464d
go: finding github.com/docker/docker v1.4.2-0.20190924003213-a8608b5b67c7
go: finding github.com/gogo/protobuf v1.3.1
go: finding github.com/afbjorklund/go-containerregistry v0.0.0-20200602203322-347d93793dc9
go: finding github.com/google/uuid v1.1.1
go: finding github.com/fsnotify/fsnotify v1.4.9
go: finding github.com/hashicorp/hcl v1.0.0
go: finding github.com/pkg/browser v0.0.0-20160118053552-9302be274faa
go: finding github.com/mitchellh/go-ps v0.0.0-20190716172923-621e5597135b
go: finding github.com/docker/cli v0.0.0-20200303162255-7d407207c304
go: finding github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51
go: finding github.com/magiconair/properties v1.8.1
go: finding github.com/shirou/gopsutil v2.18.12+incompatible
go: finding github.com/google/gofuzz v1.0.0
go: finding github.com/mitchellh/mapstructure v1.3.1
go: finding github.com/cloudfoundry-attic/jibber_jabber v0.0.0-20151120183258-bcc4c8345a21
go: finding cloud.google.com/go/storage v1.8.0
go: finding sigs.k8s.io/sig-storage-lib-external-provisioner v4.0.0+incompatible
go: finding k8s.io/klog v1.0.0
go: finding github.com/pelletier/go-toml v1.8.0
go: finding github.com/machine-drivers/docker-machine-driver-vmware v0.1.1
go: finding github.com/otiai10/copy v1.0.2
go: finding github.com/cpuguy83/go-md2man/v2 v2.0.0
go: finding github.com/spf13/afero v1.2.2
go: finding github.com/golang/protobuf v1.4.2
go: finding golang.org/x/text v0.3.2
go: finding cloud.google.com/go v0.57.0
go: finding gopkg.in/inf.v0 v0.9.1
go: finding github.com/docker/docker-credential-helpers v0.6.3
go: finding github.com/cenkalti/backoff v2.2.1+incompatible
go: finding golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a
go: finding github.com/spf13/cast v1.3.1
go: finding github.com/googleapis/gnostic v0.3.0
go: finding github.com/docker/distribution v2.7.1+incompatible
go: finding github.com/russross/blackfriday/v2 v2.0.1
go: finding golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2
go: finding github.com/spf13/jwalterweatherman v1.1.0
go: finding github.com/prometheus/client_golang v1.1.0
go: finding gopkg.in/yaml.v2 v2.3.0
go: finding k8s.io/kubernetes v1.17.3
go: finding github.com/googleapis/gax-go/v2 v2.0.5
go: finding github.com/intel-go/cpuid v0.0.0-20181003105527-1a4a6f06a1c6
go: finding golang.org/x/time v0.0.0-20200416051211-89c76fbcd5d1
go: finding github.com/subosito/gotenv v1.2.0
go: finding github.com/cheggaaa/pb/v3 v3.0.1
go: finding google.golang.org/genproto v0.0.0-20200527145253-8367513e4ece
go: finding github.com/shurcooL/sanitized_anchor_name v1.0.0
go: finding gopkg.in/ini.v1 v1.56.0
go: finding google.golang.org/protobuf v1.24.0
go: finding google.golang.org/api v0.25.0
go: finding google.golang.org/grpc v1.29.1
go: finding github.com/opencontainers/go-digest v1.0.0-rc1
go: finding go.opencensus.io v0.22.3
go: finding github.com/afbjorklund/go-getter v1.4.1-0.20190910175809-eb9f6c26742c
go: finding k8s.io/component-base v0.17.3
go: finding github.com/json-iterator/go v1.1.8
go: finding k8s.io/kubectl v0.17.3
go: finding github.com/VividCortex/ewma v1.1.1
go: finding github.com/davecgh/go-spew v1.1.1
go: finding github.com/modern-go/reflect2 v1.0.1
go: finding github.com/docker/go-connections v0.4.0
go: finding github.com/beorn7/perks v1.0.1
go: finding github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: finding github.com/fatih/color v1.9.0
go: finding github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
go: finding github.com/mattn/go-colorable v0.1.6
go: finding github.com/MakeNowJust/heredoc v0.0.0-20170808103936-bb23615498cd
go: finding k8s.io/cluster-bootstrap v0.17.3
go: finding github.com/prometheus/common v0.6.0
go: finding github.com/opencontainers/image-spec v1.0.1
go: finding github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e
go: finding github.com/russross/blackfriday v1.5.3-0.20200218234912-41c5fccfd6f6
go: finding github.com/sayboras/dockerclient v0.0.0-20191231050035-015626177a97
go: finding golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d
go: finding k8s.io/utils v0.0.0-20200229041039-0a110f9eb7ab
go: finding github.com/containerd/containerd v1.3.1-0.20191213020239-082f7e3aed57
go: finding github.com/prometheus/procfs v0.0.5
go: finding github.com/sirupsen/logrus v1.6.0
go: finding sigs.k8s.io/yaml v1.1.0
go: finding github.com/matttproud/golang_protobuf_extensions v1.0.1
go: finding github.com/mitchellh/go-wordwrap v1.0.0
go: finding k8s.io/kube-proxy v0.17.3
go: finding k8s.io/kubelet v0.17.3
go: finding github.com/aws/aws-sdk-go v1.31.6
go: finding github.com/bgentry/go-netrc v0.0.0-20140422174119-9fd32a8b3d3d
go: finding github.com/miekg/dns v1.1.4
go: finding github.com/hashicorp/go-cleanhttp v0.5.1
go: finding github.com/hashicorp/golang-lru v0.5.3
go: finding github.com/hashicorp/go-safetemp v1.0.0
go: finding github.com/hashicorp/go-version v1.2.0
go: finding github.com/mitchellh/go-homedir v1.1.0
go: finding github.com/mitchellh/go-testing-interface v1.0.0
go: finding github.com/google/go-cmp v0.4.1
go: finding github.com/ulikunitz/xz v0.5.7
go: finding k8s.io/kube-openapi v0.0.0-20191107075043-30be4d16710a
go: finding github.com/docker/spdystream v0.0.0-20160310174837-449fdfce4d96
go: finding github.com/jmespath/go-jmespath v0.3.0
go: finding github.com/phayes/freeport v0.0.0-20180830031419-95f893ade6f2
go: downloading github.com/evanphx/json-patch v4.5.0+incompatible
go: downloading github.com/pmezard/go-difflib v1.0.0
go: downloading golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543
go: extracting github.com/pmezard/go-difflib v1.0.0
go: extracting golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543
go: extracting github.com/evanphx/json-patch v4.5.0+incompatible
go: finding github.com/pmezard/go-difflib v1.0.0
go: finding golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543
go: finding github.com/evanphx/json-patch v4.5.0+incompatible
ok  	k8s.io/minikube/cmd/minikube/cmd	4.237s	coverage: 20.4% of statements
ok  	k8s.io/minikube/cmd/minikube/cmd/config	0.166s	coverage: 21.5% of statements
E0707 22:18:41.681191   12145 out.go:102] [unset errFile]: * 
E0707 22:18:41.681340   12145 out.go:102] [unset errFile]: X Error getting primary control plane: remove /tmp/profile387851723/.minikube/profiles/config.json: no such file or directory
E0707 22:18:41.681351   12145 out.go:102] [unset errFile]: * 
E0707 22:18:41.681362   12145 out.go:102] [unset errFile]: * minikube is exiting due to an error. If the above message is not useful, open an issue:
E0707 22:18:41.681372   12145 out.go:102] [unset errFile]:   - https://github.com/kubernetes/minikube/issues/new/choose
FAIL	k8s.io/minikube/pkg/addons	0.050s
ok  	k8s.io/minikube/pkg/drivers	0.010s	coverage: 19.6% of statements
ok  	k8s.io/minikube/pkg/drivers/hyperkit	0.018s	coverage: 77.3% of statements
ok  	k8s.io/minikube/pkg/drivers/kic/oci	0.008s	coverage: 0.0% of statements
ok  	k8s.io/minikube/pkg/drivers/kvm	0.028s	coverage: 2.3% of statements
ok  	k8s.io/minikube/pkg/minikube/bootstrapper	0.711s	coverage: 55.7% of statements
ok  	k8s.io/minikube/pkg/minikube/bootstrapper/bsutil	0.095s	coverage: 61.9% of statements
ok  	k8s.io/minikube/pkg/minikube/bootstrapper/bsutil/ktmpl	0.010s	coverage: 100.0% of statements
ok  	k8s.io/minikube/pkg/minikube/bootstrapper/images	0.006s	coverage: 84.2% of statements
ok  	k8s.io/minikube/pkg/minikube/cluster	0.091s	coverage: 16.7% of statements
ok  	k8s.io/minikube/pkg/minikube/command	0.030s	coverage: 4.3% of statements
ok  	k8s.io/minikube/pkg/minikube/config	0.188s	coverage: 70.1% of statements
ok  	k8s.io/minikube/pkg/minikube/cruntime	0.050s	coverage: 40.1% of statements
ok  	k8s.io/minikube/pkg/minikube/docker	0.028s	coverage: 20.8% of statements
ok  	k8s.io/minikube/pkg/minikube/driver	0.020s	coverage: 37.0% of statements
ok  	k8s.io/minikube/pkg/minikube/extract	0.006s	coverage: 58.8% of statements
ok  	k8s.io/minikube/pkg/minikube/image	0.021s	coverage: 3.0% of statements
ok  	k8s.io/minikube/pkg/minikube/kubeconfig	0.027s	coverage: 77.0% of statements
ok  	k8s.io/minikube/pkg/minikube/localpath	0.008s	coverage: 51.9% of statements
ok  	k8s.io/minikube/pkg/minikube/logs	0.083s	coverage: 1.3% of statements
ok  	k8s.io/minikube/pkg/minikube/machine	1.913s	coverage: 37.8% of statements
ok  	k8s.io/minikube/pkg/minikube/notify	0.013s	coverage: 92.9% of statements
ok  	k8s.io/minikube/pkg/minikube/out	0.041s	coverage: 67.4% of statements
ok  	k8s.io/minikube/pkg/minikube/perf	4.287s	coverage: 33.8% of statements
ok  	k8s.io/minikube/pkg/minikube/problem	0.011s	coverage: 91.7% of statements
ok  	k8s.io/minikube/pkg/minikube/proxy	0.007s	coverage: 77.0% of statements
ok  	k8s.io/minikube/pkg/minikube/registry	0.007s	coverage: 75.5% of statements
ok  	k8s.io/minikube/pkg/minikube/service	0.074s	coverage: 84.2% of statements
ok  	k8s.io/minikube/pkg/minikube/shell	0.003s	coverage: 97.1% of statements
ok  	k8s.io/minikube/pkg/minikube/storageclass	0.019s	coverage: 100.0% of statements
ok  	k8s.io/minikube/pkg/minikube/sysinit	0.022s	coverage: 6.5% of statements
ok  	k8s.io/minikube/pkg/minikube/translate	0.004s	coverage: 10.5% of statements
ok  	k8s.io/minikube/pkg/minikube/tunnel	1.636s	coverage: 64.2% of statements
ok  	k8s.io/minikube/pkg/util	0.699s	coverage: 77.9% of statements
ok  	k8s.io/minikube/pkg/util/lock	0.007s	coverage: 18.2% of statements
ok  	k8s.io/minikube/pkg/util/retry	0.008s	coverage: 0.0% of statements
FAIL
Makefile:274: recipe for target 'test' failed
make: *** [test] Error 32

TravisBuddy Request Identifier: 01290fb0-c0a0-11ea-8f0c-c508e01afe59

…d none clusters

minikube-pr-bot · 2020-07-07T22:24:52Z

kvm2 Driver
docker Driver

tstromberg · 2020-07-07T22:25:50Z

/ok-to-test

medyagh · 2020-07-08T02:10:02Z

/retest-this-please

minikube-pr-bot · 2020-07-08T21:07:33Z

kvm2 Driver
Times for minikube: [59.325464738 62.086815537 63.06711632200001]
Average time for minikube: 61.493132199

Times for Minikube (PR 8664): [58.06284744500001 61.813930489000015 60.685883151999995]
Average time for Minikube (PR 8664): 60.18755369533334

Averages Time Per Log

+--------------------------------+-----------+--------------------+
|              LOG               | MINIKUBE  | MINIKUBE (PR 8664) |
+--------------------------------+-----------+--------------------+
| * minikube v1.12.0-beta.1 on   |  0.055598 |           0.058280 |
| Debian 9.11                    |           |                    |
| * Using the kvm2 driver based  |  0.018688 |           0.019943 |
| on existing profile            |           |                    |
| * Starting control plane node  |  0.006094 |           0.003181 |
| minikube in cluster minikube   |           |                    |
| * Creating kvm2 VM (CPUs=2,    | 38.046585 |          38.688103 |
| Memory=3700MB, Disk=20000MB)   |           |                    |
| ...                            |           |                    |
| * Preparing Kubernetes v1.18.3 | 21.136666 |          19.781112 |
| on Docker 19.03.8 ...          |           |                    |
| * Verifying Kubernetes         |  1.638083 |           1.251155 |
| components...                  |           |                    |
| * Enabled addons:              |  0.521392 |           0.312370 |
| default-storageclass,          |           |                    |
| storage-provisioner            |           |                    |
| * Done! kubectl is now         |  0.065064 |           0.069620 |
| configured to use "minikube"   |           |                    |
|                                |  0.004962 |           0.003788 |
+--------------------------------+-----------+--------------------+

docker Driver
Times for minikube: [26.520107989000003 26.360925300999998 24.95626601]
Average time for minikube: 25.945766433333336

Times for Minikube (PR 8664): [24.757110355000002 25.607405939000003 39.30365392]
Average time for Minikube (PR 8664): 29.889390071333334

Averages Time Per Log

+--------------------------------+-----------+--------------------+
|              LOG               | MINIKUBE  | MINIKUBE (PR 8664) |
+--------------------------------+-----------+--------------------+
| * minikube v1.12.0-beta.1 on   |  0.070341 |           0.077418 |
| Debian 9.11                    |           |                    |
| * Using the docker driver      |  0.002354 |           0.004112 |
| based on existing profile      |           |                    |
| * Starting control plane node  |  0.054733 |           0.055894 |
| minikube in cluster minikube   |           |                    |
| * Creating docker container    |  7.654556 |           9.146623 |
| (CPUs=2, Memory=3700MB) ...    |           |                    |
| * Preparing Kubernetes v1.18.3 | 16.894044 |          17.153939 |
| on Docker 19.03.2 ...          |           |                    |
| * Verifying Kubernetes         |  1.197819 |           0.828337 |
| components...                  |           |                    |
| * Enabled addons:              |  0.002900 |           0.073990 |
| default-storageclass,          |           |                    |
| storage-provisioner            |           |                    |
| * Done! kubectl is now         |  0.064884 |           0.067780 |
| configured to use "minikube"   |           |                    |
|                                |  0.004135 |           6.324131 |
+--------------------------------+-----------+--------------------+

tstromberg added 2 commits July 7, 2020 15:10

Reduce apiserver health timeout from 8s to 5s

f82d5c6

retry.Local should respect the maxTime argument

53ce804

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jul 7, 2020

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jul 7, 2020

k8s-ci-robot requested review from josedonizetti and prasadkatti July 7, 2020 22:14

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 7, 2020

Don't propagate stopped as an error, so that 'delete' works on stoppe…

7df4075

…d none clusters

tstromberg changed the title ~~Fix infinite retry in APIServerStatus / retry.Local~~ Fix infinite retry in APIServerStatus / retry.Local - fixes "none" driver deletion timeout Jul 7, 2020

tstromberg changed the title ~~Fix infinite retry in APIServerStatus / retry.Local - fixes "none" driver deletion timeout~~ none: Fix 'minikube delete' issues when the apiserver is down Jul 7, 2020

Merge branch 'master' into none-timeout

f8068d0

medyagh approved these changes Jul 9, 2020

View reviewed changes

tstromberg merged commit ca9040b into kubernetes:master Jul 9, 2020

tstromberg mentioned this pull request Jul 9, 2020

none driver: setting CNI times out the tests #8601

Closed

fmotrifork mentioned this pull request Jul 10, 2020

minikube 1.12.0 fishworks/fish-food#837

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

none: Fix 'minikube delete' issues when the apiserver is down #8664

none: Fix 'minikube delete' issues when the apiserver is down #8664

tstromberg commented Jul 7, 2020 •

edited

Loading

tstromberg commented Jul 7, 2020

k8s-ci-robot commented Jul 7, 2020

TravisBuddy commented Jul 7, 2020

minikube-pr-bot commented Jul 7, 2020

tstromberg commented Jul 7, 2020

medyagh commented Jul 8, 2020

minikube-pr-bot commented Jul 8, 2020

none: Fix 'minikube delete' issues when the apiserver is down #8664

none: Fix 'minikube delete' issues when the apiserver is down #8664

Conversation

tstromberg commented Jul 7, 2020 • edited Loading

tstromberg commented Jul 7, 2020

k8s-ci-robot commented Jul 7, 2020

TravisBuddy commented Jul 7, 2020

Travis tests have failed

1st Build

TravisBuddy Request Identifier: 01290fb0-c0a0-11ea-8f0c-c508e01afe59

minikube-pr-bot commented Jul 7, 2020

tstromberg commented Jul 7, 2020

medyagh commented Jul 8, 2020

minikube-pr-bot commented Jul 8, 2020

tstromberg commented Jul 7, 2020 •

edited

Loading