Allow e2e tests to use CI builds of control plane images

[pipejakob]

Is this a BUG REPORT or FEATURE REQUEST?

FEATURE REQUEST (of e2es)

When our e2e tests are run, they pass a parameter to kubeadm init to specify the release of Kubernetes to use for the control plane images (i.e. --kubernetes-version latest). This means that there can be lag between a bug being merged to kubernetes/master and exercising of that code in a kubeadm e2e, since our e2e infrastructure can only use images built as part of releases (even if those are alpha/beta/rc releases).

It would be great if we could create CI builds of these control plane images and use them as part of our e2e tests to close the gap between regression and detection of non-kubeadm code.

[luxas]

See also: kubernetes/test-infra#1400 #101

Maybe you @pipejakob could take ownership over kubernetes/test-infra#1400?
That would be great and seems to be in line with the v1.7 goals

[luxas]

@mikedanese @pipejakob @ixdy Did the pushing of master images to gcr.io ever happen or is it yet to be done?

[ixdy]

still tbd.

[luxas]

Work being done now in: kubernetes/test-infra#1400

[luxas]

Sent a PR for this kubernetes/test-infra#4176

[luxas]

This is now fixed; relevant extract from latest CI e2e runs:

I0825 10:15:05.498] [BeforeEach] [sig-cli] Kubectl client
I0825 10:15:05.498]   /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:141
I0825 10:15:05.498] �[1mSTEP�[0m: Creating a kubernetes client
I0825 10:15:05.498] Aug 25 10:15:05.495: INFO: >>> kubeConfig: /workspace/kubernetes-anywhere/phase1/gce/.tmp/kubeconfig.json
I0825 10:15:05.501] �[1mSTEP�[0m: Building a namespace api object
I0825 10:15:05.536] �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace
I0825 10:15:05.538] [BeforeEach] [sig-cli] Kubectl client
I0825 10:15:05.538]   /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubectl/kubectl.go:259
I0825 10:15:05.539] [It] should check is all data is printed [Conformance]
I0825 10:15:05.539]   /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubectl/kubectl.go:1132
I0825 10:15:05.539] Aug 25 10:15:05.538: INFO: Running '/workspace/k8s.io/kubernetes/kubernetes/platforms/linux/amd64/kubectl --server=https://35.202.34.84:443 --kubeconfig=/workspace/kubernetes-anywhere/phase1/gce/.tmp/kubeconfig.json version'
I0825 10:15:05.667] Aug 25 10:15:05.666: INFO: stderr: ""
I0825 10:15:05.667] Aug 25 10:15:05.666: INFO: stdout: "Client Version: version.Info{Major:\"1\", Minor:\"8+\", GitVersion:\"v1.8.0-alpha.3.152+41aeccd088622d\", GitCommit:\"41aeccd088622d00b26ed5d93eae981ed5403a5f\", GitTreeState:\"clean\", BuildDate:\"2017-08-25T08:19:59Z\", GoVersion:\"go1.8.3\", Compiler:\"gc\", Platform:\"linux/amd64\"}\nServer Version: version.Info{Major:\"1\", Minor:\"8+\", GitVersion:\"v1.8.0-alpha.3.141+0cf928f6813741\", GitCommit:\"0cf928f6813741d9699786ee61ec014c90119e2c\", GitTreeState:\"clean\", BuildDate:\"2017-08-25T06:25:15Z\", GoVersion:\"go1.8.3\", Compiler:\"gc\", Platform:\"linux/amd64\"}\n"
I0825 10:15:05.667] [AfterEach] [sig-cli] Kubectl client
I0825 10:15:05.667]   /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:142
I0825 10:15:05.668] Aug 25 10:15:05.667: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
I0825 10:15:05.671] �[1mSTEP�[0m: Destroying namespace "e2e-tests-kubectl-vfhjh" for this suite.
I0825 10:15:11.758] Aug 25 10:15:11.758: INFO: namespace: e2e-tests-kubectl-vfhjh, resource: bindings, ignored listing per whitelist
I0825 10:15:11.800] Aug 25 10:15:11.800: INFO: namespace e2e-tests-kubectl-vfhjh deletion completed in 6.129059987s
I0825 10:15:11.801] 

vs earlier

I0822 21:06:42.449] [BeforeEach] [sig-cli] Kubectl client
I0822 21:06:42.449]   /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:141
I0822 21:06:42.449] �[1mSTEP�[0m: Creating a kubernetes client
I0822 21:06:42.449] Aug 22 21:06:42.446: INFO: >>> kubeConfig: /workspace/kubernetes-anywhere/phase1/gce/.tmp/kubeconfig.json
I0822 21:06:42.451] �[1mSTEP�[0m: Building a namespace api object
I0822 21:06:42.489] �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace
I0822 21:06:42.490] [BeforeEach] [sig-cli] Kubectl client
I0822 21:06:42.491]   /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubectl/kubectl.go:259
I0822 21:06:42.491] [It] should check is all data is printed [Conformance]
I0822 21:06:42.491]   /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubectl/kubectl.go:1130
I0822 21:06:42.491] Aug 22 21:06:42.490: INFO: Running '/workspace/k8s.io/kubernetes/kubernetes/platforms/linux/amd64/kubectl --server=https://146.148.68.16:443 --kubeconfig=/workspace/kubernetes-anywhere/phase1/gce/.tmp/kubeconfig.json version'
I0822 21:06:42.610] Aug 22 21:06:42.610: INFO: stderr: ""
I0822 21:06:42.610] Aug 22 21:06:42.610: INFO: stdout: "Client Version: version.Info{Major:\"1\", Minor:\"8+\", GitVersion:\"v1.8.0-alpha.2.2266+6858bb4e602521\", GitCommit:\"6858bb4e6025218b2f4013173dc027709819bc8c\", GitTreeState:\"clean\", BuildDate:\"2017-08-22T19:35:21Z\", GoVersion:\"go1.8.3\", Compiler:\"gc\", Platform:\"linux/amd64\"}\nServer Version: version.Info{Major:\"1\", Minor:\"8+\", GitVersion:\"v1.8.0-alpha.2\", GitCommit:\"8e5584fe95c01f2b3a9d60fcccef8fadbb4c8f88\", GitTreeState:\"clean\", BuildDate:\"2017-07-12T21:08:54Z\", GoVersion:\"go1.8.3\", Compiler:\"gc\", Platform:\"linux/amd64\"}\n"
I0822 21:06:42.611] [AfterEach] [sig-cli] Kubectl client
I0822 21:06:42.611]   /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:142
I0822 21:06:42.611] Aug 22 21:06:42.610: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
I0822 21:06:42.616] �[1mSTEP�[0m: Destroying namespace "e2e-tests-kubectl-gdtnn" for this suite.
I0822 21:06:48.709] Aug 22 21:06:48.708: INFO: namespace: e2e-tests-kubectl-gdtnn, resource: bindings, ignored listing per whitelist
I0822 21:06:48.716] Aug 22 21:06:48.716: INFO: namespace e2e-tests-kubectl-gdtnn deletion completed in 6.099842715s
I0822 21:06:48.716] 

Now we can detect regressions at HEAD much faster, yay 🎉