Webhook certificate reconciliation relies on deprecated v1beta1 API

[pebrc]

When reconciling the certificates for the webhook client we rely on the deprecated v1beta1 version of the ValidatingWebhookConfiguration API. This API will be unavailable in v1.22

Moving to v1 would break compatibility with older versions of Kubernetes. As of ECK version 1.6. we only support K8s 1.16+ but we also need to support OpenShift 3.11.

We could:

• just use v1 and document the webhook is not supported on older versions.
• call both versions for as long as we have the 1.21 API available (controller-runtime 0.9.0 moves to 1.21)

[barkbay]

I just hit this one while testing #4489 on K8S v1.22.0-alpha.3, it prevents the operator from starting:

{"log.level":"error","@timestamp":"2021-06-09T13:43:11.286Z","log.logger":"manager","message":"unable to setup and fill the webhook certificates","service.version":"1.7.0-SNAPSHOT+76d9ee07","service.type":"eck","ecs.version":"1.4.0","error":"the server could not find the requested resource","error.stack_trace":"github.com/elastic/cloud-on-k8s/cmd/manager.startOperator\n\t/go/src/github.com/elastic/cloud-on-k8s/cmd/manager/main.go:558\ngh.neting.cc/elastic/cloud-on-k8s/cmd/manager.doRun.func2\n\t/go/src/github.com/elastic/cloud-on-k8s/cmd/manager/main.go:328"}

Admission controller is not enabled by default on OCP 3.11. But given that 1.7.0 is supposed to support that version of OpenShift I would be tempted to say that we should still support v1beta1 here (not a super strong opinion knowing that 1.7.0 will be the last version to support OCP 3.11 😕 )

A few ideas to check that v1 is available (or not):

• Use the discovery API to detect the preferred version:

kubectl get --raw /apis/admissionregistration.k8s.io | jq

{
  "kind": "APIGroup",
  "apiVersion": "v1",
  "name": "admissionregistration.k8s.io",
  "versions": [
    {
      "groupVersion": "admissionregistration.k8s.io/v1",
      "version": "v1"
    },
    {
      "groupVersion": "admissionregistration.k8s.io/v1beta1",
      "version": "v1beta1"
    }
  ],
  "preferredVersion": {
    "groupVersion": "admissionregistration.k8s.io/v1",
    "version": "v1"
  }
}

^ I think we should be able to achieve something similar using clientset.Discovery()

• Poll the API server to check if "admissionregistration.k8s.io/v1" is available:

clientset.Discovery().ServerResourcesForGroupVersion("admissionregistration.k8s.io/v1")`

It should return 404 if not.

• Retry on v1beta1 if we get a 404 from clientset.AdmissionregistrationV1().ValidatingWebhookConfigurations().Get(...), but we wouldn't know if we get a 404 because the webhook has not been created or because admissionregistration.k8s.io/v1 is not available.