Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add log for cert timeout #2477

Merged
merged 3 commits into from
Feb 4, 2020
Merged

Add log for cert timeout #2477

merged 3 commits into from
Feb 4, 2020

Conversation

anyasabo
Copy link
Contributor

@anyasabo anyasabo commented Jan 28, 2020

On a newly deployed 1.0.0 operator, I noticed that there's no message indicating why the pod exited if it takes too long for the webhook cert to be present. This PR corrects that so it is easier to understand why the operator exited. It may also be worth increasing this timeout since it happened on a new GKE cluster with nothing else running in it.

Example full log:

$ kubectl logs -n elastic-system elastic-operator-0 -p
{"level":"info","@timestamp":"2020-01-28T21:38:19.510Z","logger":"manager","message":"Setting up client for manager","ver":"1.0.0-6881438d"}
{"level":"info","@timestamp":"2020-01-28T21:38:19.510Z","logger":"manager","message":"Setting up scheme","ver":"1.0.0-6881438d"}
{"level":"info","@timestamp":"2020-01-28T21:38:19.514Z","logger":"manager","message":"Setting up manager","ver":"1.0.0-6881438d"}
{"level":"info","@timestamp":"2020-01-28T21:38:19.514Z","logger":"manager","message":"Operator configured to manage all namespaces","ver":"1.0.0-6881438d"}
{"level":"info","@timestamp":"2020-01-28T21:38:20.270Z","logger":"controller-runtime.metrics","message":"metrics server is starting to listen","ver":"1.0.0-6881438d","addr":":0"}
{"level":"info","@timestamp":"2020-01-28T21:38:20.303Z","logger":"manager","message":"Setting up controllers","ver":"1.0.0-6881438d","roles":["all"]}
{"level":"info","@timestamp":"2020-01-28T21:38:20.303Z","logger":"manager","message":"Automatic management of the webhook certificates enabled","ver":"1.0.0-6881438d"}
{"level":"info","@timestamp":"2020-01-28T21:38:20.312Z","logger":"webhook-certificates-controller","message":"Creating new webhook certificates","ver":"1.0.0-6881438d","webhook":"elastic-webhook.k8s.elastic.co","secret_namespace":"elastic-system","secret_name":"elastic-webhook-server-cert"}
{"level":"info","@timestamp":"2020-01-28T21:38:21.034Z","logger":"controller-runtime.builder","message":"skip registering a mutating webhook, admission.Defaulter interface is not implemented","ver":"1.0.0-6881438d","GVK":"elasticsearch.k8s.elastic.co/v1, Kind=Elasticsearch"}
{"level":"info","@timestamp":"2020-01-28T21:38:21.034Z","logger":"controller-runtime.builder","message":"Registering a validating webhook","ver":"1.0.0-6881438d","GVK":"elasticsearch.k8s.elastic.co/v1, Kind=Elasticsearch","path":"/validate-elasticsearch-k8s-elastic-co-v1-elasticsearch"}
{"level":"info","@timestamp":"2020-01-28T21:38:21.034Z","logger":"controller-runtime.webhook","message":"registering webhook","ver":"1.0.0-6881438d","path":"/validate-elasticsearch-k8s-elastic-co-v1-elasticsearch"}
{"level":"info","@timestamp":"2020-01-28T21:38:21.034Z","logger":"controller-runtime.builder","message":"skip registering a mutating webhook, admission.Defaulter interface is not implemented","ver":"1.0.0-6881438d","GVK":"elasticsearch.k8s.elastic.co/v1beta1, Kind=Elasticsearch"}
{"level":"info","@timestamp":"2020-01-28T21:38:21.034Z","logger":"controller-runtime.builder","message":"Registering a validating webhook","ver":"1.0.0-6881438d","GVK":"elasticsearch.k8s.elastic.co/v1beta1, Kind=Elasticsearch","path":"/validate-elasticsearch-k8s-elastic-co-v1beta1-elasticsearch"}
{"level":"info","@timestamp":"2020-01-28T21:38:21.034Z","logger":"controller-runtime.webhook","message":"registering webhook","ver":"1.0.0-6881438d","path":"/validate-elasticsearch-k8s-elastic-co-v1beta1-elasticsearch"}
{"level":"info","@timestamp":"2020-01-28T21:38:21.034Z","logger":"manager","message":"Polling for the webhook certificate to be available","ver":"1.0.0-6881438d","path":"/tmp/k8s-webhook-server/serving-certs/tls.crt"}

@anyasabo anyasabo added >enhancement Enhancement of existing functionality v1.0.1 labels Jan 28, 2020
cmd/manager/main.go Outdated Show resolved Hide resolved
@@ -426,6 +426,7 @@ func setupWebhook(mgr manager.Manager, certRotation certificates.RotationParams,
})

if err != nil {
log.Error(err, "Timeout elapsed waiting for webhook certificate to be available", "path", keyPath, "timeoutSeconds", timeout.Seconds())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

timeout_seconds to match the way we handle the other logs?

@anyasabo anyasabo merged commit 75ec43b into elastic:master Feb 4, 2020
anyasabo added a commit to anyasabo/cloud-on-k8s that referenced this pull request Feb 4, 2020
barkbay pushed a commit that referenced this pull request Feb 4, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>enhancement Enhancement of existing functionality v1.0.1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants