Setting kopf.EventsConfig.events_loglevel doesn't prevent Type=Normal Events to be posted

[kopf-archiver]

An issue by thilp at 2019-09-18 09:42:52+00:00
Original URL: zalando-incubator/kopf#188
 

Kopf is creating way more Event objects than my Kubernetes cluster's administrators would like, potentially leading to etcd performance problems.

This is slightly related to #117, as merging similar events would probably help. But in my case, I simply would like to not have any Events of Type=Normal published. The Kopf documentation (Configure logging events) says that is possible, but it seems that following that documentation doesn't change anything in how Kopf behaves.

Expected Behavior

With kopf.EventsConfig.events_loglevel == kopf.config.LOGLEVEL_WARNING (documented here), I understand that no Type=Normal events should be published on Kubernetes resources watched by Kopf cause handlers.

With the steps provided below, I would expect the Events section of kubectl describe pod to contain only:

Warning  Logging    119s  kopf     warn 30

Actual Behavior

With the steps provided below, I observe that the Events section of kubectl describe pod contains:

Warning  Logging    119s  kopf     warn 30
Normal   Logging    119s  kopf     Handler 'handle' succeeded.
Normal   Logging    119s  kopf     info 30
Normal   Logging    119s  kopf     All handlers succeeded for creation.

Steps to Reproduce the Problem

Using kind to get a playground Kubernetes cluster:

1. bash$ kind create cluster --name demo

2. bash$ export KUBECONFIG="$(kind get kubeconfig-path --name="demo")"

3. bash$ docker build -t demo-image . (using the Dockerfile included below, along with handlers.py)

4. bash$ kind load docker-image --name demo demo-image

5. bash$ kubectl apply -f operator-deployment.yaml

6. bash$ kubectl describe pod

7. Observe in the Events section:

    Warning  Logging    119s  kopf     warn 30
    Normal   Logging    119s  kopf     Handler 'handle' succeeded.
    Normal   Logging    119s  kopf     info 30
    Normal   Logging    119s  kopf     All handlers succeeded for creation.
   

Dockerfile

FROM python:3.7-slim

RUN pip install kopf

ENV PYTHONPATH=/src
ADD handlers.py ${PYTHONPATH}/handlers.py

ENTRYPOINT ["kopf", "run", "-m", "handlers", "--standalone"]

handlers.py

import kopf

kopf.EventsConfig.events_loglevel = kopf.config.LOGLEVEL_WARNING

@kopf.on.create("", "v1", "pods")
@kopf.on.resume("", "v1", "pods")
@kopf.on.delete("", "v1", "pods")
async def handle(logger, **_):
    logger.info("info %s", kopf.EventsConfig.events_loglevel)
    logger.warn("warn %s", kopf.EventsConfig.events_loglevel)

operator-deployment.yaml

---
apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: "default"
  name: demo-account
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: demo-role-cluster
rules:

  # Framework: posting the events about the handlers progress/errors.
  - apiGroups: [events.k8s.io, ""]
    resources: [events]
    verbs: [create]

  - apiGroups: [""]
    resources: [pods]
    verbs: [list, watch, get, patch]

  - nonResourceURLs: ["/"]
    verbs: ["get"]

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  namespace: "default"
  name: demo-role-namespaced
rules:

  # Framework: posting the events about the handlers progress/errors.
  - apiGroups: [events.k8s.io, ""]
    resources: [events]
    verbs: [create]

  - apiGroups: [""]
    resources: [pods]
    verbs: [list, watch, get, patch]

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: demo-rolebinding-cluster
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: demo-role-cluster
subjects:
  - kind: ServiceAccount
    name: demo-account
    namespace: "default"
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  namespace: "default"
  name: demo-rolebinding-namespaced
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: demo-role-namespaced
subjects:
  - kind: ServiceAccount
    name: demo-account
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo
  labels:
    application: demo
spec:
  replicas: 1
  selector:
    matchLabels:
      application: demo
  template:
    metadata:
      labels:
        application: demo
    spec:
      serviceAccountName: demo-account
      containers:
        - name: demo-container
          image: demo-image
          imagePullPolicy: IfNotPresent  # For `kind load docker-image`.

Specifications

• Platform: Zalando's kubernetes-on-aws, Kubernetes-in-Docker
• Kubernetes version: v1.14.2
• Python version: Python 3.7.4
• Python packages installed:

aiohttp==3.6.0
aiojobs==0.2.2
async-timeout==3.0.1
attrs==19.1.0
certifi==2019.9.11
chardet==3.0.4
Click==7.0
idna==2.8
iso8601==0.1.12
kopf==0.21
multidict==4.5.2
pip==19.1.1
pykube-ng==19.9.2
PyYAML==5.1.2
requests==2.22.0
setuptools==41.0.1
urllib3==1.25.3
wheel==0.33.4
yarl==1.3.0

---

Commented by nolar at 2019-09-26 11:00:05+00:00
 

thilp Can you please try it with kopf==0.22rc1? Should be fixed now (I've tested with one of the examples.)

---

Commented by thilp at 2019-09-26 12:24:17+00:00
 

Thank you, it works perfectly now! Please keep making Kopf amazing.