Kuma injector overrides annotation configuration with default values

[rejmond]

What happened?

I am trying to configure a pod with annotations, using this documentation.
The Kuma injector don't apply them and use default configuration.

How to reproduce?

I use Kind cluster for development.

Create a cluster.

cat > cluster-config.yaml <<EOF
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
 - role: control-plane
 - role: worker
EOF

kind create cluster --config ./cluster-config.yaml

Install Kuma control plane.

kumactl install control-plane | kubectl apply -f -

Create Kuma meshed pod.

cat > pod.yaml <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: hello-world
  labels:
    kuma.io/sidecar-injection: enabled
  annotations:
    kuma.io/builtindns: enabled
    kuma.io/builtindnsport: "15058"
spec:
  restartPolicy: Never
  containers:
    - name: hello-world
      image: strm/helloworld-http
      imagePullPolicy: IfNotPresent
      stdin: true
      tty: true
EOF

kubectl apply -f pod.yaml

Check pod annotations.

kubectl describe pod hello-world

Expected output:

Name:         hello-world
Namespace:    default
Priority:     0
Node:         kind-worker/172.18.0.3
Start Time:   Fri, 10 Jun 2022 14:41:35 +0700
Labels:       kuma.io/sidecar-injection=enabled
Annotations:  kuma.io/builtindns: enabled
              kuma.io/builtindnsport: 15058
              ...

Real output:

Name:         hello-world
Namespace:    default
Priority:     0
Node:         kind-worker/172.18.0.3
Start Time:   Fri, 10 Jun 2022 14:41:35 +0700
Labels:       kuma.io/sidecar-injection=enabled
Annotations:  kuma.io/builtindns: enabled
              kuma.io/builtindnsport: 15053
              ...

Pod annotation kuma.io/builtindnsport has a default value 15053, not 15058 as I set in pod's configuration.

Kuma version
Kuma: 1.6.0

Kuma configuration
Default configuration file without changes.

kuma-cp.conf.txt

[github-actions]

This issue was inactive for 30 days it will be reviewed in the next triage meeting and might be closed.
If you think this issue is still relevant please comment on it promptly or attend the next triage meeting.

[tharun208]

@lahabana I tested it with the master and injector is able to override the default values with the user specified one..

[lahabana]

I've just tried and I see @rejmond 's problem on master (the port doesn't take the annotation):

➜  kuma git:(master) ✗ kubectl describe pod hello-world | grep kuma.io
Labels:       kuma.io/sidecar-injection=enabled
Annotations:  kuma.io/builtindns: enabled
              kuma.io/builtindnsport: 15053
              kuma.io/envoy-admin-port: 9901
              kuma.io/mesh: default
              kuma.io/sidecar-injected: true
              kuma.io/sidecar-uid: 5678
              kuma.io/transparent-proxying: enabled
              kuma.io/transparent-proxying-inbound-port: 15006
              kuma.io/transparent-proxying-inbound-v6-port: 15010
              kuma.io/transparent-proxying-outbound-port: 15001
              kuma.io/virtual-probes: enabled
              kuma.io/virtual-probes-port: 9000
  Normal  CreatedKumaDataplane  97s                k8s.kuma.io/dataplane-generator  Created Kuma Dataplane: hello-world
  Normal  UpdatedKumaDataplane  67s (x4 over 97s)  k8s.kuma.io/dataplane-generator  Updated Kuma Dataplane: hello-world