add featuregate for k8s 1.28 native sidecar container #2801

frzifus · 2024-03-29T22:03:50Z

Description:

Link to tracking Issue(s):

Resolves: Use k8s 1.28 native sidecar for OTEL collector sidecar mode #2376

Testing:

Documentation:

pkg/sidecar/pod.go

jaronoff97

I think this makes sense... is there a way to write an e2e for this?

swiatekm · 2024-04-09T09:01:41Z

I think this makes sense... is there a way to write an e2e for this?

Sure, you'd need to add a new test suite and run it with the gate enabled. Then checking if the sidecar is an initContainer should be straightforward.

frzifus · 2024-07-16T17:18:40Z

@swiatekm whats the best way to configure a k8s featuregate for a specific e2e test suite?

Do we need to create a new e.g. kind-1.30-native-sidecar.yaml and register it here?

opentelemetry-operator/.github/workflows/e2e.yaml

Lines 22 to 24 in a515234

    
           kube-version: 
        
             - "1.23" 
        
             - "1.30"

Do we need then to create a new step like e2e-test-sidecar here too?

opentelemetry-operator/.github/workflows/e2e.yaml

Line 14 in a515234

e2e-tests:

swiatekm · 2024-07-18T12:38:30Z

@swiatekm whats the best way to configure a k8s featuregate for a specific e2e test suite?

Do we need to create a new e.g. kind-1.30-native-sidecar.yaml and register it here?

opentelemetry-operator/.github/workflows/e2e.yaml

Lines 22 to 24 in a515234

kube-version:

- "1.23"

- "1.30"

Do we need then to create a new step like e2e-test-sidecar here too?

opentelemetry-operator/.github/workflows/e2e.yaml

Line 14 in a515234

e2e-tests:

You should do the same thing we do for enabling other options in e2e tests. See here.

You do need to add a new entry to the e2e test matrix, because you need a separate operator deployment with the flag enabled. Then you add a make target that adds the command-line option to the operator Deployment (same as we do for other operator flags), and use that make target for setup in the e2e tests.

pavolloffay · 2024-07-18T13:03:25Z

pkg/featuregate/featuregate.go

+	// It needs to be enabled with +featureGate=SidecarContainers.
+	// See:
+	// https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/#feature-gates-for-alpha-or-beta-features
+	EnableNativeSidecarContainers = featuregate.GlobalRegistry().MustRegister(


I wonder if we could on k8s 1.28+ always use the native sidecar pattern and keep using the existing approach on older k8s versions.

This way no feature flag is needed and we would always use a recommended approach.

We can do that, but we would expect different results in our e2e tests. Which makes it hart to execute the same tests on both platform versions.

I think I'm ok with that as well. I can't think of any issues it might cause.

ok, then I will enable it for the 1.28+ e2e tests. Would be nice it it makes it into v0.111.0.

frzifus · 2024-10-14T05:13:14Z

Seems we dont need to enable it in Kind.
https://kubernetes.io/docs/concepts/workloads/pods/sidecar-containers/

Provided that your cluster has the SidecarContainers feature gate enabled (the feature is active by default since Kubernetes v1.29), you can specify a restartPolicy for containers listed in a Pod's initContainers field.

pkg/sidecar/pod.go

Signed-off-by: Benedikt Bongartz <bongartz@klimlive.de>

pkg/sidecar/pod.go

pkg/sidecar/pod_test.go

jaronoff97 · 2024-10-14T18:35:52Z

tests/e2e-native-sidecar/00-assert.yaml

+  containers:
+  - name: myapp
+  initContainers:
+  - name: otc-container


should we verify that this has the correct restart policy?

tests/e2e-native-sidecar/00-install.yaml

pavolloffay · 2024-10-15T14:01:59Z

.chloggen/native_sidecar.yaml

+change_type: enhancement
+
+# The name of the component, or a single word describing the area of concern, (e.g. collector, target allocator, auto-instrumentation, opamp, github action)
+component: pkg/sidecar


pavolloffay · 2024-10-15T14:03:01Z

.chloggen/native_sidecar.yaml

+# (Optional) One or more lines of additional information to render under the primary note.
+# These lines will be padded with 2 spaces and then inserted directly into the document.
+# Use pipe (|) for multiline entries.
+subtext:


Could you please document here the feature gate as a flag e.g. --. Could you please document which k8s version support by default native sidecar?

Signed-off-by: Benedikt Bongartz <bongartz@klimlive.de>

frzifus · 2024-10-15T15:42:50Z

Updated the changelog @pavolloffay

pavolloffay

LGTM,

just some docs nits

.github/workflows/e2e.yaml

pkg/featuregate/featuregate.go

Signed-off-by: Benedikt Bongartz <bongartz@klimlive.de>

frzifus · 2024-10-16T07:21:27Z

Changed it to k8s 1.29 and we now mention in the fg desc that it should only be enabled on k8s v1.29+.

frzifus force-pushed the native_sidecar branch from 058c14f to 8384910 Compare April 2, 2024 10:19

frzifus commented Apr 2, 2024

View reviewed changes

pkg/sidecar/pod.go Outdated Show resolved Hide resolved

frzifus marked this pull request as ready for review April 2, 2024 10:24

frzifus requested a review from a team April 2, 2024 10:24

frzifus added the help wanted Extra attention is needed label Apr 2, 2024

jaronoff97 reviewed Apr 5, 2024

View reviewed changes

frzifus force-pushed the native_sidecar branch from 8384910 to 5261059 Compare July 16, 2024 17:15

frzifus requested a review from swiatekm July 16, 2024 17:18

pavolloffay reviewed Jul 18, 2024

View reviewed changes

frzifus removed the help wanted Extra attention is needed label Oct 7, 2024

frzifus force-pushed the native_sidecar branch from 5261059 to f141dcb Compare October 10, 2024 16:32

frzifus requested a review from a team as a code owner October 10, 2024 16:32

frzifus force-pushed the native_sidecar branch from f141dcb to c910e80 Compare October 10, 2024 16:43

frzifus mentioned this pull request Oct 11, 2024

Release the operator v0.111.0 #3323

Closed

frzifus force-pushed the native_sidecar branch from c910e80 to aebc2cb Compare October 14, 2024 05:14

frzifus requested review from pavolloffay and jaronoff97 October 14, 2024 05:16

frzifus force-pushed the native_sidecar branch from aebc2cb to 60e8a8e Compare October 14, 2024 11:01

swiatekm reviewed Oct 14, 2024

View reviewed changes

pkg/sidecar/pod.go Outdated Show resolved Hide resolved

pkg/sidecar/pod.go Outdated Show resolved Hide resolved

frzifus force-pushed the native_sidecar branch from 30ba9b7 to 54bd8d4 Compare October 14, 2024 17:25

frzifus added 2 commits October 14, 2024 19:26

add featuregate for k8s 1.28 native sidecar container

a80ac46

Signed-off-by: Benedikt Bongartz <bongartz@klimlive.de>

use slices methods to deal with sidecars

5773a44

Signed-off-by: Benedikt Bongartz <bongartz@klimlive.de>

frzifus force-pushed the native_sidecar branch from 54bd8d4 to 5773a44 Compare October 14, 2024 17:26

frzifus requested a review from swiatekm October 14, 2024 17:27

jaronoff97 reviewed Oct 14, 2024

View reviewed changes

frzifus requested a review from jaronoff97 October 14, 2024 20:24

swiatekm approved these changes Oct 15, 2024

View reviewed changes

pavolloffay reviewed Oct 15, 2024

View reviewed changes

frzifus mentioned this pull request Oct 15, 2024

Offer native-sidecar mode #3356

Open

apply recommendations

9471885

Signed-off-by: Benedikt Bongartz <bongartz@klimlive.de>

frzifus force-pushed the native_sidecar branch from f9df64a to 9471885 Compare October 15, 2024 15:42

frzifus requested a review from pavolloffay October 15, 2024 15:42

pavolloffay reviewed Oct 15, 2024

View reviewed changes

.github/workflows/e2e.yaml Outdated Show resolved Hide resolved

pkg/featuregate/featuregate.go Outdated Show resolved Hide resolved

test with k8s 1.29

efaf294

Signed-off-by: Benedikt Bongartz <bongartz@klimlive.de>

frzifus requested a review from pavolloffay October 16, 2024 07:21

pavolloffay approved these changes Oct 16, 2024

View reviewed changes

pavolloffay merged commit 9a1b585 into open-telemetry:main Oct 16, 2024
36 checks passed

frzifus deleted the native_sidecar branch October 16, 2024 07:45

alexkruc mentioned this pull request Dec 5, 2024

Merge from Upstream (#1) #3514

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add featuregate for k8s 1.28 native sidecar container #2801

add featuregate for k8s 1.28 native sidecar container #2801

frzifus commented Mar 29, 2024

jaronoff97 left a comment

swiatekm commented Apr 9, 2024

frzifus commented Jul 16, 2024

swiatekm commented Jul 18, 2024

pavolloffay Jul 18, 2024

frzifus Jul 18, 2024

swiatekm Jul 18, 2024

frzifus Oct 7, 2024

frzifus commented Oct 14, 2024

jaronoff97 Oct 14, 2024

pavolloffay Oct 15, 2024

pavolloffay Oct 15, 2024

frzifus commented Oct 15, 2024

pavolloffay left a comment

frzifus commented Oct 16, 2024

add featuregate for k8s 1.28 native sidecar container #2801

add featuregate for k8s 1.28 native sidecar container #2801

Conversation

frzifus commented Mar 29, 2024

jaronoff97 left a comment

Choose a reason for hiding this comment

swiatekm commented Apr 9, 2024

frzifus commented Jul 16, 2024

swiatekm commented Jul 18, 2024

pavolloffay Jul 18, 2024

Choose a reason for hiding this comment

frzifus Jul 18, 2024

Choose a reason for hiding this comment

swiatekm Jul 18, 2024

Choose a reason for hiding this comment

frzifus Oct 7, 2024

Choose a reason for hiding this comment

frzifus commented Oct 14, 2024

jaronoff97 Oct 14, 2024

Choose a reason for hiding this comment

pavolloffay Oct 15, 2024

Choose a reason for hiding this comment

pavolloffay Oct 15, 2024

Choose a reason for hiding this comment

frzifus commented Oct 15, 2024

pavolloffay left a comment

Choose a reason for hiding this comment

frzifus commented Oct 16, 2024