Container configurations need to be updated for PodSecurity on k8s 1.23+

[abayer]

I've verified this on 1.23 and 1.24 in clusters created via kind. The error message is the same.

The deployments fail to create pods - from kubectl get deployment -n tekton-pipelines tekton-pipelines-controller:

    message: 'pods "tekton-pipelines-controller-fcfbc554b-9fcdj" is forbidden: violates
      PodSecurity "restricted:latest": seLinuxOptions (pod set forbidden securityContext.seLinuxOptions:
      ), unrestricted capabilities (container "tekton-pipelines-controller" must set
      securityContext.capabilities.drop=["ALL"]), seccompProfile (pod or container
      "tekton-pipelines-controller" must set securityContext.seccompProfile.type to
      "RuntimeDefault" or "Localhost")'

This is due to b506b77#diff-d98ea73731da6e49a8552d6917d796aab6613198b269eda1b4bbfb4b53d90111R22, added in #5536 as part of addressing #4112.

While the move to PodSecurity is being reverted in #5605, this issue will be used to track the additional container configuration work needed to move to PodSecurity.

[abayer]

cc @tektoncd/core-maintainers - this definitely feels like a blocker for v0.41.0.

[abayer]

cc @JeromeJu

[abayer]

Ok, so I'm not sure what exactly we need to change on the controller/webhook/resolver container configurations, or whether we can do those changes while still working on k8s 1.22. The right answer may be to revert #5536 for now, move v0.41 to requiring k8s 1.23, and then bringing back #5536 and whatever additional configuration is needed on the containers for v0.42.

[lbernick]

It sounds like we should be able to fix this without bumping the min required version by setting the correct security context for the pipelines controller (although I think moving our HPA to v2 will require 1.23 anyway). I'll work with Jerome to try to address this but it probably does make sense to revert until this can be addressed.

[abayer]

Yeah, https://kubernetes.io/blog/2021/12/09/pod-security-admission-beta/ makes me think we don't need to go to 1.23 to get things working, so that's good. =)

[abayer]

FYI - I'm going to repurpose this issue to be more generally about needing to do more work to move to PodSecurity.

[dibyom]

(removing critical/urgent since we reverted the initial change)

[abayer]

For the record, I think it should still be critical-urgent since we need to support k8s 1.25, but hey, I seem to have found the fix - see #5652 (comment)