Report downwardAPI volumes are not allowed to be used in a shared cluster

[zhangtbj]

Hi all,
I am using Tekton with (buildpacks) to build my source code to the container image.
But I didn't run the Tekton task on my own cluster, it is a shared cluster with some security policy.

It works fine on my own cluster, but in this shared cluster, it reported an error:
Warning BuildCreationFailed 55m taskrun-controller Failed to create build pod "buildpack-nodejs-b95mx": pods "buildpack-nodejs-b95mx-pod-8gh7s" is forbidden: unable to validate against any pod security policy: [spec.volumes[5]: Invalid value: "downwardAPI": downwardAPI volumes are not allowed to be used spec.volumes[5]: Invalid value: "downwardAPI": downwardAPI volumes are not allowed to be used]

And I heard from Slack channel that Tekton plan to the downwardAPI can be configurable. Is that done now?

And I confirmed with cluster admin. We had the PSP like this:

apiVersion: extensions/v1beta1
kind: PodSecurityPolicy
spec:
  volumes:
  - downwardAPI

But still fail.

I just want to build a container image, is there any way to workaround this problem or disable the downwardAPI during the build in Tekton?

Thanks a lot

[imjasonh]

Enforcing security policies and reporting conflicts seems like it should remain K8s' responsibility, and Tekton shouldn't get involved. The author of a Task that depends on a downwardAPI volume should make it clear to users that this might not always be available on a particular cluster, either due to PSP, or other limitations of the cluster in question (e.g., other exotic volume types are only supported on certain cloud providers, for example).

[vdemeester]

Enforcing security policies and reporting conflicts seems like it should remain K8s' responsibility, and Tekton shouldn't get involved. The author of a Task that depends on a downwardAPI volume should make it clear to users that this might not always be available on a particular cluster, either due to PSP, or other limitations of the cluster in question (e.g., other exotic volume types are only supported on certain cloud providers, for example).

I think this particular downwardAPI volume is from an internal step / container though 😅

[zhangtbj]

yes, from internal task step / container :)

[ghost]

Just to help clarify,

And I heard from Slack channel that Tekton plan to the downwardAPI can be configurable. Is that done now?

I originally thought we would be removing the use of the downwardAPI in #2158 but this is not the case. Instead we are changing when the sidecar readiness annotation will appear. The downward API is still used to project that annotation into step containers.

Unfortunately I do not think we support this use case - a cluster must support downwardAPI volumes for Tekton to work.

So, I'm going to mark this issue as a feature request.

[imjasonh]

I think this particular downwardAPI volume is from an internal step / container though 😅

Oh whoops! 😓

In that case, we should document that Tekton currently requires access to the Downward API for volumes.

Longer term, if this is serious issue, we have ideas for better signaling (#1570) that might allow us to remove the downward dependency.

[skaegi]

Hmm... very nasty. FWIW we allow use of downward api in our multi-tenant cluster however each tenant workload is restricted to a single namespace so we can get away with a slightly less restrictive policy here.

[ghost]

I guess the real question here is why the PodSecurityPolicy allows downwardAPI but this doesn't seem to affect the pods that Tekton creates.

I'm going to add a bug label based on that and remove the feature label. This seems like something that should work when a pod security policy is in place and correct.

/kind bug

[ghost]

It looks like we have our own PodSecurityPolicy which does not list downwardApi as a supported volume type. This is attached to our tekton-pipelines-admin service account in our clusterrole configuration.

I'm not 100% sure on how this works but am wondering if this might be the reason that the error is happening despite the PodSecurityPolicy mentioned in the issue description?

@zhangtbj could you try updating the tekton-pipelines PodSecurityPolicy in your cluster with 'downwardAPI' in the list of volumes, to see if that fixes the issue for you?

[bobcatfish]

I'm going to close this issue for now but @zhangtbj please reopen if you are still having trouble!

[hWorblehat]

I've raised #4937 because I have an entirely separate scenario where downwardAPI volumes are causing a problem (virtual kubelets).