title | authors | creation-date | last-updated | status | |
---|---|---|---|---|---|
step-and-sidecar-workspaces |
|
2020-10-02 |
2022-07-22 |
implemented |
- Summary
- Motivation
- Requirements
- Proposal
- Design Details
- Drawbacks
- Alternatives
- Upgrade & Migration Strategy (optional)
- References (optional)
- Implementation Pull Requests
This TEP is motivated by 3 major goals:
- Limit access to sensitive credentials.
- Add blessed support for Sidecars to access workspaces.
- Make Workspace behaviour uniform across Steps and Sidecars.
- Provide a mechanism to limit the exposure of sensitive workspaces to only those Steps and Sidecars in a Task that actually require access to them.
- Provide explicit access to Task workspaces from Sidecars without using
volumeMounts
so that Sidecars can access workspaces independent of the platform-specific concept of volumes. - Normalize behaviour of Workspaces across Steps and Sidecars.
- A Task author can limit access to a
Workspace
to only thoseSteps
that actually require the contents of thatWorkspace
. By doing so they can limit the running code that has access to those contents as well. - A Task author can use a Workspace from a
Sidecar
. - A Task author can still use the volume "hack" to attach
Workspaces
toSidecars
in combination with the feature proposed here. - A Task author can choose different
mountPaths
for each Step that receives theWorkspace
.
Add workspaces
lists to Steps and Sidecars. Here's the existing YAML and behaviour:
spec:
workspaces:
- name: foo
steps:
- image: ubuntu
script: # ... This Step automatically mounts "foo" workspace.
- image: ubuntu
script: # ... And so does this Step.
And here's a Task using the new Step Workspaces feature:
spec:
workspaces:
- name: foo
steps:
- image: ubuntu
workspaces:
- name: foo
script: # ... This Step mounts "foo" workspace.
- image: ubuntu
script: # ... But this Step does not.
The existing YAML and behaviour will continue to be supported.
-
Add a
workspaces
list toSteps
. -
Allow
workspaces
from theTask
to be explicitly named in that list like this:workspaces: - name: my-sensitive-workspace steps: - name: foo workspaces: - name: my-sensitive-workspace
-
When a
workspace
is listed in a Step, it is no longer automatically mounted - either toSteps
orSidecars
- unless they also have theworkspace
in their ownworkspaces
list.
Example YAML:
# task spec
spec:
workspaces:
- name: git-ssh-credentials
mountPath: /root/.ssh
steps:
- name: clone-repo
image: alpine/git:v2.26.2
workspaces:
- name: git-ssh-credentials
script: |
git clone $(params.repo-url) /workspace/source
- name: run-unit-tests
script: |
cd /workspace/source
go test ./...
In the above example only the clone-repo
Step
will receive access to the git-ssh-credentials
Workspace
. The run-unit-tests
Step
will not receive access to the Workspace volume. Importantly
this also means that the user-supplied code does not have access to the credential files. Compromising
the code for the unit tests does not also compromise the SSH credentials.
-
Automatically mount
workspaces
toSidecars
just as they're automatically mounted toSteps
today. -
Add a
workspaces
list toSidecars
. -
Allow
workspaces
from theTask
to be explicitly named in that list like this:workspaces: - name: my-workspace sidecars: - name: watch-workspace workspaces: - name: my-workspace
-
When a
workspace
is listed in a Sidecar, it is no longer automatically mounted - either toSteps
orSidecars
- unless they also have theworkspace
in their ownworkspaces
list.
When declaring the Workspace in the Step or Sidecar a custom mountPath can also be specified.
This allows for situations where different images may have different expectations for the location
of files from a workspace. This mountPath overrides whatever mountPath is set on the Task Spec's
workspaces
entry.
# Task Spec
workspaces:
- name: ws
mountPath: /workspaces/ws
steps:
- name: edit-files-1
workspaces:
- name: foo
mountPath: /foo # overrides mountPath
- name: edit-files-2
workspaces:
- name: foo # no mountPath specified so will use /workspaces/ws
sidecars:
- name: watch-files-on-workspace
workspaces:
- name: foo
mountPath: /files # overrides mountPath
An author of the buildpacks-phases
Catalog task may want to rewrite the Task to reduce the possible blast radius of
running untrusted images by limiting exposure of Docker credentials to only
the Step which needs them to push images.
In the buildpacks-phases Task there are 7 Steps and only 1 appears to need docker credentials. There are 6 other Steps that will currently receive creds-init docker credentials, running different images with different scripts and programs that could each be a vector to compromise those credentials.
As the author of an API Testing Task that mocks API responses with fixtures I want to write a Sidecar that can access a user-provided Workspace that contains API fixture data so that my mock API can respond with that fixture data when requested during test runs in the Steps.
As the author of a Task that needs to spin up an SSH server parallel to my Task's Steps
for testing against I want to use a Sidecar with access to a Workspace so that my Task's
Steps can generate a public key and share it with the Sidecar, allowing for quick configuration
of a temporary authorized_keys
file which in turn allows the Steps to successfully connect to
the Sidecar over SSH.
For an existing example where this could be useful, see the authenticating-git-commands example from the Pipelines repo.
As the author of a deployment PipelineRun that uses a kubectl
Catalog Task I want to be able
to trust that the certificate I provide via a Workspace for kubectl
to deploy to my production
environment is only being mounted in the single isolated Step
which calls kubectl apply
and not
to other Steps
in the same Task performing ancillary work.
As a Pipeline author I want to be able to quickly audit that the git-fetch
Task I am using in
my Pipeline is only exposing the git SSH key for my team's source repo in the single Step
that
performs git clone
, and not to any Steps
in the same Task performing ancillary work.
Add a new WorkspaceUsage
struct to the task_types.go
file:
// WorkspaceUsage declares that a Step or Sidecar utilizes a Task's Workspace.
type WorkspaceUsage struct {
// Name is the name of the Task's WorkspaceDeclaration that this Step or Sidecar is utilizing. It is required.
Name string `json:"name"`
// MountPath is an optional path that overrides where the Workspace will be mounted in the Task's filesystem.
MountPath string `json:"mountPath"`
}
Update the Step
struct to include a slice of WorkspaceUsage
:
type Step struct {
corev1.Container `json:",inline"`
Script string `json:"script,omitempty"`
// Workspaces is a list of workspaces that this Step declares it will use in some way. The presence
// of a Workspace in this list prevents other Steps from automatically receiving the Workspace -
// they must also explicitly opt-in to receiving it as well in their own workspaces list.
Workspaces []WorkspaceUsage `json:"workspaces,omitempty"`
}
Update the Sidecar
struct to include a slice of WorkspaceUsage
:
// Sidecar embeds the Container type, which allows it to include fields not
// provided by Container.
type Sidecar struct {
corev1.Container `json:",inline"`
Script string `json:"script,omitempty"`
// Workspaces is a list of workspaces that this Sidecar declares it will use in some way.
Workspaces []WorkspaceUsage `json:"workspaces,omitempty"`
}
- If we were to pursue the idea of a shareable
Step
type then we would need to find some way to map from the workspaces a Task declares into those that a referencedStep
declares. We'd similarly need to do this for params and results. - It takes the
Step
andSidecar
types further from being "pure" k8s Container. However we've already made moves in this direction by introducing our ownScript
field toSteps
andSidecars
. - Automounting workspaces into sidecars could have unexpected side effects. If a user is already mounting
a workspace to a specific location for their
Steps
but are relying on the fact that the sidecar does not mount to the same spot then they could potentially be broken by this change.
Instead of adding a workspaces
list to Steps
, we could instead lean on the existing
volumeMounts
list like this:
# task spec
spec:
workspaces:
- name: git-ssh-credentials
steps:
- name: clone-repo
image: alpine/git:v2.26.2
volumeMounts:
- name: $(workspaces.git-ssh-credentials.volume)
script: |
git clone $(params.repo-url) /workspace/source
- name: run-unit-tests
script: |
cd /workspace/source
go test ./...
And then add a rule that says "if you use a volumeMount to mount a workspace to your Step then Tekton will not automatically add volumeMounts to any other Steps." We would also need to document this rule.
Uses an existing Kubernetes container field that the user may already be familiar with.
It's surprising. This approach overloads the meaning of volumeMounts
with
additional Tekton-specific context and behaviour (i.e. the additional constraint
that adding a workspace volume to a volumeMount
entry will prevent other Steps
from receiving that workspace
unless they too include it in their
volumeMounts
list).
A further drawback is that workspaces
provide a useful abstraction for API implementers.
Leaning on volumeMounts
would move a Kubernetes-specific field into Tekton's API. We'd have
to decide which of the fields at https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.19/#volumemount-v1-core
to support.
Allow Steps
to fully-specify Workspace declarations so that they're not also required
at the top-level of the Task spec. Here's how that might look:
steps:
- name: foo
workspaces:
- name: my-sensitive-workspace
In contrast with the existing proposal:
workspaces:
- name: my-sensitive-workspace
steps:
- name: foo
workspaces:
- name: my-sensitive-workspace
- Shorter syntax for isolating
Workspaces
to singleSteps
. - Allows for immediate use of fields like
mountPath
as part of theWorkspace
entry in theStep
.
-
What's the behaviour when a
Task
declares aWorkspace
andSteps
declare aWorkspace
with the same name? Possibly the same behaviour as is being proposed by this document? Or a validation error? -
Open question whether a Task author would be able to share a
Workspace
this way across multipleSteps
if they share a commonWorkspace
name:steps: - name: foo workspaces: - name: my-sensitive-workspace - name: bar workspaces: - name: my-sensitive-workspace
Here the
Task
would presumably only expose a singleWorkspace
named "my-sensitive-workspace" to be populated by aTaskRun
/PipelineRun
?Validation might be made more difficult here - if two
Workspaces
in twoSteps
are named almost the same thing the reconciler won't be able to tell if they're "supposed" to be the same or not. Consider the following example:steps: - name: foo workspaces: - name: my-sensitive-workspace - name: bar workspaces: - name: my-senistive-workspace
Was this a mis-spelling by the
Task
author or intentionally separateWorkspace
declarations?One further extension to this idea would be to allow
TaskRuns
orPipelineRuns
to bindWorkspaces
to specificSteps
andSidecars
:workspaces: - name: my-workspace emptyDir: {} - name: sensitive-workspace secretRef: name: foo-secret sidecars: - name: do-something-not-secret workspaces: - name: my-workspace steps: - name: do-something-secret workspaces: - name: some-sensitive-workspace
The feature as proposed is entirely backwards-compatible. Omitting the
workspaces
field from Steps
leaves the existing behaviour exactly as
it works today - all Steps will receive all workspaces.
- Original design part of the Credentials UX issue.