Skip to content

Commit

Permalink
Add pipeline results
Browse files Browse the repository at this point in the history
  • Loading branch information
othomann committed Apr 6, 2020
1 parent 44ccf4d commit 1e89bd7
Show file tree
Hide file tree
Showing 20 changed files with 1,000 additions and 494 deletions.
5 changes: 2 additions & 3 deletions docs/pipelineruns.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,6 @@ spec:
[Here](../examples/v1beta1/pipelineruns/pipelinerun-with-pipelinespec.yaml) is a sample `PipelineRun` to display different
greetings while embedding the spec of the `Pipeline` directly in the `PipelineRun`.


After creating such a `PipelineRun`, the logs from this pod are displaying morning greetings:

```bash
Expand All @@ -101,6 +100,7 @@ Good Morning, Bob!
```

And the logs from this pod are displaying evening greetings:

```bash
kubectl logs $(kubectl get pods -o name | grep pipelinerun-echo-greetings-echo-good-night)
Good Night, Bob!
Expand All @@ -121,7 +121,6 @@ spec:
[Here](../examples/v1beta1/pipelineruns/pipelinerun-with-pipelinespec-and-taskspec.yaml) is a sample `PipelineRun` with embedded
the spec of the `Pipeline` directly in the `PipelineRun` along with the spec of the `Task` under `PipelineSpec`.


### Resources

When running a [`Pipeline`](pipelines.md), you will need to specify the
Expand Down Expand Up @@ -154,7 +153,6 @@ spec:

Or you can embed the spec of the `Resource` directly in the `PipelineRun`:


```yaml
spec:
resources:
Expand Down Expand Up @@ -224,6 +222,7 @@ spec:
- taskName: build-task
serviceAccountName: sa-for-build
```

If used with this `Pipeline`, `test-task` will use the `ServiceAccount` `sa-1`, while `build-task` will use `sa-for-build`.

```yaml
Expand Down
21 changes: 19 additions & 2 deletions docs/pipelines.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ following fields:
fail. There is no default timeout for a Pipeline Task timeout. If no timeout is specified for
the Pipeline Task, the only timeout taken into account for running a `Pipeline` will be a
[timeout for the `PipelineRun`](https://github.com/tektoncd/pipeline/blob/master/docs/pipelineruns.md#syntax).
- [`results`](#pipeline-results) - Specifies which `results` is defined for the pipeline

[kubernetes-overview]:
https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/#required-fields
Expand Down Expand Up @@ -419,7 +420,7 @@ spec:
The Timeout property is specified as part of the Pipeline Task on the `Pipeline` spec. The above
example has a timeout of one minute and 30 seconds.

### Results
#### Results

Tasks can declare [results](./tasks.md#results) that they will emit during their execution. These results can be used as values for params in subsequent tasks of a Pipeline. Tekton will infer the ordering of these Tasks to ensure that the Task emitting the results runs before the Task consuming those results in its parameters.

Expand All @@ -435,7 +436,7 @@ In this example the previous pipeline task has name "previous-task-name" and its

For a complete example demonstrating Task Results in a Pipeline see the [pipelinerun example](../examples/v1beta1/pipelineruns/task_results_example.yaml).

### Ordering
#### Ordering

The [Pipeline Tasks](#pipeline-tasks) in a `Pipeline` can be connected and run
in a graph, specifically a _Directed Acyclic Graph_ or DAG. Each of the Pipeline
Expand Down Expand Up @@ -528,6 +529,22 @@ build-app build-frontend
1. The entire `Pipeline` will be finished executing after `lint-repo` and
`deploy-all` have completed.

### Pipeline Results

A pipeline can declare results that they will emit during their execution. These results can be defined as reference to task results executed
during the pipeline execution.

```yaml
results:
- name: sum
description: the sum of all three operands
value: $(tasks.second-add.results.sum)
```

In this example the pipeline result has name "sum" and its result is declared as the task result value from the tasks named `second-add`.

For a complete example demonstrating pipeline Results in a Pipeline see the [pipeline example](../examples/pipelineruns/pipelinerun-results.yaml).

## Examples

For complete examples, see
Expand Down
79 changes: 79 additions & 0 deletions examples/v1beta1/pipelineruns/pipelinerun-results.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: add-task
spec:
params:
- name: first
description: the first operand
- name: second
description: the second operand
results:
- name: sum
description: the sum of the first and second operand
steps:
- name: add
image: alpine
env:
- name: OP1
value: $(params.first)
- name: OP2
value: $(params.second)
command: ["/bin/sh", "-c"]
args:
- echo -n $((${OP1}+${OP2})) | tee $(results.sum.path);
---
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: sum-three-pipeline
spec:
params:
- name: first
description: the first operand
- name: second
description: the second operand
- name: third
description: the third operand
tasks:
- name: first-add
taskRef:
name: add-task
params:
- name: first
value: $(params.first)
- name: second
value: $(params.second)
- name: second-add
taskRef:
name: add-task
params:
- name: first
value: $(tasks.first-add.results.sum)
- name: second
value: $(params.third)
results:
- name: sum
description: the sum of all three operands
value: $(tasks.second-add.results.sum)
- name: partial-sum
description: the sum of first two operands
value: $(tasks.first-add.results.sum)
- name: all-sum
description: the sum of everything
value: $(tasks.second-add.results.sum)-$(tasks.first-add.results.sum)
---
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
name: sum-three-pipeline-run
spec:
pipelineRef:
name: sum-three-pipeline
params:
- name: first
value: "2"
- name: second
value: "10"
- name: third
value: "10"
2 changes: 2 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ go 1.13

require (
cloud.google.com/go v0.47.0 // indirect
cloud.google.com/go/storage v1.0.0
contrib.go.opencensus.io/exporter/stackdriver v0.12.8 // indirect
github.com/GoogleCloudPlatform/cloud-builders/gcs-fetcher v0.0.0-20191203181535-308b93ad1f39
github.com/cloudevents/sdk-go/v2 v2.0.0-preview8
Expand Down Expand Up @@ -40,6 +41,7 @@ require (
golang.org/x/time v0.0.0-20191024005414-555d28b269f0 // indirect
golang.org/x/tools v0.0.0-20200214144324-88be01311a71 // indirect
gomodules.xyz/jsonpatch/v2 v2.1.0 // indirect
google.golang.org/api v0.15.0
google.golang.org/appengine v1.6.5 // indirect
k8s.io/api v0.17.3
k8s.io/apiextensions-apiserver v0.17.3 // indirect
Expand Down
15 changes: 12 additions & 3 deletions pkg/apis/pipeline/v1alpha1/pipeline_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,14 @@ type PipelineSpec struct {
// provided by a PipelineRun.
// +optional
Workspaces []WorkspacePipelineDeclaration `json:"workspaces,omitempty"`
// Results are values that this pipeline can output once run
// +optional
Results []PipelineResult `json:"results,omitempty"`
}

// PipelineResult used to describe the results of a pipeline
type PipelineResult = v1beta1.PipelineResult

// Check that Pipeline may be validated and defaulted.
// TaskKind defines the type of Task used by the pipeline.
type TaskKind = v1beta1.TaskKind
Expand Down Expand Up @@ -164,9 +170,12 @@ func (pt PipelineTask) Deps() []string {
}
// Add any dependents from task results
for _, param := range pt.Params {
if resultRefs, err := v1beta1.NewResultRefs(param); err == nil {
for _, resultRef := range resultRefs {
deps = append(deps, resultRef.PipelineTask)
expressions, ok := v1beta1.GetVarSubstitutionExpressionsForParam(param)
if ok {
if resultRefs, err := v1beta1.NewResultRefs(expressions); err == nil {
for _, resultRef := range resultRefs {
deps = append(deps, resultRef.PipelineTask)
}
}
}
}
Expand Down
27 changes: 25 additions & 2 deletions pkg/apis/pipeline/v1alpha1/pipeline_validation.go
Original file line number Diff line number Diff line change
Expand Up @@ -140,8 +140,26 @@ func validateGraph(tasks []PipelineTask) error {
func validateParamResults(tasks []PipelineTask) error {
for _, task := range tasks {
for _, param := range task.Params {
if v1beta1.LooksLikeContainsResultRefs(param) {
if _, err := v1beta1.NewResultRefs(param); err != nil {
expressions, ok := v1beta1.GetVarSubstitutionExpressionsForParam(param)
if ok {
if v1beta1.LooksLikeContainsResultRefs(expressions) {
if _, err := v1beta1.NewResultRefs(expressions); err != nil {
return err
}
}
}
}
}
return nil
}

// validatePipelineResults ensure that task result variables are properly configured
func validatePipelineResults(results []PipelineResult) error {
for _, result := range results {
expressions, ok := v1beta1.GetVarSubstitutionExpressionsForPipelineResult(result)
if ok {
if v1beta1.LooksLikeContainsResultRefs(expressions) {
if _, err := v1beta1.NewResultRefs(expressions); err != nil {
return err
}
}
Expand Down Expand Up @@ -228,6 +246,11 @@ func (ps *PipelineSpec) Validate(ctx context.Context) *apis.FieldError {
return err
}

// Validate the pipeline's results
if err := validatePipelineResults(ps.Results); err != nil {
return apis.ErrInvalidValue(err.Error(), "spec.tasks.params.value")
}

return nil
}

Expand Down
5 changes: 5 additions & 0 deletions pkg/apis/pipeline/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 1e89bd7

Please sign in to comment.