Allow Pipelines to be used with different Resources

This change makes it possible to reuse a Pipeline with different
Resources without having to change the Pipeline itself. It does this by
moving the linking of the Resources a Task requires from the Pipeline
into the PipelineRun.

The relationship between Resources and the Tasks that are expected to be
executed on them is still expressed in the Pipeline (via `providedBy`).

ResourceBindings move from Pipeline to PipelineRun and become
ResourceDependencies. No longer calling these "bindings" in the API or
using the term "source" fixes + the additional docs in using.md fixes #139.

The most difficult part of this change was updating
validatePipelineTaskAndTask - hoping to refactor this in a follow up
(which would also address #213).

This interface is still probably not in its final form and hopefully we
can iterate on it!

Fixes #200

docs/using.md

@@ -3,7 +3,7 @@
 * [How do I create a new Pipeline?](#creating-a-pipeline)
 * [How do I make a Task?](#creating-a-task)
 * [How do I make Resources?](#creating-resources)
-* [How do I run a Pipeline?](#running-a-task)
+* [How do I run a Pipeline?](#running-a-pipeline)
 * [How do I run a Task on its own?](#running-a-task)
 * [How do I troubleshoot a PipelineRun?](#troubleshooting)
 
@@ -136,10 +136,46 @@ ${inputs.params.NAME}
 
 ## Running a Pipeline
 
-1. To run your `Pipeline`, create a new `PipelineRun` which links your `Pipeline` to the
-   `PipelineParams` it will run with.
-2. Creation of a `PipelineRun` will trigger the creation of [`TaskRuns`](#running-a-task)
-   for each `Task` in your pipeline.
+In order to run a Pipeline, you will need to provide:
+
+1. A Pipeline to run (see [creating a Pipeline](#creating-a-pipeline))
+2. `PipelineParams` which specify the service account to use and where to put logs (see [the example `PipelineParams`](examples/pipelineparams.yaml))
+3. The `PipelineResources` to use with this Pipeline.
+
+On its own, a `Pipeline` declares what `Tasks` to run, and what order to run them in
+(implied by [`providedBy`](#providedby)). When running a `Pipeline`, you will
+need to specify the `PipelineResources` to use with it. One `Pipeline` may need to be run
+with different `PipelineResources` in cases such as:
+
+* When triggering the run of a `Pipeline` against a pull request, the triggering system must
+  specify the commitish of a git `PipelineResource` to use
+* When invoking a `Pipeline` manually against one's own setup, one will need to ensure that
+  one's own github fork (via the git `PipelineResource`), image registry (via the image
+  `PipelineResource`) and kubernetes cluster (via the cluster `PipelineResource`).
+
+Specify the `PipelineResources` in the PipelineRun using the `resources` section, for example:
+
+```yaml
+resources:
+- name: push-kritis
+  inputs:
+    - key: workspace
+      resourceRef:
+        name: kritis-resources-git
+  outputs:
+    - key: builtImage
+      resourceRef:
+        name: kritis-resources-image
+```
+
+This example section says:
+
+* For the `Task` in the `Pipeline` called `push-kritis`
+* For the input called `workspace`, use the existing resource called `kritis-resources-git`
+* For the output called `builtImage`, use the existing resource called `kritis-resources-image`
+
+Creation of a `PipelineRun` will trigger the creation of [`TaskRuns`](#running-a-task)
+for each `Task` in your pipeline.
 
 See [the example PipelineRun](../examples/invocations/kritis-pipeline-run.yaml).
 

examples/README.md

@@ -15,24 +15,28 @@ kubectl apply -f examples/invocations
 
 We have 2 example [Pipelines](../README.md#pipeline) in [./pipelines](./pipelines)
 
-1. [The Kritis Pipline](./pipelines/kritis.yaml): This example builds a Pipeline for the
-   [kritis project](https://github.com/grafeas/kritis), and demonstrates how to configure
-    a pipeline which:
+### Kritis Pipeline
 
-    1. Runs unit tests
-    2. Build an image
-    3. Deploys it to a test environment
-    4. Runs integration tests
+[The Kritis Pipline](./pipelines/kritis.yaml): This example builds a Pipeline for the
+[kritis project](https://github.com/grafeas/kritis), and demonstrates how to configure
+a pipeline which:
 
-   ![Pipeline Configuration](./pipelines/kritis-pipeline.png)
+1. Runs unit tests
+2. Build an image
+3. Deploys it to a test environment
+4. Runs integration tests
 
-2. [Guestbook](./pipelines/guestbook.yaml): This Pipeline is based on example application in
-   [the Kubernetes example Repo](https://github.com/kubernetes/examples/tree/master/guestbook)
-   This pipeline demonstartes how to integrate frontend
-   [guestbook app code](https://github.com/kubernetes/examples/tree/master/guestbook-go) with
-   backend [redis-docker image](https://github.com/GoogleCloudPlatform/redis-docker/tree/master/4) provided by GCP.
+![Pipeline Configuration](./pipelines/kritis-pipeline.png)
 
-   ![Pipeline Configuration](./pipelines/guestbook-pipeline.png)
+### Guestbook Pipeline
+
+[Guestbook](./pipelines/guestbook.yaml): This Pipeline is based on example application in
+[the Kubernetes example Repo](https://github.com/kubernetes/examples/tree/master/guestbook)
+This pipeline demonstartes how to integrate frontend
+[guestbook app code](https://github.com/kubernetes/examples/tree/master/guestbook-go) with
+backend [redis-docker image](https://github.com/GoogleCloudPlatform/redis-docker/tree/master/4) provided by GCP.
+
+![Pipeline Configuration](./pipelines/guestbook-pipeline.png)
 
 ## Example Tasks
 
@@ -52,14 +56,11 @@ This task then calls `docker run` which will run the test code. This follows the
 
 ### Example Runs
 
-The [invocations](./invocations/) directory contains an example [TaskRun](../docs/Concepts.md#taskrun) and an example [PipelineRun](../docs/Concepts.md#pipelinerun).
+The [invocations](./invocations/) directory contains an example [TaskRun](../docs/Concepts.md#taskrun) and two example [PipelineRuns](../docs/Concepts.md#pipelinerun).
 
 [run-kritis-test.yaml](./invocations/run-kritis-test.yaml) shows an example of how to manually run kritis unit test off your development branch.
 
-[kritis-pipeline-run.yaml](./invocations/kritis-pipeline-run.yaml) shows an example of
-what it would look like to invoke the [kritis example pipeline](#example-pipelines)
-manually. In the `conditions` field for type `Successful` you can see that the status
-is `False`, which indicates that the Pipeline was not run successfully. The field
-`message` contains a human readable error indicating that one of the `TaskRuns` failed.
-This `condition` (and everything else in the `status` section) would be populated by the
-controller as it realized the PipelineRun (i.e. ran the Pipeline).
+The example `PipelineRuns are`:
+
+* [kritis-pipeline-run.yaml](./invocations/kritis-pipeline-run.yaml) invokes [the kritis example pipeline](#kritis-pipeline)
+* [guestbook-pipeline-run.yaml](./invocations/guestbook-pipeline-run.yaml) invokes [the guestbook example pipeline](#guestbook-pipeline)

examples/invocations/guestbook-pipeline-run.yaml

@@ -0,0 +1,69 @@
+apiVersion: pipeline.knative.dev/v1alpha1
+kind: PipelineRun
+metadata:
+  name: guestbook-pipeline-run-12321312984
+  namespace: default
+spec:
+  pipelineRef:
+    name: guestbook-example
+  pipelineParamsRef:
+    name: pipelineparams-sample
+  triggerRef:
+    type: manual
+  resources:
+  - name: build-guestbook
+    inputs:
+    - name: workspace
+      resourceRef:
+        name: guestbook-resources-git
+    outputs:
+    - name: builtImage
+      resourceRef:
+        name: guestbookstagingimage
+  - name: build-redis
+    inputs:
+    - name: workspace
+      resourceRef:
+        name: guestbook-resources-redis-docker
+    outputs:
+    - name: builtImage
+      resourceRef:
+        name: redisstagingimage
+  - name: deploy-bundle-test
+    inputs:
+    - name: imageToDeploy1
+      resourceRef:
+        name: redisstagingimage
+    - name: imageToDeploy2
+      resourceRef:
+        name: guestbookstagingimage
+    - name: workspace
+      resourceRef:
+        name: guestbook-resources-redis-docker
+    - name: testCluster
+      resourceRef:
+        name: testcluster
+  - name: int-test-osx
+    inputs:
+    - name: workspace
+      resourceRef:
+        name: guestbook-resources-git
+  - name: int-test-linux
+    inputs:
+    - name: workspace
+      resourceRef:
+        name: guestbook-resources-git
+  - name: deploy-bundle-test
+    inputs:
+    - name: redisImage
+      resourceRef:
+        name: redisstagingimage
+    - name: guestbookImage
+      resourceRef:
+        name: guestbookstagingimage
+    - name: workspace
+      resourceRef:
+        name: guestbook-resources-redis-docker
+    - name: testCluster
+      resourceRef:
+        name: testcluster

examples/invocations/kritis-pipeline-run.yaml

@@ -10,3 +10,34 @@ spec:
     name: pipelineparams-sample
   triggerRef:
     type: manual
+  resources:
+  - name: unit-test-kritis
+    inputs:
+    - key: workspace
+      resourceRef:
+        name: kritis-resources-git
+  - name: push-kritis
+    inputs:
+      - key: workspace
+        resourceRef:
+          name: kritis-resources-git
+    outputs:
+      - key: builtImage
+        resourceRef:
+          name: kritis-resources-image
+  - name: deploy-test-env
+    inputs:
+    - name: workspace
+      resourceRef:
+        name: kritis-resources-git
+    - name: builtImage
+      resourceRef:
+        name: kritis-resources-image
+    - name: testCluster
+      resourceRef:
+        name: kritistestcluster
+  - name: integration-test
+    inputs:
+    - name: workspace
+      resourceRef:
+          name: kritis-resources-git

examples/pipelines/guestbook.yaml

@@ -5,103 +5,61 @@ metadata:
   namespace: default
 spec:
   tasks:
-  - name: build-guestbook                  # 1.a Build guestbook go sample code.
+  - name: build-guestbook
     taskRef:
       name: build-push
-    inputSourceBindings:
-    - name: workspace
-      resourceRef:
-        name: guestbook-resources-git
-    outputSourceBindings:
-    - name: builtImage
-      resourceRef:
-        name: guestbookstagingimage
     params:
     - name: pathToDockerfile
       value: guestbook-go/Dockerfile
-  - name: build-redis                      # 1.b Build and push redis docker image.
+  - name: build-redis
     taskRef:
       name: build-push
-    inputSourceBindings:
-    - name: workspace
-      resourceRef:
-        name: guestbook-resources-redis-docker
-    outputSourceBindings:
-    - name: builtImage
-      resourceRef:
-        name: redisstagingimage
     params:
     - name: pathToDockerfile
       value: 4/debian9/4.0/Dockerfile
-  - name: deploy-bundle-test               # 2. Deploy GuestBook and Redis to test cluster
+  - name: deploy-bundle-test
     taskRef:
       name: deploy-with-kubectl
-    inputSourceBindings:
+    resources:
     - name: imageToDeploy1
-      resourceRef:
-        name: redisstagingimage
       providedBy:
       - build-redis
     - name: imageToDeploy2
-      resourceRef:
-        name: guestbookstagingimage
       providedBy:
-      - build-guestbook  
-    - name: workspace
-      resourceRef:
-        name: guestbook-resources-redis-docker
-    - name: testCluster
-      resourceRef:
-        name: testcluster
+      - build-guestbook
     params:
     - name: pathToFiles
       value: guestbook/all-in-one/guestbook-all-in-one.yaml
-  - name: int-test-osx                    # 3.a Run Integration tests for osx
-    taskRef:
-      name: integration-test-in-docker
-    inputSourceBindings:
+  - name: int-test-osx
+    resources:
     - name: workspace
-      resourceRef:
-        name: guestbook-resources-git
       providedBy:
       - deploy-bundle-test
     params:
     - name: dockerBuildFile
       value: guestbook-int/Dockerfile
-  - name: int-test-linux                  # 3.b Run Integration tests for linux
+  - name: int-test-linux
     taskRef:
       name: integration-test-in-docker
-    inputSourceBindings:
+    resources:
     - name: workspace
-      resourceRef:
-        name: guestbook-resources-git
       providedBy:
       - deploy-bundle-test
     params:
     - name: dockerBuildFile
       value: guestbook-int/Dockerfile
-  - name: deploy-bundle-test               # 4. Deploy GuestBook and Redis to staging cluster
+  - name: deploy-bundle-test
     taskRef:
       name: deploy-with-kubectl
-    inputSourceBindings:
+    resources:
     - name: redisImage
-      resourceRef:
-        name: redisstagingimage
       providedBy:
       - int-test-osx
       - int-test-linux
     - name: guestbookImage
-      resourceRef:
-        name: guestbookstagingimage
       providedBy:
       - int-test-osx
       - int-test-linux
-    - name: workspace
-      resourceRef:
-        name: guestbook-resources-redis-docker
-    - name: testCluster
-      resourceRef:
-        name: testcluster        
     params:
     - name: pathToFiles
       value: guestbook/all-in-one/guestbook-all-in-one.yaml