Upgrade to Tekton version 0.14.0 (#216)

README.md

@@ -15,7 +15,7 @@ has Steps. Please look for more details in [Tekton repo](https://github.com/tekt
 ## Kubeflow Pipeline with Tekton backend
 
 We are currently using [Kubeflow Pipelines 0.5.1](https://github.com/kubeflow/pipelines/releases/tag/0.5.1) and
-[Tekton 0.13.0](https://github.com/tektoncd/pipeline/releases/tag/v0.13.0) for this project.
+[Tekton 0.14.0](https://github.com/tektoncd/pipeline/releases/tag/v0.14.0) for this project.
 
 ![kfp-tekton](images/kfp-tekton.png)
 

samples/README.md

@@ -8,7 +8,7 @@ If you are interested more in the larger list of pipelines samples we are testin
 
 ## Prerequisites 
 - Install [Kubeflow 1.0.2+](https://www.kubeflow.org/docs/started/getting-started/) and connect the cluster to the current shell with `kubectl`
-- Install [Tekton 0.13.0](https://github.com/tektoncd/pipeline/releases/tag/v0.13.0)
+- Install [Tekton 0.14.0](https://github.com/tektoncd/pipeline/releases/tag/v0.14.0)
     - For KFP, we shouldn't be modifying the default work directory for any component. Therefore, please run the below command to disable the [home and work directories overwrite](https://github.com/tektoncd/pipeline/blob/master/docs/install.md#customizing-the-pipelines-controller-behavior) from Tekton default.
         ```shell
         kubectl patch cm feature-flags -n tekton-pipelines -p '{"data":{"disable-home-env-overwrite":"true","disable-working-directory-overwrite":"true"}}'

sdk/FEATURES.md

@@ -1,8 +1,11 @@
-# KFP Tekton compiler features
+# KFP-Tekton Compiler Features
 
-Below are the list of features that are currently available in the KFP Tekton compiler along with its implementation. With the current implementation, we are getting a greater than [80% success rate for compilation tests we are running over approximately 90 pipelines](/sdk/python/tests/README.md) spread across different parts of the Kubeflow Pipeline samples and tests. 
+This document describes the features supported by the _KFP-Tekton_ compiler.
+With the current implementation, the _KFP-Tekton_ compiler is capable of
+[compiling 80%](/sdk/python/tests/README.md) of approximately 90 sample
+and test pipelines found in the KFP repository.
 
-- [Pipeline DSL features with native Tekton implementation](#pipeline-dsl-features-with-native-tekton-implementation)
+- [Pipeline DSL Features with Native Tekton Implementation](#pipeline-dsl-features-with-native-tekton-implementation)
     + [pod_annotations and pod_labels](#pod_annotations-and-pod_labels)
     + [Retries](#retries)
     + [Volumes](#volumes)
@@ -12,43 +15,48 @@ Below are the list of features that are currently available in the KFP Tekton co
     + [ContainerOp](#containerop)
     + [Affinity, Node Selector, and Tolerations](#affinity-node-selector-and-tolerations)
     + [ImagePullSecrets](#imagepullsecrets)
-- [Pipeline DSL features with custom Tekton implementation](#pipeline-dsl-features-with-custom-tekton-implementation)
-  * [Features with same behavior as Argo](#features-with-same-behavior-as-argo)
+    + [Exit Handler](#exit-handler)
+- [Pipeline DSL Features with a Custom Tekton Implementation](#pipeline-dsl-features-with-a-custom-tekton-implementation)
+  * [Features with the Same Behavior as Argo](#features-with-the-same-behavior-as-argo)
     + [InitContainers](#initcontainers)
     + [Conditions](#conditions)
     + [ResourceOp, VolumeOp, and VolumeSnapshotOp](#resourceop-volumeop-and-volumesnapshotop)
     + [Output Parameters](#output-parameters)
     + [Input Artifacts](#input-artifacts)
     + [Output Artifacts](#output-artifacts)
-  * [Features with limitations](#features-with-limitations)
+  * [Features with Limitations](#features-with-limitations)
     + [ParallelFor](#parallelfor) - [Tracking issue][ParallelFor]
     + [Variable Substitutions](#variable-substitutions) - [Tracking issue][VarSub]
-  * [Features with different behavior than Argo](#features-with-different-behavior-than-argo)
+  * [Features with a Different Behavior than Argo](#features-with-a-different-behavior-than-argo)
     + [Sidecars](#sidecars) - [Tracking issue][Sidecars]
-- [Pipeline features that are unavailable on Tekton](#pipeline-features-that-are-unavailable-on-tekton)
-    + [Exit Handler](#exit-handler) - [Tracking PR][exitHandler]
 
-# Pipeline DSL features with native Tekton implementation
+
+# Pipeline DSL Features with Native Tekton Implementation
+
 Below are the features using Tekton's native support without any custom workaround.
 
 ### pod_annotations and pod_labels
-pod_annotations and pod_labels are for assigning custom annotations or labels to a pipeline component. They are implemented with
+
+`pod_annotations` and `pod_labels` are for assigning custom annotations or labels to a pipeline component. They are implemented with
 Tekton's [task metadata](https://github.com/tektoncd/pipeline/blob/master/docs/tasks.md#configuring-a-task) field under Tekton
-Task. The [pipeline transformers](/sdk/python/tests/compiler/testdata/transfromers.py) example shows how to apply
+Task. The [pipeline transformers](/sdk/python/tests/compiler/testdata/pipeline_transfromers.py) example shows how to apply
 custom annotations and labels to one or more components in the pipeline. 
 
 ### Retries
+
 Retry feature allows users to specify the number of times a particular component should retry its execution when it fails. It is
 implemented with Tekton's [task spec](https://github.com/tektoncd/pipeline/blob/master/docs/pipelines.md#using-the-retries-parameter)
 under Tekton Pipeline. The [retry](/sdk/python/tests/compiler/testdata/retry.py) python test is an example of how to use
 this feature.
 
 ### Volumes
+
 Volumes are for mounting existing Kubernetes resources onto the components. It is implemented with Tekton's
 [task volumes](https://github.com/tektoncd/pipeline/blob/master/docs/tasks.md#specifying-volumes) feature under Tekton Task. The
 [volume](/sdk/python/tests/compiler/testdata/volume.py) python test is an example of how to use this feature.
 
 ### Timeout for Tasks and Pipelines
+
 Timeout can be used for setting the amount of time allowed on executing a component within the Pipeline or setting the amount of
 time allowed on executing the whole pipeline. The task timeout is implemented with Tekton's
 [task failure timeout](https://github.com/tektoncd/pipeline/blob/master/docs/pipelines.md#configuring-the-failure-timeout) under
@@ -58,18 +66,21 @@ under Tekton PipelineRun. The [timeout](/sdk/python/tests/compiler/testdata/time
 how to use this feature.
 
 ### RunAfter
+
 RunAfter is for indicating that a component must execute after one or more other components. It is implemented with Tekton's
 [runAfter](https://github.com/tektoncd/pipeline/blob/master/docs/pipelines.md#using-the-runafter-parameter) feature under Tekton
 Pipeline. The [sequential](/sdk/python/tests/compiler/testdata/sequential.py) python test is an example of how to use this
 feature.
 
 ### Input Parameters
+
 Input Parameters are for passing pipeline parameters or other component outputs to the next running component. It is implemented
 with Tekton's [parameters](https://github.com/tektoncd/pipeline/blob/master/docs/tasks.md#specifying-parameters) features under Tekton
 task. The [parallel_join](/sdk/python/tests/compiler/testdata/parallel_join.py) python test is an example of how to use this
 feature.
 
 ### ContainerOp
+
 ContainerOp defines the container spec for a pipeline component. It is implemented with Tekton's
 [steps](https://github.com/tektoncd/pipeline/blob/master/docs/tasks.md#defining-steps) features under Tekton task. The generated
 Tekton task name is the same as the containerOp name whereas the step name is always called "main". The
@@ -84,37 +95,52 @@ The [affinity](/sdk/python/tests/compiler/testdata/affinity.py),
 [node_selector](/sdk/python/tests/compiler/testdata/node_selector.py), and
 [tolerations](/sdk/python/tests/compiler/testdata/tolerations.py) Python tests are examples of how to use these features.
 
-This feature is recently implemented in Tekton and is available on Tekton v0.13.0 onwards.
+This feature has been implemented in Tekton version `0.13.0`.
 
 ### ImagePullSecrets
 
 ImagePullSecret is a feature for the components to know which secret to use when pulling container images from private registries. It is implemented
 with Tekton's [podTemplate](https://github.com/tektoncd/pipeline/blob/master/docs/podtemplates.md) field under Tekton
-PipelineRun. The [imagepullsecrets](/sdk/python/tests/compiler/testdata/imagepullsecrets.py) python test is an example of how to use this
+PipelineRun. The [imagepullsecrets](/sdk/python/tests/compiler/testdata/imagepullsecrets.py) Python test is an example of how to use this
 feature.
 
-This feature is recently implemented in Tekton and is available on Tekton v0.13.0 onwards.
+This feature has been available since Tekton version `0.13.0`.
+
+### Exit Handler
+
+An _exit handler_ is a component that always executes, irrespective of success or failure,
+at the end of the pipeline. It is implemented using Tekton's 
+[finally](https://github.com/tektoncd/pipeline/blob/v0.14.0/docs/pipelines.md#adding-finally-to-the-pipeline) 
+section under the Pipeline `spec`. An example of how to use an _exit handler_ can be found in
+the [exit_handler](/sdk/python/tests/compiler/testdata/exit_handler.py) compiler test.
+
+The `finally` syntax is supported since Tekton version `0.14.0`.
+
+
+# Pipeline DSL Features with a Custom Tekton Implementation
 
-# Pipeline DSL features with custom Tekton implementation
+## Features with the Same Behavior as Argo
 
-## Features with same behavior as Argo
 Below are the features that don't have one to one mapping to Tekton's native implementation, but the same behaviors can be replicated with
 extra custom processing code or workaround within the compiler.
 
 ### InitContainers
+
 InitContainers are containers that executed before the main component within the same pod. Since Tekton already has the concept of task
 steps, initContainers are placed as [steps](https://github.com/tektoncd/pipeline/blob/master/docs/tasks.md#defining-steps) before the
 main component task under Tekton Task. The [init_container](/sdk/python/tests/compiler/testdata/init_container.py) python test
 is an example of how to use this feature.
 
 ### Conditions
+
 Conditions are for determining whether to execute certain components based on the output of the condition checks. Since Tekton required users to define an image for doing the [condition check](https://github.com/tektoncd/pipeline/blob/master/docs/conditions.md), we created a custom python image to replicate the same condition checks from Argo and made it as the default in our compiler. The
 [flip-coin](/samples/flip-coin) example demonstrates how to use multiple conditions within the same pipeline.
 
 Please be aware that the current Condition feature is using Tekton V1alpha1 API because the Tekton community is still designing the V1beta1 API.
 We will be migrating to the V1beta1 API once it's available in Tekton. Please refer to the [design proposal](https://docs.google.com/document/d/1kESrgmFHnirKNS4oDq3mucuB_OycBm6dSCSwRUHccZg/edit?usp=sharing) for more details.
 
 ### ResourceOp, VolumeOp, and VolumeSnapshotOp
+
 [ResourceOp, VolumeOp, and VolumeSnapshotOp](https://www.kubeflow.org/docs/pipelines/sdk/manipulate-resources/) are special operations for
 creating Kubernetes resources on the pipeline cluster. ResourceOp is a basic operation for manipulating any Kubernetes resource. VolumeOp
 and VolumeSnapshotOp are operations for creating a unique Volume/VolumeSnapshot per pipeline and can be used as volume/snapshot with any pipeline ContainerOp. Because Tekton has no support for these features natively, we have to implement our own custom task called kubectl-wrapper for creating these Kubernetes resources. The [resourceop_basic](/sdk/python/tests/compiler/testdata/resourceop_basic.py),
@@ -126,22 +152,27 @@ One thing to pay attention to is that VolumeSnapshot won't be available by defau
 in order to use VolumeSnapshotOp.
 
 ### Output Parameters
+
 Output parameters are a dictionary of string files that users can define as a component's outputs. In Tekton, we implemented this with Tekton [task results](https://github.com/tektoncd/pipeline/blob/master/docs/tasks.md#storing-execution-results) under Tekton Task. Since Tekton task results are only able to output parameters to its `/tekton/results` directory, we add an extra ending step to copy any non-Tekton user output to the `/tekton/results` directory. The [parallel_join](/sdk/python/tests/compiler/testdata/parallel_join.py) python test is an example of how to use this feature.
 
 ### Input Artifacts
+
 Input Artifacts in Kubeflow pipelines are used for passing raw text or local files as files placed in the component pod. Since Input Artifacts can only be raw or in a compressed format as strings, we created a
 [custom step](https://github.com/kubeflow/kfp-tekton/blob/master/sdk/python/kfp_tekton/compiler/_op_to_template.py#L435) for passing these strings as files before the main task is executed. The [input_artifact_raw_value](/sdk/python/tests/compiler/testdata/input_artifact_raw_value.py)
 python test is an example of how to use this feature.
 
 ### Output Artifacts
+
 Output Artifacts are files that need to be persisted to the default/destination object storage. Additionally, by default, all Kubeflow pipeline 'Output Parameters' are also stored as output artifacts. Since Tekton deprecated pipelineResource and the recommended gsutil task is not capable of moving files to the minio object storage without proper DNS address, we decided to create a step based on the [minio mc](https://github.com/minio/mc) image for moving output artifacts. This feature also includes the ArtifactLocation support where users can set their own object storage credentials during execution. The [artifact_location](/sdk/python/tests/compiler/testdata/artifact_location.py) python test is an example of how to use this feature.
 
 It also includes two annontations `tekton.dev/input_artifacts` and `tekton.dev/output_artifacts` for metadata tracking. Refer to the
 [Tetkon Artifact design proposal](http://bit.ly/kfp-tekton) for more details. 
 
 The current implementation is relying on the existing KFP's minio setup for getting the default credentials. This feature probably needs to be deprecated and merged with the output parameters once KFP finalizes the artifact management for the multi-user scenario. 
 
-## Features with limitations
+
+## Features with Limitations
+
 Below are the features that have certain limitation in the current Tekton release.
 
 ### ParallelFor
@@ -168,7 +199,9 @@ argo -> tekton
 [parallel_join_with_argo_vars](/sdk/python/tests/compiler/testdata/parallel_join_with_argo_vars.py) is an example of how Argo variables are
 used and it can still be converted to Tekton variables with our Tekton compiler. However, other Argo variables will throw out an error because those Argo variables are very unique to Argo's pipeline system. 
 
-## Features with different behavior than Argo
+
+## Features with a Different Behavior than Argo
+
 Below are the KFP-Tekton features with different behavior than Argo.
 
 ### Sidecars
@@ -183,22 +216,12 @@ However, when you run kfp-tekton pipeline with sidecars, you may notice a comple
 
 When the nop image does provide the sidecar's command, the sidecar will continue to run even after nop has been swapped into the sidecar container's image field. Until this issue is resolved the best way to avoid it is to avoid overriding the nop image when deploying the tekton controller, or ensuring that the overridden nop image contains as few commands as possible.
 
-kubectl get pods will show a 'Completed' pod when a sidecar exits successfully but an Error when the sidecar exits with an error. This is only apparent when using kubectl to get the pods of a TaskRun, not when describing the Pod using kubectl describe pod ... nor when looking at the TaskRun, but can be quite confusing. However, it has no functional impact. 
+`kubectl get pods` will show a 'Completed' pod when a sidecar exits successfully but an _Error_ when the sidecar exits with an error. This is only apparent when using `kubectl` to get the pods of a TaskRun, not when describing the Pod using `kubectl describe pod ...` nor when looking at the TaskRun, but can be quite confusing. However, it has no functional impact. 
 [Tekton pipeline readme](https://github.com/tektoncd/pipeline/blob/master/docs/developers/README.md#handling-of-injected-sidecars) has documented this limitation. 
 
-# Pipeline features that are unavailable on Tekton
-
-### Exit Handler
-
-An exit handler is a component that always executes, irrespective of success or failure, at the end of the pipeline.
-
-The `finally` syntax is now supported in Tekton preview mode. The final tasks can be specified but not executed yet,
-implementation and execution of final tasks are tracked in separate PR [Tracking pull request #2661][exitHandler]
-
 
 <!-- Issue and PR links-->
 
 [ParallelFor]: https://github.com/tektoncd/pipeline/issues/2050
 [VarSub]: https://github.com/tektoncd/pipeline/issues/1522
 [Sidecars]: https://github.com/tektoncd/pipeline/issues/1347
-[exitHandler]: https://github.com/tektoncd/pipeline/pull/2661

sdk/README.md

@@ -50,7 +50,7 @@ SDK provides a `TektonCompiler` and a `TektonClient`:
 ## Project Prerequisites
 
  - Python: `3.5.3` or later
- - Tekton: [`0.13.0`](https://github.com/tektoncd/pipeline/releases/tag/v0.13.0)
+ - Tekton: [`0.14.0`](https://github.com/tektoncd/pipeline/releases/tag/v0.14.0)
  - Tekton CLI: [`0.10.0`](https://github.com/tektoncd/cli/releases/tag/v0.10.0)
  - Kubeflow Pipelines: [KFP with Tekton backend](/tekton_kfp_guide.md)
 

sdk/python/README.md

@@ -36,7 +36,7 @@ the [SDK README](/sdk/README.md)
 1. [`Python`](https://www.python.org/downloads/): version `3.5.3` or later (new code must maintain compatibility with `3.5`)
 2. [`Kubernetes` Cluster](https://v1-15.docs.kubernetes.io/docs/setup/): version `1.15` ([required by Kubeflow](https://www.kubeflow.org/docs/started/k8s/overview/) and Tekton 0.11)
 3. [`kubectl` CLI](https://kubernetes.io/docs/tasks/tools/install-kubectl/): required to deploy Tekton pipelines to Kubernetes cluster
-4. [`Tekton` Deployment](https://github.com/tektoncd/pipeline/releases/tag/v0.13.0/): version `0.13.0` (or greater to support Tekton API version `v1beta1`), required for end-to-end testing
+4. [`Tekton` Deployment](https://github.com/tektoncd/pipeline/releases/tag/v0.14.0/): version `0.14.0` or greater (minimum version `0.13.0` to support Tekton API version `v1beta1`), required for end-to-end testing
 5. [`tkn` CLI](https://github.com/tektoncd/cli#installing-tkn): required to work with Tekton pipelines
 6. [`Kubeflow Pipelines` Deployment](https://www.kubeflow.org/docs/pipelines/installation/overview/): required for some end-to-end tests
 
@@ -48,10 +48,10 @@ A working Tekton cluster deployment is required to perform end-to-end tests of t
 
 #### Tekton Cluster
 
-Follow the instructions listed [here](https://github.com/tektoncd/pipeline/blob/v0.13.0/docs/install.md#installing-tekton-pipelines-on-kubernetes)
+Follow the instructions listed [here](https://github.com/tektoncd/pipeline/blob/v0.14.0/docs/install.md#installing-tekton-pipelines-on-kubernetes)
 or simply run:
 
-    kubectl apply -f https://storage.googleapis.com/tekton-releases/pipeline/previous/v0.13.0/release.yaml
+    kubectl apply -f https://storage.googleapis.com/tekton-releases/pipeline/previous/v0.14.0/release.yaml
 
 **Note**, if your container runtime does not support image-reference:tag@digest (like cri-o used in OpenShift 4.x),
 use `release.notags.yaml` instead.

sdk/python/tests/compiler/testdata/exit_handler.log

@@ -1,4 +1,3 @@
-Pipeline still running ...
 [gcs-download : main] With which he yoketh your rebellious necks Razeth your cities and subverts your towns And in a moment makes them desolate
 
 [gcs-download : copy-artifacts] Added `storage` successfully.
@@ -8,5 +7,3 @@ Pipeline still running ...
 
 [echo-2 : main] With which he yoketh your rebellious necks Razeth your cities and subverts your towns And in a moment makes them desolate
 [echo-2 : main]
-
-[echo : main] exit!