[TEP-0075] Support Dictionary in Params #4786

chuangw6 · 2022-04-22T05:03:36Z

Changes

/kind feature

Note: This new feature is gated by the alpha flag.

This PR serves as an initial implementation of [TEP-0075] which is part of #4723 . Highlights:

Add dictionary type

Add properties section in ParamSpec struct (key:string, val: PropertySpec struct)
Add ObjectVal in ArrayOrString struct (needs to rename this struct in future pr)
Implement UnmarshalJSON and MarshalJSON for ArrayOrString of object type.

Mutating webhook (SetDefault)

Infer ParamSpec type if properties section is provided
Infer ParamSpec type if objectVal is provided in default
Set default type (string) for PropertySpec if properties is provided and PropertySpec's type field is not provided.

Task/TaskRun level Validation

ParamSpec Type validation (for object type)
PropertySpec Type validation
Object type param definition validation i.e. properties is provided, required keys are provided etc.
Object param's names and its key names are referenced correctly

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

Docs included if any changes are user facing
Tests included if any functionality added or changed
Follows the commit message standard
Meets the Tekton contributor standards (including
functionality, content, code)
Release notes block below has been filled in
(if there are no user facing changes, use release note "NONE")

Release Notes

Add support for dictionary in Params values (and Results as well). This should affect *backward* compatibilities *but* it will break forward compatibilities in some cases (a.k.a. old client with new pipeline version)
action required: This changes the `ArrayOrStruct` structure, which will have effect on project using the go API as a library.

tekton-robot · 2022-04-22T05:06:07Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/param_types.go	98.1%	96.6%	-1.6
pkg/apis/pipeline/v1beta1/task_validation.go	97.1%	90.9%	-6.2
pkg/reconciler/pipelinerun/resources/apply.go	100.0%	96.3%	-3.7
pkg/reconciler/taskrun/resources/apply.go	99.2%	96.2%	-3.0

tekton-robot · 2022-04-22T17:28:47Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/param_types.go	98.1%	96.6%	-1.6
pkg/apis/pipeline/v1beta1/task_validation.go	97.1%	90.9%	-6.2
pkg/reconciler/pipelinerun/resources/apply.go	100.0%	96.3%	-3.7
pkg/reconciler/taskrun/resources/apply.go	99.2%	96.2%	-3.0

tekton-robot · 2022-04-25T22:49:08Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/param_types.go	98.1%	94.9%	-3.2
pkg/apis/pipeline/v1beta1/task_validation.go	97.1%	90.9%	-6.2
pkg/reconciler/pipelinerun/resources/apply.go	100.0%	96.3%	-3.7
pkg/reconciler/taskrun/resources/apply.go	99.2%	96.2%	-3.0
pkg/reconciler/taskrun/validate_resources.go	95.7%	96.4%	0.7

tekton-robot · 2022-04-26T04:10:32Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/param_types.go	98.1%	94.9%	-3.2
pkg/apis/pipeline/v1beta1/task_validation.go	97.1%	90.9%	-6.2
pkg/reconciler/pipelinerun/resources/apply.go	100.0%	96.3%	-3.7
pkg/reconciler/taskrun/resources/apply.go	99.2%	96.2%	-3.0
pkg/reconciler/taskrun/validate_resources.go	95.7%	96.4%	0.7

tekton-robot · 2022-04-26T05:00:26Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/param_types.go	98.1%	96.6%	-1.5
pkg/apis/pipeline/v1beta1/task_validation.go	97.1%	97.4%	0.3
pkg/reconciler/pipelinerun/resources/apply.go	100.0%	96.3%	-3.7
pkg/reconciler/taskrun/resources/apply.go	99.2%	96.2%	-3.0
pkg/reconciler/taskrun/validate_resources.go	95.7%	96.4%	0.7

tekton-robot · 2022-04-29T16:24:14Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/param_types.go	98.1%	96.6%	-1.5
pkg/apis/pipeline/v1beta1/task_validation.go	97.1%	97.4%	0.3
pkg/reconciler/pipelinerun/resources/apply.go	100.0%	96.3%	-3.7
pkg/reconciler/taskrun/resources/apply.go	99.2%	96.2%	-3.0
pkg/reconciler/taskrun/validate_resources.go	95.7%	96.4%	0.7

pkg/apis/pipeline/v1beta1/param_types.go

ywluogg · 2022-04-29T18:25:25Z

/assign @ywluogg

tekton-robot · 2022-05-01T20:15:12Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/param_types.go	98.1%	98.4%	0.3
pkg/apis/pipeline/v1beta1/task_validation.go	97.0%	96.8%	-0.2
pkg/reconciler/pipelinerun/resources/apply.go	100.0%	96.3%	-3.7
pkg/reconciler/taskrun/resources/apply.go	99.2%	96.2%	-3.0
pkg/reconciler/taskrun/validate_resources.go	95.7%	96.4%	0.7

tekton-robot · 2022-05-01T20:18:38Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/param_types.go	98.1%	98.4%	0.3
pkg/apis/pipeline/v1beta1/task_validation.go	97.0%	96.8%	-0.2
pkg/reconciler/pipelinerun/resources/apply.go	100.0%	96.3%	-3.7
pkg/reconciler/taskrun/resources/apply.go	99.2%	96.2%	-3.0
pkg/reconciler/taskrun/validate_resources.go	95.7%	96.4%	0.7

tekton-robot · 2022-05-01T20:23:37Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/param_context.go	96.3%	94.6%	-1.7
pkg/apis/pipeline/v1beta1/param_types.go	98.1%	98.4%	0.3
pkg/apis/pipeline/v1beta1/task_validation.go	97.0%	96.8%	-0.2
pkg/reconciler/pipelinerun/resources/apply.go	100.0%	96.3%	-3.7
pkg/reconciler/taskrun/resources/apply.go	99.2%	96.2%	-3.0
pkg/reconciler/taskrun/validate_resources.go	95.7%	96.4%	0.7

tekton-robot · 2022-05-01T23:26:06Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/param_context.go	96.3%	95.1%	-1.2
pkg/apis/pipeline/v1beta1/param_types.go	98.1%	98.5%	0.3
pkg/apis/pipeline/v1beta1/task_validation.go	97.0%	96.8%	-0.2
pkg/reconciler/pipelinerun/resources/apply.go	100.0%	96.3%	-3.7
pkg/reconciler/taskrun/resources/apply.go	99.2%	96.2%	-3.0
pkg/reconciler/taskrun/validate_resources.go	95.7%	96.4%	0.7

tekton-robot · 2022-05-01T23:32:03Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/param_context.go	96.3%	95.1%	-1.2
pkg/apis/pipeline/v1beta1/param_types.go	98.1%	98.5%	0.3
pkg/apis/pipeline/v1beta1/task_validation.go	97.0%	96.8%	-0.2
pkg/reconciler/pipelinerun/resources/apply.go	100.0%	96.3%	-3.7
pkg/reconciler/taskrun/resources/apply.go	99.2%	96.2%	-3.0
pkg/reconciler/taskrun/validate_resources.go	95.7%	96.4%	0.7

tekton-robot · 2022-05-01T23:36:45Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/param_context.go	96.3%	95.1%	-1.2
pkg/apis/pipeline/v1beta1/param_types.go	98.1%	98.5%	0.3
pkg/apis/pipeline/v1beta1/task_validation.go	97.0%	96.8%	-0.2
pkg/reconciler/pipelinerun/resources/apply.go	100.0%	96.3%	-3.7
pkg/reconciler/taskrun/resources/apply.go	99.2%	96.2%	-3.0
pkg/reconciler/taskrun/validate_resources.go	95.7%	96.4%	0.7

tekton-robot · 2022-05-01T23:46:08Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/param_context.go	96.3%	95.1%	-1.2
pkg/apis/pipeline/v1beta1/param_types.go	98.1%	98.5%	0.3
pkg/apis/pipeline/v1beta1/task_validation.go	97.0%	96.8%	-0.2
pkg/reconciler/taskrun/validate_resources.go	95.7%	96.4%	0.7

chuangw6 · 2022-05-10T18:59:31Z

/retest

chuangw6 · 2022-05-10T19:42:59Z

/test pull-tekton-pipeline-go-coverage

tekton-robot · 2022-05-10T20:05:02Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/param_context.go	96.3%	94.7%	-1.6
pkg/apis/pipeline/v1beta1/param_types.go	98.1%	96.9%	-1.3
pkg/apis/pipeline/v1beta1/task_validation.go	97.0%	97.5%	0.5
pkg/reconciler/taskrun/validate_resources.go	96.0%	95.9%	-0.1

chuangw6 · 2022-05-10T21:17:02Z

/test pull-tekton-pipeline-integration-tests

tekton-robot · 2022-05-11T16:29:50Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/param_context.go	96.3%	94.7%	-1.6
pkg/apis/pipeline/v1beta1/param_types.go	98.1%	96.9%	-1.3
pkg/apis/pipeline/v1beta1/task_validation.go	97.0%	97.5%	0.5
pkg/reconciler/taskrun/validate_resources.go	95.3%	93.7%	-1.6

chuangw6 · 2022-05-11T16:58:26Z

/test pull-tekton-pipeline-alpha-integration-tests

dibyom

Thanks @chuangw6

Have a couple of minor comments around TODOs but otherwise LGTM!

pkg/apis/pipeline/v1beta1/param_types.go

pkg/apis/pipeline/v1beta1/task_validation.go

pkg/reconciler/taskrun/validate_resources.go

tekton-robot · 2022-05-11T17:23:57Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dibyom, ywluogg

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [dibyom]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Note: This new feature is gated by the alpha flag. Changes: - Add `properties` section in ParamSpec struct - Add `ObjectVal` in ArrayOrString struct - Implement UnmarshalJSON and MarshalJSON for ArrayOrString of object type. - Change SetDefault function to include object type - Add type validation for object type in task_validation.go - Add object param validation in validate_resources.go

tekton-robot · 2022-05-11T18:21:19Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/param_context.go	96.3%	94.7%	-1.6
pkg/apis/pipeline/v1beta1/param_types.go	98.1%	96.9%	-1.3
pkg/apis/pipeline/v1beta1/task_validation.go	97.0%	97.5%	0.5
pkg/reconciler/taskrun/validate_resources.go	95.3%	93.7%	-1.6

dibyom · 2022-05-11T18:55:21Z

/lgtm

All comments have been addressed

pritidesai · 2022-05-31T17:09:04Z

I am sorry to bring this up @wlynch @dibyom 🙏

This PR is introducing a new feature and was merged without release notes. Please make sure we check the PR description before its merged!

vdemeester · 2022-06-01T07:08:11Z

I've updated the description to add a release-note block.

chuangw6 · 2022-06-01T15:07:07Z

I've updated the description to add a release-note block.

Thanks @vdemeester !

This PR is introducing a new feature and was merged without release notes. Please make sure we check the PR description before its merged!

Thanks! @pritidesai I'll keep in mind to include release note for new features added in future.

tekton-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Apr 22, 2022

chuangw6 marked this pull request as draft April 22, 2022 05:03

tekton-robot requested review from pritidesai and vdemeester April 22, 2022 05:03

tekton-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 25, 2022

ywluogg reviewed Apr 29, 2022

View reviewed changes

pkg/apis/pipeline/v1beta1/param_types.go Show resolved Hide resolved

tekton-robot assigned ywluogg Apr 29, 2022

chuangw6 force-pushed the object-param-result branch from 7f61593 to 8116469 Compare May 1, 2022 20:16

chuangw6 force-pushed the object-param-result branch from 8116469 to 955f414 Compare May 1, 2022 20:21

tekton-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels May 1, 2022

chuangw6 force-pushed the object-param-result branch from c8f6c2b to 9f28404 Compare May 1, 2022 23:29

chuangw6 force-pushed the object-param-result branch from 9f28404 to dab9308 Compare May 1, 2022 23:34

chuangw6 force-pushed the object-param-result branch from dab9308 to 3a486ee Compare May 1, 2022 23:43

chuangw6 marked this pull request as ready for review May 1, 2022 23:53

tekton-robot removed the lgtm Indicates that a PR is ready to be merged. label May 10, 2022

ywluogg approved these changes May 10, 2022

View reviewed changes

chuangw6 force-pushed the object-param-result branch from 37adff2 to b1bfccf Compare May 11, 2022 16:22

dibyom approved these changes May 11, 2022

View reviewed changes

pkg/apis/pipeline/v1beta1/param_types.go Outdated Show resolved Hide resolved

pkg/apis/pipeline/v1beta1/task_validation.go Outdated Show resolved Hide resolved

pkg/reconciler/taskrun/validate_resources.go Outdated Show resolved Hide resolved

tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 11, 2022

chuangw6 force-pushed the object-param-result branch from b1bfccf to fc36bfb Compare May 11, 2022 18:18

tekton-robot assigned dibyom May 11, 2022

tekton-robot added the lgtm Indicates that a PR is ready to be merged. label May 11, 2022

tekton-robot merged commit 1c8447d into tektoncd:main May 11, 2022

ywluogg mentioned this pull request May 25, 2022

Add a new struct to replace ArrayOrString struct #4858

Closed

tekton-robot added release-note-action-required Denotes a PR that introduces potentially breaking changes that require user action. and removed release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Jun 1, 2022

michaelsauter mentioned this pull request Jun 9, 2022

Reconsider "pipeline context" when dictionary params / results are available opendevstack/ods-pipeline#556

Open

bradbeck mentioned this pull request Jun 9, 2022

Updated Tekton Pipeline to v0.36.0 buildsec/frsca#237

Merged

ywluogg mentioned this pull request Jun 22, 2022

Add ywluogg into Pipeline reviewer roles tektoncd/community#737

Merged

Yongxuanzhang mentioned this pull request Jul 18, 2022

Unable to use array parameter substitution in pipeline #5149

Closed

Yongxuanzhang mentioned this pull request Sep 27, 2022

Mark tep75&76 implemented tektoncd/community#833

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[TEP-0075] Support Dictionary in Params #4786

[TEP-0075] Support Dictionary in Params #4786

chuangw6 commented Apr 22, 2022 •

edited by vdemeester

Loading

tekton-robot commented Apr 22, 2022

tekton-robot commented Apr 22, 2022

tekton-robot commented Apr 25, 2022

tekton-robot commented Apr 26, 2022

tekton-robot commented Apr 26, 2022

tekton-robot commented Apr 29, 2022

ywluogg commented Apr 29, 2022

tekton-robot commented May 1, 2022

tekton-robot commented May 1, 2022

tekton-robot commented May 1, 2022

tekton-robot commented May 1, 2022

tekton-robot commented May 1, 2022

tekton-robot commented May 1, 2022

tekton-robot commented May 1, 2022

chuangw6 commented May 10, 2022

chuangw6 commented May 10, 2022

tekton-robot commented May 10, 2022

chuangw6 commented May 10, 2022

tekton-robot commented May 11, 2022

chuangw6 commented May 11, 2022

dibyom left a comment

tekton-robot commented May 11, 2022

tekton-robot commented May 11, 2022

dibyom commented May 11, 2022

pritidesai commented May 31, 2022

vdemeester commented Jun 1, 2022

chuangw6 commented Jun 1, 2022 •

edited

Loading

[TEP-0075] Support Dictionary in Params #4786

[TEP-0075] Support Dictionary in Params #4786

Conversation

chuangw6 commented Apr 22, 2022 • edited by vdemeester Loading

Changes

Add dictionary type

Mutating webhook (SetDefault)

Task/TaskRun level Validation

Submitter Checklist

Release Notes

tekton-robot commented Apr 22, 2022

tekton-robot commented Apr 22, 2022

tekton-robot commented Apr 25, 2022

tekton-robot commented Apr 26, 2022

tekton-robot commented Apr 26, 2022

tekton-robot commented Apr 29, 2022

ywluogg commented Apr 29, 2022

tekton-robot commented May 1, 2022

tekton-robot commented May 1, 2022

tekton-robot commented May 1, 2022

tekton-robot commented May 1, 2022

tekton-robot commented May 1, 2022

tekton-robot commented May 1, 2022

tekton-robot commented May 1, 2022

chuangw6 commented May 10, 2022

chuangw6 commented May 10, 2022

tekton-robot commented May 10, 2022

chuangw6 commented May 10, 2022

tekton-robot commented May 11, 2022

chuangw6 commented May 11, 2022

dibyom left a comment

Choose a reason for hiding this comment

tekton-robot commented May 11, 2022

tekton-robot commented May 11, 2022

dibyom commented May 11, 2022

pritidesai commented May 31, 2022

vdemeester commented Jun 1, 2022

chuangw6 commented Jun 1, 2022 • edited Loading

chuangw6 commented Apr 22, 2022 •

edited by vdemeester

Loading

chuangw6 commented Jun 1, 2022 •

edited

Loading