Create Tekton task and Pipeline for Tekton bundles #284

zregvart · 2023-01-26T17:00:19Z

This adds tkn-bundle Tekton Task that simply finds all *.yaml or *.yml files in the current params.CONTEXT directory and creates a Tekton Bundle by pushing it to params.IMAGE. The result of the task are IMAGE_URL image reference with digest (no tag) and IMAGE_DIGEST containing the image digest.

Also adds tekton-bundle-builder builder pipeline that uses the tkn-bundle task for the build step.

Tests are provided.

Ref. https://issues.redhat.com/browse/HACBS-1675

robnester-rh

lgtm

pipelines/tekton-bundle-builder/patch.yaml

task/tkn-bundle/0.1/TESTING.md

simonbaird

Approve but consider my suggestion about the docs and extra comments.

simonbaird · 2023-01-26T17:32:28Z

I guess the tests might go stale unless there's some automation to run them. Would it be crazy to add a GitHub workflow in this repo with the kind cluster setup to exercise them?

zregvart · 2023-01-27T11:19:55Z

I guess the tests might go stale unless there's some automation to run them. Would it be crazy to add a GitHub workflow in this repo with the kind cluster setup to exercise them?

Entirely possible, let's do this as a followup: HACBS-1698

zregvart · 2023-01-27T14:44:31Z

Something is not setup correctly on redhat-appstudio-tekton-catalog/pull-request-builds, perhaps it is not public?

$ curl https://quay.io/api/v1/repository/redhat-appstudio-tekton-catalog/pull-request-builds:nodejs-builder-94b327bfea218b7ff71ef0a0cc73cbc278c8e888 
{"detail": "Requires authentication", "error_message": "Requires authentication", "error_type": "invalid_token", "title": "invalid_token", "type": "https://quay.io/api/v1/error/invalid_token", "status": 401}

vs.

$ curl https://quay.io/api/v1/repository/redhat-appstudio-tekton-catalog/pipeline-nodejs-builder                                                     
{"namespace": "redhat-appstudio-tekton-catalog", "name": "pipeline-nodejs-builder", "kind": "image", "description": "", "is_public": true,...

Also not sure why the tkn-bundle task is pushed with tkn-bundle-0.1-237b8766e22ddb3376f391d825adf511cb4a8bb6 and tkn-bundle-0.1 tags.

zregvart · 2023-01-27T14:59:28Z

Oh, I think I get it

zregvart · 2023-01-30T14:05:54Z

The check is now showing green. @Michkov and @tkdchen I would appreciate your feedback.

pipeline-bundle-list

.tekton/pull-request.yaml

lcarva · 2023-01-30T21:49:51Z

pipelines/tekton-bundle-builder/kustomization.yaml

+patches:
+# Use the template-build as a template replacing the Pipeline name and the
+# `build-container` step's task reference
+- patch: |-


NIT: Maybe worth setting these labels? pipelines.openshift.io/runtime, pipelines.openshift.io/strategy?

(although... I'm not quite sure how those are used)

Found no documentation on this, no way of telling what those need to be set to so I left them unset

lcarva · 2023-01-30T21:51:57Z

pipelines/tekton-bundle-builder/kustomization.yaml

+# Use the template-build as a template replacing the Pipeline name and the
+# `build-container` step's task reference
+- patch: |-
+    - op: replace


NIT: It might be worth loading a patch from its own yaml (like the other pipelines do). It's a bit easier to edit than an embedding yaml in yaml.

Not really, this documents the patch much better

task/tkn-bundle/0.1/tkn-bundle.yaml

Michkov

I am slightly confused about purpose of this pipeline, who is going to use it? Will it be provided to customers?

Michkov · 2023-01-31T07:45:03Z

task/tkn-bundle/0.1/tkn-bundle.yaml

+        && exit 1
+      exec 3>&1;
+      OUT="$(tkn bundle push "$(params.IMAGE)" \
+        $(printf ' -f %s' "${FILES[@]}") \


limitation of 10?

IMO, this is something that needs to be dealt with in Tekton. The task will fail with more than 20 (see this PR). I don't think it is worth taking additional complexity.

task/tkn-bundle/OWNERS

task/tkn-bundle/0.1/tkn-bundle.yaml

zregvart · 2023-01-31T09:56:20Z

I am slightly confused about purpose of this pipeline, who is going to use it? Will it be provided to customers?

This is a requirement for implementing SLSA build as code requirement (see HACBS-1282). We need to start somewhere and with time migrate or adapt our own pipelines in this repository to generate provenance information.

So, this is for customers who want to be SLSA level 3, and for the pipelines/tasks from this repository if we chose to do so as well.

.tekton/pull-request.yaml

tkdchen

The CI failed as commented above the oc command does not exist.

This adds `tkn-bundle` Tekton Task that simply finds all *.yaml or *.yml files in the current `params.CONTEXT` directory and creates a Tekton Bundle by pushing it to `params.IMAGE`. The result of the task are `IMAGE_URL` image reference with digest (no tag) and `IMAGE_DIGEST` containing the image digest. Also adds `tekton-bundle-builder` builder pipeline that uses the `tkn-bundle` task for the `build` step. Tests are provided. Ref. https://issues.redhat.com/browse/HACBS-1675

Passes the pull request commit id to the `check-task-pipeline-repo-existence` step to make the validation consider pull request image references; otherwise the check checks for images that are have resulted from changes pushed to the default branch. The URL being checked for existence now checks against the Docker registry API, the quay.io REST API has no endpoint to check for existence of tags. Also the non-pull request artifacts are checked for their correct tag/version now.

zregvart · 2023-02-01T10:28:26Z

Last pipeline run was successful. I want to do one more quality of life change: for faster feedback it would be better if check-task-pipeline-repo-existence ran after build-container and not in finally. Let me try to do that and then I think this should be okay to merge. Though do check for any bash issues, those are always lurking.

...r build-container

zregvart · 2023-02-01T10:39:56Z

@tkdchen @Michkov @lcarva can you have another look at this?

FYI @Michkov looks like registry.redhat.io/openshift4/ose-tools-rhel8 contains good amount of tools we usually need (oc, git, jq, bash...)

zregvart · 2023-02-01T15:52:24Z

Thanks for the reviews, merging now...

robnester-rh approved these changes Jan 26, 2023

View reviewed changes

simonbaird reviewed Jan 26, 2023

View reviewed changes

pipelines/tekton-bundle-builder/patch.yaml Outdated Show resolved Hide resolved

simonbaird reviewed Jan 26, 2023

View reviewed changes

task/tkn-bundle/0.1/TESTING.md Show resolved Hide resolved

simonbaird approved these changes Jan 26, 2023

View reviewed changes

zregvart force-pushed the issue/HACBS-1675 branch from bc57f22 to 9eec08d Compare January 27, 2023 11:14

zregvart force-pushed the issue/HACBS-1675 branch 4 times, most recently from dd49758 to 94b327b Compare January 27, 2023 12:46

zregvart force-pushed the issue/HACBS-1675 branch 4 times, most recently from a3b8b66 to e17bb0a Compare January 30, 2023 13:54

lcarva reviewed Jan 30, 2023

View reviewed changes

pipeline-bundle-list Outdated Show resolved Hide resolved

zregvart force-pushed the issue/HACBS-1675 branch from e17bb0a to c346b30 Compare January 30, 2023 14:13

lcarva reviewed Jan 30, 2023

View reviewed changes

Michkov reviewed Jan 31, 2023

View reviewed changes

zregvart force-pushed the issue/HACBS-1675 branch from c346b30 to f539cd1 Compare January 31, 2023 09:49

zregvart force-pushed the issue/HACBS-1675 branch from f539cd1 to e660c6a Compare January 31, 2023 16:43

tkdchen approved these changes Feb 1, 2023

View reviewed changes

tkdchen reviewed Feb 1, 2023

View reviewed changes

.tekton/pull-request.yaml Outdated Show resolved Hide resolved

tkdchen requested changes Feb 1, 2023

View reviewed changes

zregvart force-pushed the issue/HACBS-1675 branch 2 times, most recently from 4353ed5 to ff6b6ef Compare February 1, 2023 10:15

zregvart added 2 commits February 1, 2023 11:15

zregvart force-pushed the issue/HACBS-1675 branch from ff6b6ef to 82967f7 Compare February 1, 2023 10:16

Run check-task-pipeline-repo-existence ran afte...

e2dccf3

...r build-container

Michkov approved these changes Feb 1, 2023

View reviewed changes

tkdchen approved these changes Feb 1, 2023

View reviewed changes

lcarva approved these changes Feb 1, 2023

View reviewed changes

zregvart merged commit d44f241 into konflux-ci:main Feb 1, 2023

zregvart deleted the issue/HACBS-1675 branch February 1, 2023 15:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create Tekton task and Pipeline for Tekton bundles #284

Create Tekton task and Pipeline for Tekton bundles #284

zregvart commented Jan 26, 2023

robnester-rh left a comment

simonbaird left a comment

simonbaird commented Jan 26, 2023

zregvart commented Jan 27, 2023

zregvart commented Jan 27, 2023

zregvart commented Jan 27, 2023

zregvart commented Jan 30, 2023

lcarva Jan 30, 2023 •

edited

Loading

zregvart Jan 31, 2023

lcarva Jan 30, 2023 •

edited

Loading

zregvart Jan 31, 2023

Michkov left a comment

Michkov Jan 31, 2023

zregvart Jan 31, 2023

zregvart commented Jan 31, 2023

tkdchen left a comment

zregvart commented Feb 1, 2023

zregvart commented Feb 1, 2023

zregvart commented Feb 1, 2023

Create Tekton task and Pipeline for Tekton bundles #284

Create Tekton task and Pipeline for Tekton bundles #284

Conversation

zregvart commented Jan 26, 2023

robnester-rh left a comment

Choose a reason for hiding this comment

simonbaird left a comment

Choose a reason for hiding this comment

simonbaird commented Jan 26, 2023

zregvart commented Jan 27, 2023

zregvart commented Jan 27, 2023

zregvart commented Jan 27, 2023

zregvart commented Jan 30, 2023

lcarva Jan 30, 2023 • edited Loading

Choose a reason for hiding this comment

zregvart Jan 31, 2023

Choose a reason for hiding this comment

lcarva Jan 30, 2023 • edited Loading

Choose a reason for hiding this comment

zregvart Jan 31, 2023

Choose a reason for hiding this comment

Michkov left a comment

Choose a reason for hiding this comment

Michkov Jan 31, 2023

Choose a reason for hiding this comment

zregvart Jan 31, 2023

Choose a reason for hiding this comment

zregvart commented Jan 31, 2023

tkdchen left a comment

Choose a reason for hiding this comment

zregvart commented Feb 1, 2023

zregvart commented Feb 1, 2023

zregvart commented Feb 1, 2023

lcarva Jan 30, 2023 •

edited

Loading

lcarva Jan 30, 2023 •

edited

Loading