Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TEP-0091] use VerificationResult in verify #6673

Merged

Conversation

Yongxuanzhang
Copy link
Member

@Yongxuanzhang Yongxuanzhang commented May 16, 2023

Changes

This commits uses the VerificationResult as the return value for VerifyTask and VerifyPipeline. Previously returned error will be replaced with a VerificationError type VerificationResult and error is in Err field. The cases when nil is returned are currently changed to 3 types of VerificationResult:
1) Verification is skipped
2) Verification passed
3) Warning is logged during verification.

2nd PR of #6665

Signed-off-by: Yongxuan Zhang yongxuanzhang@google.com

/kind feature

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

  • Has Docs if any changes are user facing, including updates to minimum requirements e.g. Kubernetes version bumps
  • Has Tests included if any functionality added or changed
  • Follows the commit message standard
  • Meets the Tekton contributor standards (including functionality, content, code)
  • Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
  • Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings). See some examples of good release notes.
  • Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes

VerificationResult is the return value for instead of error for VerifyTask and VerifyPipeline.

@tekton-robot
Copy link
Collaborator

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@tekton-robot tekton-robot added release-note-none Denotes a PR that doesnt merit a release note. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/feature Categorizes issue or PR as related to a new feature. labels May 16, 2023
@tekton-robot tekton-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed release-note-none Denotes a PR that doesnt merit a release note. labels May 16, 2023
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/trustedresources/verify.go 95.6% 96.1% 0.5

@Yongxuanzhang Yongxuanzhang force-pushed the use-verificationresults-in-verify branch from 634e66e to 8d6e034 Compare May 16, 2023 21:42
@Yongxuanzhang Yongxuanzhang marked this pull request as ready for review May 16, 2023 21:43
@tekton-robot tekton-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 16, 2023
@tekton-robot tekton-robot requested review from abayer and jerop May 16, 2023 21:43
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/trustedresources/verify.go 95.6% 96.1% 0.5

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/trustedresources/verify.go 95.6% 96.1% 0.5

@lbernick lbernick self-assigned this May 17, 2023
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/trustedresources/verify.go 95.6% 94.5% -1.1

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/trustedresources/verify.go 95.6% 94.5% -1.1

@Yongxuanzhang Yongxuanzhang force-pushed the use-verificationresults-in-verify branch from 09d4506 to aa07dee Compare May 17, 2023 14:39
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/trustedresources/verify.go 95.6% 94.5% -1.1

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/trustedresources/verify.go 95.6% 94.5% -1.1

Copy link
Member

@lbernick lbernick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for breaking into small PRs! It is a big help with review.

pkg/reconciler/pipelinerun/resources/pipelineref.go Outdated Show resolved Hide resolved
pkg/trustedresources/verify.go Outdated Show resolved Hide resolved
pkg/trustedresources/verify.go Outdated Show resolved Hide resolved
}
}
return fmt.Errorf("failed to get matched policies: %w", err)
return &VerificationResult{VerificationResultType: VerificationError, Err: fmt.Errorf("failed to get matched policies: %w", err)}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this different from ErrNoMatchedPolicies?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this err could be ErrNoMatchedPolicies or other errors from getMatchedPolicies, e.g. ErrRegexMatch

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wouldn't it make sense to do the same thing for the warning then? if this error adds more context, that's probably also useful for the warning

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh sorry, I see the issue, I should return a wrapped error for warning as well, we do have information wrapped in the err and shouldn't return a new err directly

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

pkg/trustedresources/verify.go Outdated Show resolved Hide resolved
pkg/trustedresources/verify.go Outdated Show resolved Hide resolved
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/trustedresources/verify.go 95.6% 94.5% -1.1

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/trustedresources/verify.go 95.6% 94.5% -1.1

@Yongxuanzhang
Copy link
Member Author

/retest

@Yongxuanzhang Yongxuanzhang force-pushed the use-verificationresults-in-verify branch from 265cf3d to 8dde0b1 Compare May 18, 2023 15:10
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/trustedresources/verify.go 95.6% 95.5% -0.1

pkg/reconciler/pipelinerun/resources/pipelineref.go Outdated Show resolved Hide resolved
pkg/trustedresources/verify.go Outdated Show resolved Hide resolved
}
}
return fmt.Errorf("failed to get matched policies: %w", err)
return &VerificationResult{VerificationResultType: VerificationError, Err: fmt.Errorf("failed to get matched policies: %w", err)}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wouldn't it make sense to do the same thing for the warning then? if this error adds more context, that's probably also useful for the warning

@Yongxuanzhang Yongxuanzhang force-pushed the use-verificationresults-in-verify branch from 8dde0b1 to dcda49f Compare May 18, 2023 15:38
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/pipelinerun/resources/pipelineref.go 93.8% 92.0% -1.7
pkg/reconciler/taskrun/resources/taskref.go 90.0% 88.9% -1.1
pkg/trustedresources/verify.go 95.6% 95.6% 0.0

@Yongxuanzhang Yongxuanzhang force-pushed the use-verificationresults-in-verify branch from dcda49f to 5abf73e Compare May 18, 2023 16:05
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/pipelinerun/resources/pipelineref.go 93.8% 92.0% -1.7
pkg/reconciler/taskrun/resources/taskref.go 90.0% 88.9% -1.1
pkg/trustedresources/verify.go 95.6% 95.6% 0.0

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/pipelinerun/resources/pipelineref.go 93.8% 92.0% -1.7
pkg/reconciler/taskrun/resources/taskref.go 90.0% 88.9% -1.1
pkg/trustedresources/verify.go 95.6% 95.6% 0.0

@Yongxuanzhang
Copy link
Member Author

/retest
flake: #4169

@chuangw6
Copy link
Member

/assign

logger.Warnf(warn.Error())
return VerificationResult{VerificationResultType: VerificationWarn, Err: warn}
}
if pass := doesAnyVerifierPass(resource, signature, verifiers); pass {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why create the variables pass and passverification? do they have different purposes?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh yes, thanks! That was a mistake 😞

@tekton-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lbernick

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tekton-robot tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 18, 2023
@Yongxuanzhang Yongxuanzhang force-pushed the use-verificationresults-in-verify branch 2 times, most recently from ce7403e to b583ba6 Compare May 18, 2023 18:58
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/pipelinerun/resources/pipelineref.go 93.8% 92.0% -1.7
pkg/reconciler/taskrun/resources/taskref.go 90.0% 88.9% -1.1
pkg/trustedresources/verify.go 95.6% 96.3% 0.8

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/pipelinerun/resources/pipelineref.go 93.8% 92.0% -1.7
pkg/reconciler/taskrun/resources/taskref.go 90.0% 88.9% -1.1
pkg/trustedresources/verify.go 95.6% 96.3% 0.8

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/pipelinerun/resources/pipelineref.go 93.8% 92.0% -1.7
pkg/reconciler/taskrun/resources/taskref.go 90.0% 88.9% -1.1
pkg/trustedresources/verify.go 95.6% 96.3% 0.8

Copy link
Member

@chuangw6 chuangw6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @Yongxuanzhang

Comment on lines 80 to 92
if errors.Is(err, ErrNoMatchedPolicies) {
switch config.GetVerificationNoMatchPolicy(ctx) {
case config.IgnoreNoMatchPolicy:
return nil
return VerificationResult{VerificationResultType: VerificationSkip}
case config.WarnNoMatchPolicy:
logger := logging.FromContext(ctx)
logger.Warnf("failed to get matched policies: %v", err)
return nil
warning := fmt.Errorf("failed to get matched policies: %w", err)
logger.Warnf(warning.Error())
return VerificationResult{VerificationResultType: VerificationWarn, Err: warning}
}
}
return fmt.Errorf("failed to get matched policies: %w", err)
return VerificationResult{VerificationResultType: VerificationError, Err: fmt.Errorf("failed to get matched policies: %w", err)}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel that having this logic after getting matched policies inside VerifyTask has 2 problems: 1). it makes the steps in function VerifyTask look like too long and hard to understand. 2). duplicated code exists in VerifyPipeline.

2 suggestions:

  • Can we nest this logic checking config into getMatchedPolicies function itself by passing the context to it? We can refactor getMatchedPolicies so that it returns 2 things: []*v1alpha1.VerificationPolicy and *VerificationResult? If returned *VerificationResult is not nil, we just return the result from inside VerifyTask. If nil, proceed to next step.
  • Also, can we just rename getMatchedPolicies to matchPolicies? see https://google.github.io/styleguide/go/decisions#getters. Perhaps also add a doc string to this function though it's not exported b/c it has a bit more logics.

This will help mitigate the 2 problems mentioned above.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! This is a reasonable suggestion. I'm planing to merge these 2 functions into 1. Maybe we could leave this non-related change in the refactoring PR?

If think we have docstrings for getMatchedPolicies.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the naming, I think the doc you linked is for getter, which is used as a member function to return some data from the struct, I think it is different from what we have here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doing this in another PR SGTM.

The naming standard is for both functions and member methods. "Function and method names should not use a Get or get prefix".

pkg/trustedresources/verify.go Outdated Show resolved Hide resolved
Comment on lines +154 to +159
vr := trustedresources.VerifyPipeline(ctx, obj, k8s, refSource, verificationPolicies)
if vr.VerificationResultType == trustedresources.VerificationError {
if vr.Err != nil {
return nil, fmt.Errorf("remote Pipeline verification failed for object %s: %w", obj.GetName(), vr.Err)
}
return nil, fmt.Errorf("remote Pipeline verification failed for object %s", obj.GetName())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also returned error in WarnNoMatchPolicy case. Just curious: shouldn't we propagate that as well instead of cutting it off here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WarnNoMatchPolicy case will get VerificationWarn type of VerificationResult right? So we don't consider that to be an error.

This commits uses the VerificationResult as the return value for
VerifyTask and VerifyPipeline. Previously returned error will be
replaced with a VerificationError type VerificationResult and error is
in Err field. The cases when nil is returned are currently changed to 3
types of VerificationResult:
    1) Verification is skipped
    2) Verification passed
    3) Warning is logged during verification.

Signed-off-by: Yongxuan Zhang yongxuanzhang@google.com
@Yongxuanzhang Yongxuanzhang force-pushed the use-verificationresults-in-verify branch from b583ba6 to ac7d2ff Compare May 18, 2023 22:15
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/pipelinerun/resources/pipelineref.go 93.8% 92.0% -1.7
pkg/reconciler/taskrun/resources/taskref.go 90.0% 88.9% -1.1
pkg/trustedresources/verify.go 95.6% 96.3% 0.8

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/pipelinerun/resources/pipelineref.go 93.8% 92.0% -1.7
pkg/reconciler/taskrun/resources/taskref.go 90.0% 88.9% -1.1
pkg/trustedresources/verify.go 95.6% 96.3% 0.8

@chuangw6
Copy link
Member

/lgtm

@tekton-robot tekton-robot added the lgtm Indicates that a PR is ready to be merged. label May 19, 2023
@Yongxuanzhang
Copy link
Member Author

@tekton-robot tekton-robot merged commit 96212a1 into tektoncd:main May 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/feature Categorizes issue or PR as related to a new feature. lgtm Indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants