Improve CLI e2e go tests #505

praveen4g0 · 2019-12-07T08:57:18Z

Changes

I have Added E2E cli tests for pipelines alone.

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

Includes tests (if functionality changed/added)
Includes docs (if user facing)
Regenerate the manpages and docs with make docs and make man if needed.
Run the code checkers with make check
Commit messages follow commit message best practices

See the contribution guide
for more details.

Release Notes

release-note

tekton-robot · 2019-12-07T08:57:33Z

Hi @praveen4g0. Thanks for your PR.

I'm waiting for a tektoncd member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

tekton-robot · 2019-12-07T08:58:11Z

@praveen4g0: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/ok-to-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

piyush-garg · 2019-12-07T17:33:24Z

/ok-to-test

praveen4g0 · 2019-12-09T09:14:00Z

/rerun pull-tekton-cli-unit-tests

vdemeester · 2019-12-12T08:51:18Z

/test pull-tekton-cli-unit-tests

vdemeester

/lgtm

Let's get this in and iterate over it

tekton-robot · 2019-12-20T09:01:10Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: vdemeester

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [vdemeester]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

vdemeester · 2019-12-20T09:12:35Z

/retest

chmouel · 2019-12-20T10:06:47Z

IMHO we need to figure out about the flakyness of go-netflix before merging this, see #499 for details.

Feel free to remove this /hold if you tink otherwise

/hold

praveen4g0 · 2019-12-23T07:51:57Z

/test pull-tekton-cli-build-tests

tekton-robot · 2019-12-23T10:57:16Z

The following is the coverage report on pkg/.
Say /test pull-tekton-cli-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage
third_party/VENDOR-LICENSE/github.com/hashicorp/golang-lru/2q.go	Do not exist	91.7%
third_party/VENDOR-LICENSE/github.com/hashicorp/golang-lru/arc.go	Do not exist	94.4%
third_party/VENDOR-LICENSE/github.com/hashicorp/golang-lru/lru.go	Do not exist	91.1%
third_party/VENDOR-LICENSE/github.com/hashicorp/golang-lru/simplelru/lru.go	Do not exist	88.3%

praveen4g0 · 2019-12-24T06:28:46Z

/test pull-tekton-cli-unit-tests

praveen4g0 · 2020-01-06T13:49:07Z

/test pull-tekton-cli-unit-tests

praveen4g0 · 2020-02-19T17:05:57Z

/test pull-tekton-cli-integration-tests

praveen4g0 · 2020-02-25T12:02:10Z

@vdemeester do you see still see any blocker to merge this pr??

vdemeester · 2020-02-25T13:08:45Z

/hold cancel

danielhelfand

Overall, I am struggling with the structure of these e2e tests in this pr. My apologies as this is the first time I am reviewing this and this pr is large in general, but I wanted to clarify some things before this moves forward:

Do we know how much of the old suite of tests is covered in this conversion/ how similar the tests are? I'll start doing a more thorough review of this as well.
I get that this is a first attempt to get all the e2e tests over to go, but the organization of the tests themselves is hard to follow. pipeline_e2e_test.go should be broken up into different files, but that's perhaps something we need to iterate on.

danielhelfand · 2020-03-02T15:35:30Z

test/e2e/build_templates.go

+func GetTaskRun(c *Clients, name string) *v1alpha1.TaskRun {
+
+	taskRun, err := c.TaskRunClient.Get(name, metav1.GetOptions{})
+	// 	require.Nil(t, err)


Can all the commented out code be removed?

// require.Nil(t, err)

test/environment/environment.go

vdemeester · 2020-03-03T15:51:53Z

1. Do we know how much of the old suite of tests is covered in this conversion/ how similar the tests are? I'll start doing a more thorough review of this as well.

I'm temporarly adding back the old ones to make sure we don't loose coverage

2. I get that this is a first attempt to get all the e2e tests over to go, but the organization of the tests themselves is hard to follow. `pipeline_e2e_test.go` should be broken up into different files, but that's perhaps something we need to iterate on.

Yep, I plan to follow-up to clean a little bit of stuff and split those tests and organize them a little bit better 😉

googlebot · 2020-03-03T15:52:05Z

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

googlebot · 2020-03-03T17:50:37Z

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

danielhelfand · 2020-03-05T21:46:38Z

test/e2e/pipeline_e2e_test.go

+		})
+
+	})
+	// Bug to fix


Can this comment be removed, or is there something to denote here?

danielhelfand · 2020-03-05T21:47:30Z

vendor/gotest.tools/v3/icmd/command.go

+
+// Assert compares the Result against the Expected struct, and fails the test if
+// any of the expectations are not met.
+//


This can be removed

danielhelfand · 2020-03-05T21:49:43Z

I am fine with moving forward with this after the nit comments are dealt with. If we could have just one other person review, I think this should be good to go.

ppitonak · 2020-03-06T15:31:00Z

go.mod

 	go.uber.org/multierr v1.1.0
 	golang.org/x/crypto v0.0.0-20191206172530-e9b2fee46413
+	golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898


https://godoc.org/golang.org/x/xerrors

These functions were incorporated into the standard library's errors package in Go 1.13

@ppitonak indeed (with a slight signature change). Anyway, we can get rid of those in a follow-up (also, depending on our dependency it might become a transitive dep anway)

updated e2e.sh Added tkn environment varaiable Updated pipelines E2E tests Update go.sum Deleted third-party updated go.mod update pipeline, interactive cli Tests Fix: update client.go with lates pipeline bump (v0.10.1) fixed: golint error Fix: Pipeline start assertion Update README.md Fix: build failures Signed-off-by: Vincent Demeester <vdemeest@redhat.com> Add old e2e tests back while we port them to go Signed-off-by: Vincent Demeester <vdemeest@redhat.com> Small fixes to commented code Signed-off-by: Vincent Demeester <vdemeest@redhat.com> Update test/resources/output-pipelinerun.yaml Drop the flag to InitializeLogger

danielhelfand

/lgtm

Thanks for all your work on this @praveen4g0!

googlebot added the cla: yes label Dec 7, 2019

tekton-robot requested review from hrishin and sthaha December 7, 2019 08:57

tekton-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Dec 7, 2019

tekton-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Dec 7, 2019

vdemeester approved these changes Dec 20, 2019

View reviewed changes

tekton-robot assigned vdemeester Dec 20, 2019

tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Dec 20, 2019

tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 20, 2019

tekton-robot removed the lgtm Indicates that a PR is ready to be merged. label Dec 20, 2019

tekton-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 20, 2019

tekton-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 25, 2020

vdemeester added this to the 0.9.0 🐺 milestone Feb 27, 2020

danielhelfand reviewed Mar 2, 2020

View reviewed changes

vdemeester mentioned this pull request Mar 3, 2020

Improve e2e Testing Coverage for CLI #769

Closed

googlebot added cla: no and removed cla: yes labels Mar 3, 2020

googlebot added cla: yes and removed cla: no labels Mar 3, 2020

danielhelfand reviewed Mar 5, 2020

View reviewed changes

test/e2e/pipeline_e2e_test.go

})

})

// Bug to fix

Copy link

Member

danielhelfand Mar 5, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this comment be removed, or is there something to denote here?

danielhelfand reviewed Mar 5, 2020

View reviewed changes

ppitonak reviewed Mar 6, 2020

View reviewed changes

danielhelfand reviewed Mar 10, 2020

View reviewed changes

tekton-robot assigned danielhelfand Mar 10, 2020

tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Mar 10, 2020

tekton-robot merged commit 39ad796 into tektoncd:master Mar 10, 2020

danielhelfand mentioned this pull request Mar 26, 2020

Add ROADMAP.md 🤖 #835

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve CLI e2e go tests #505

Improve CLI e2e go tests #505

praveen4g0 commented Dec 7, 2019

tekton-robot commented Dec 7, 2019

tekton-robot commented Dec 7, 2019

piyush-garg commented Dec 7, 2019

praveen4g0 commented Dec 9, 2019

vdemeester commented Dec 12, 2019

vdemeester left a comment

tekton-robot commented Dec 20, 2019

vdemeester commented Dec 20, 2019

chmouel commented Dec 20, 2019 •

edited

Loading

praveen4g0 commented Dec 23, 2019

tekton-robot commented Dec 23, 2019

praveen4g0 commented Dec 24, 2019

praveen4g0 commented Jan 6, 2020

praveen4g0 commented Feb 19, 2020

praveen4g0 commented Feb 25, 2020

vdemeester commented Feb 25, 2020

danielhelfand left a comment

danielhelfand Mar 2, 2020

vdemeester commented Mar 3, 2020

googlebot commented Mar 3, 2020

googlebot commented Mar 3, 2020

danielhelfand Mar 5, 2020

danielhelfand Mar 5, 2020

danielhelfand commented Mar 5, 2020

ppitonak Mar 6, 2020

vdemeester Mar 6, 2020

danielhelfand left a comment

Improve CLI e2e go tests #505

Improve CLI e2e go tests #505

Conversation

praveen4g0 commented Dec 7, 2019

Changes

Submitter Checklist

Release Notes

tekton-robot commented Dec 7, 2019

tekton-robot commented Dec 7, 2019

piyush-garg commented Dec 7, 2019

praveen4g0 commented Dec 9, 2019

vdemeester commented Dec 12, 2019

vdemeester left a comment

Choose a reason for hiding this comment

tekton-robot commented Dec 20, 2019

vdemeester commented Dec 20, 2019

chmouel commented Dec 20, 2019 • edited Loading

praveen4g0 commented Dec 23, 2019

tekton-robot commented Dec 23, 2019

praveen4g0 commented Dec 24, 2019

praveen4g0 commented Jan 6, 2020

praveen4g0 commented Feb 19, 2020

praveen4g0 commented Feb 25, 2020

vdemeester commented Feb 25, 2020

danielhelfand left a comment

Choose a reason for hiding this comment

danielhelfand Mar 2, 2020

Choose a reason for hiding this comment

vdemeester commented Mar 3, 2020

googlebot commented Mar 3, 2020

googlebot commented Mar 3, 2020

danielhelfand Mar 5, 2020

Choose a reason for hiding this comment

danielhelfand Mar 5, 2020

Choose a reason for hiding this comment

danielhelfand commented Mar 5, 2020

ppitonak Mar 6, 2020

Choose a reason for hiding this comment

vdemeester Mar 6, 2020

Choose a reason for hiding this comment

danielhelfand left a comment

Choose a reason for hiding this comment

chmouel commented Dec 20, 2019 •

edited

Loading