[3.4] Fix all the pipeline failues for release 3.4 #14136

ahrtr · 2022-06-20T05:32:07Z

Fix issues/14135

Items resolved:

fix the vet error: possible misuse of reflect.SliceHeader;
fix the vet error: call to (*T).Fatal from a non-test goroutine;
bump package golang.org/x/crypto, net and sys;
bump boltdb from 1.3.3 to 1.3.6;
remove the vendor directory;
remove go 1.12.17 and 1.15.15, add go 1.16.15 into pipeline;
bump go version to 1.16 in go.mod;
fix the issue: compile: version go1.16.15 does not match go tool version go1.17.11;
fix data race on compactMainRev and watcherGauge;
fix test failure for TestLeasingTxnOwnerGet in cluster_proxy mode.

Signed-off-by: Benjamin Wang wachao@vmware.com

Please read https://github.com/etcd-io/etcd/blob/main/CONTRIBUTING.md#contribution-flow.

ahrtr · 2022-06-20T07:50:45Z

cc @hexfusion @serathius @ptabor @spzala @mitake

ptabor

Thank you. LGTM.

.github/workflows/tests.yaml

ptabor · 2022-06-21T11:42:10Z

.github/workflows/tests.yaml

        echo "${TARGET}"
        case "${TARGET}" in
          linux-amd64-fmt)
            GOARCH=amd64 PASSES='fmt bom dep' ./test
            ;;
          linux-amd64-integration-1-cpu)
-            GOARCH=amd64 CPU=1 PASSES='integration' RACE="${RACE}" ./test
+            GOARCH=amd64 CPU=1 PASSES='integration' RACE='false' ./test


Is it intended to turn RACE of permanently ?
I see you fix some of the races in the other places in this PR.

I just keep it as it's. Previously when go version is 1.15.15, then RACE is set to false, see tests.yaml#L47

serathius · 2022-06-21T11:46:29Z

.github/workflows/tests.yaml

            ;;
          linux-amd64-unit)
-            GOARCH=amd64 PASSES='unit' RACE="${RACE}" ./test
+            GOARCH=amd64 PASSES='unit' RACE='false' ./test


I think we should have at least one test running with RACE='true'

Yes, I agree. But let's add any new pipeline/test in a separate PR. I just bumped the go version to 1.16.15 in this PR, and keep all others unchanged.

Previously when go version is 1.15.15, then RACE is set to false, see tests.yaml#L47. So I just keep it as it's.

lavacat · 2022-06-21T21:02:38Z

go.mod

@@ -1,6 +1,6 @@
 module go.etcd.io/etcd

-go 1.12
+go 1.16


I think the idea was to keep this at 1.12 #12840 (comment) CC @ptabor

No need to support go 1.12. It's out of support, and is subject to vulnerability. IIRC It is also not compatible with some system packages in newer version.

Note that it's more than 1 year before when ptabor added that comment, and go 1.12 wasn't that old although it's also out of support at that time.

I'm not 100% sure but I think go 1.16 keeps go compatibility promise, so its not subject to vulnerabilities.
I also run into package issues but go mod tidy fixed it ff89836

Either way is fine with me, but both will work IMO.

Note that the promise of compatibility is for golang language of version < 2.0. There is no guarantee on the core library and tool-chain.

etcdserver/server_test.go

lavacat · 2022-06-21T21:26:13Z

I've made some of the similar fixes here as a stand-alone commits with references to corresponding 3.5 fixes. I think it's easier to track fixes across branches. You can cherry-pick my commits.

You also want this flaky test fix 4aeadec

etcdserver/api/v2store/node_extern_test.go

clientv3/concurrency/election_test.go

Items resolved: 1. fix the vet error: possible misuse of reflect.SliceHeader; 2. fix the vet error: call to (*T).Fatal from a non-test goroutine; 3. bump package golang.org/x/crypto, net and sys; 4. bump boltdb from 1.3.3 to 1.3.6; 5. remove the vendor directory; 6. remove go 1.12.17 and 1.15.15, add go 1.16.15 into pipeline; 7. bump go version to 1.16 in go.mod; 8. fix the issue: compile: version go1.16.15 does not match go tool version go1.17.11, refer to actions/setup-go#107; 9. fix data race on compactMainRev and watcherGauge; 10. fix test failure for TestLeasingTxnOwnerGet in cluster_proxy mode. Signed-off-by: Benjamin Wang <wachao@vmware.com>

ahrtr · 2022-06-21T22:07:18Z

I've made some of the similar fixes here as a stand-alone commits with references to corresponding 3.5 fixes. I think it's easier to track fixes across branches. You can cherry-pick my commits.

You also want this flaky test fix 4aeadec

Let's keep the PR as it's for now. It's just a starting point for the 3.4 pipeline and all test. Just as I mentioned in issues/14135, 3.4 pipeline has never green since the first day (on Jun 24, 2021) created.

We can continue to work on other issues & enhancement in separate PRs.

But the top priority is still to work on the pipeline & test failures/flaky. Let's try to add more pipelines (such as RACE=true) in a followup PR, and make sure everything is green.

ahrtr · 2022-06-23T10:32:17Z

Plan for 3.4.19: 14105

ahrtr force-pushed the 3.4_pipeline_issues branch from b0e451b to b045141 Compare June 20, 2022 05:39

ahrtr mentioned this pull request Jun 20, 2022

etcd Go maintenance plan #12840

Closed

ahrtr force-pushed the 3.4_pipeline_issues branch from b045141 to 9a93ce9 Compare June 20, 2022 07:40

ptabor approved these changes Jun 21, 2022

View reviewed changes

serathius reviewed Jun 21, 2022

View reviewed changes

ahrtr force-pushed the 3.4_pipeline_issues branch 2 times, most recently from 94d19a2 to 93bb977 Compare June 21, 2022 20:50

lavacat reviewed Jun 21, 2022

View reviewed changes

etcdserver/server_test.go Outdated Show resolved Hide resolved

lavacat reviewed Jun 21, 2022

View reviewed changes

etcdserver/api/v2store/node_extern_test.go Show resolved Hide resolved

clientv3/concurrency/election_test.go Outdated Show resolved Hide resolved

ahrtr force-pushed the 3.4_pipeline_issues branch from 93bb977 to 1abf085 Compare June 21, 2022 21:30

ahrtr mentioned this pull request Jun 22, 2022

Update 3.4 change log item for 3.4.19 #14141

Merged

ahrtr merged commit 953376e into etcd-io:release-3.4 Jun 22, 2022

This was referenced Jun 23, 2022

3.4 pipeline is broken #14135

Closed

Plans for v3.4.19 release #14105

Closed

ahrtr mentioned this pull request Jun 30, 2022

[3.4] Fix pipeline failures in 3.4 #14183

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[3.4] Fix all the pipeline failues for release 3.4 #14136

[3.4] Fix all the pipeline failues for release 3.4 #14136

ahrtr commented Jun 20, 2022 •

edited

Loading

ahrtr commented Jun 20, 2022 •

edited

Loading

ptabor left a comment

ptabor Jun 21, 2022

ahrtr Jun 21, 2022

serathius Jun 21, 2022

ahrtr Jun 21, 2022 •

edited

Loading

lavacat Jun 21, 2022

ahrtr Jun 21, 2022

lavacat Jun 22, 2022

ahrtr Jun 22, 2022

lavacat commented Jun 21, 2022 •

edited

Loading

ahrtr commented Jun 21, 2022

ahrtr commented Jun 23, 2022

[3.4] Fix all the pipeline failues for release 3.4 #14136

[3.4] Fix all the pipeline failues for release 3.4 #14136

Conversation

ahrtr commented Jun 20, 2022 • edited Loading

ahrtr commented Jun 20, 2022 • edited Loading

ptabor left a comment

Choose a reason for hiding this comment

ptabor Jun 21, 2022

Choose a reason for hiding this comment

ahrtr Jun 21, 2022

Choose a reason for hiding this comment

serathius Jun 21, 2022

Choose a reason for hiding this comment

ahrtr Jun 21, 2022 • edited Loading

Choose a reason for hiding this comment

lavacat Jun 21, 2022

Choose a reason for hiding this comment

ahrtr Jun 21, 2022

Choose a reason for hiding this comment

lavacat Jun 22, 2022

Choose a reason for hiding this comment

ahrtr Jun 22, 2022

Choose a reason for hiding this comment

lavacat commented Jun 21, 2022 • edited Loading

ahrtr commented Jun 21, 2022

ahrtr commented Jun 23, 2022

ahrtr commented Jun 20, 2022 •

edited

Loading

ahrtr commented Jun 20, 2022 •

edited

Loading

ahrtr Jun 21, 2022 •

edited

Loading

lavacat commented Jun 21, 2022 •

edited

Loading