[openshift-4.4] Bug 1815637: bump etcd v3.3.19 #40

hexfusion · 2020-03-20T16:45:16Z

This PR bumps etcd to v3.3.19 which includes a few important bugfixes:

etcd server

Fix corruption bug in defrag. mvcc/backend: Fix corruption bug in defrag etcd-io/etcd#11613

client v3

Fix "hasleader" metadata embedding. clientv3: fix "hasleader" metadata key, embed client version etcd-io/etcd#11687

gRPC Proxy

Fix panic on error for metrics handler. proxy/grpcproxy: add return on error for metrics handler etcd-io/etcd#11694

Full 3.3.19 changelog: https://github.com/etcd-io/etcd/blob/master/CHANGELOG-3.3.md#v3319-2020-03-18

Once chk(ai) fails with auth.ErrAuthOldRevision it will always do, regardless how many times you retry. So the error is better be returned to fail the pending request and make the client re-authenticate.

Remove auth validation loop in v3_server.raftRequest(). Re-validation when error ErrAuthOldRevision occurs should be handled on client side.

Disable TestV3AuthOldRevConcurrent for now. See etcd-io#10468 (comment)

…f-#10218-etcd-io#10468-upstream-release-3.3 Automated cherry pick of etcd-io#10218 etcd-io#10468 on release 3.3

…-#11613-origin-release-3.3 Automated cherry pick of etcd-io#11613 to release-3.3

Check if boltOpenOptions is nil before use it.

…f-#11630-upstream-release-3.3 Automated cherry pick of etcd-io#11630 to release-3.3

Use members information from member add response, which is guaranteed to be up to date.

…f-#11638-upstream-release-3.3 Automated cherry pick of etcd-io#11638 on release-3.3

Signed-off-by: Sam Batschelet <sbatsche@redhat.com>

ref. etcd-io#11704 Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

ref. etcd-io#11687 Signed-off-by: Gyuho Lee <leegyuho@amazon.com> clientv3: fix racy writes to context key === RUN TestWatchOverlapContextCancel ================== WARNING: DATA RACE Write at 0x00c42110dd40 by goroutine 99: runtime.mapassign() /usr/local/go/src/runtime/hashmap.go:485 +0x0 github.com/coreos/etcd/clientv3.metadataSet() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/ctx.go:61 +0x8c github.com/coreos/etcd/clientv3.withVersion() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/ctx.go:47 +0x137 github.com/coreos/etcd/clientv3.newStreamClientInterceptor.func1() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/client.go:309 +0x81 google.golang.org/grpc.NewClientStream() /go/src/github.com/coreos/etcd/gopath/src/google.golang.org/grpc/stream.go:101 +0x10e github.com/coreos/etcd/etcdserver/etcdserverpb.(*watchClient).Watch() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/etcdserver/etcdserverpb/rpc.pb.go:3193 +0xe9 github.com/coreos/etcd/clientv3.(*watchGrpcStream).openWatchClient() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:788 +0x143 github.com/coreos/etcd/clientv3.(*watchGrpcStream).newWatchClient() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:700 +0x5c3 github.com/coreos/etcd/clientv3.(*watchGrpcStream).run() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:431 +0x12b Previous read at 0x00c42110dd40 by goroutine 130: reflect.maplen() /usr/local/go/src/runtime/hashmap.go:1165 +0x0 reflect.Value.MapKeys() /usr/local/go/src/reflect/value.go:1090 +0x43b fmt.(*pp).printValue() /usr/local/go/src/fmt/print.go:741 +0x1885 fmt.(*pp).printArg() /usr/local/go/src/fmt/print.go:682 +0x1b1 fmt.(*pp).doPrintf() /usr/local/go/src/fmt/print.go:998 +0x1cad fmt.Sprintf() /usr/local/go/src/fmt/print.go:196 +0x77 github.com/coreos/etcd/clientv3.streamKeyFromCtx() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:825 +0xc8 github.com/coreos/etcd/clientv3.(*watcher).Watch() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:265 +0x426 github.com/coreos/etcd/clientv3/integration.testWatchOverlapContextCancel.func1() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:959 +0x23e Goroutine 99 (running) created at: github.com/coreos/etcd/clientv3.(*watcher).newWatcherGrpcStream() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:236 +0x59d github.com/coreos/etcd/clientv3.(*watcher).Watch() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:278 +0xbb6 github.com/coreos/etcd/clientv3/integration.testWatchOverlapContextCancel.func1() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:959 +0x23e Goroutine 130 (running) created at: github.com/coreos/etcd/clientv3/integration.testWatchOverlapContextCancel() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:979 +0x76d github.com/coreos/etcd/clientv3/integration.TestWatchOverlapContextCancel() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:922 +0x44 testing.tRunner() /usr/local/go/src/testing/testing.go:657 +0x107 ================== Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

ref. etcd-io#11687 Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

This reverts commit 3f6b978.

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

This reverts commit acb9746.

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

openshift-ci-robot · 2020-03-20T16:45:18Z

@hexfusion: No Bugzilla bug is referenced in the title of this pull request.
To reference a bug, add 'Bug XXX:' to the title of this pull request and request another bug refresh with /bugzilla refresh.

In response to this:

[openshift-4.4] bump etcd v3.3.19

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2020-03-20T16:45:42Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hexfusion

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [hexfusion]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot · 2020-03-20T16:47:02Z

@hexfusion: No Bugzilla bug is referenced in the title of this pull request.
To reference a bug, add 'Bug XXX:' to the title of this pull request and request another bug refresh with /bugzilla refresh.

In response to this:

[wip] [openshift-4.4] bump etcd v3.3.19

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

…xists, go ahead and start.

hexfusion · 2020-03-21T00:47:34Z

level=fatal msg="failed to fetch Cluster: failed to fetch dependency of "Cluster": failed to generate asset "Platform Permissions Check": validate AWS credentials: checking install permissions: error simulating policy: Throttling: Rate exceeded\n\tstatus code: 400, request id: c33e690c-1c03-468c-8cf7-359a1d476f8a"

/retest

hexfusion · 2020-03-21T02:15:12Z

/retest

hexfusion · 2020-03-21T15:37:33Z

retesting everything
/test all

hexfusion · 2020-03-21T18:48:59Z

/test e2e-gcp-upgrade

hexfusion · 2020-03-21T21:46:38Z

one more run
/test all

hexfusion · 2020-03-21T21:52:24Z

--- FAIL: TestRoundRobinBalancedResolvableFailoverFromRequestFail (0.04s)
balancer_test.go:308: expected balanced loads for 20 requests, got switches 3

I have seen this before

/test unit

hexfusion · 2020-03-21T23:54:33Z

/test e2e-gcp-upgrade

hexfusion · 2020-03-25T10:14:17Z

more soak
/test all

hexfusion · 2020-03-25T17:32:36Z

one more last was all green

/test all

hexfusion · 2020-03-25T17:44:25Z

cc @deads2k @smarterclayton @abhinavdahiya @sdodson @crawford

sdodson · 2020-03-25T18:16:12Z

Sorry for silly first time looking at this repo questions, but are we forked from upstream or does this match upstream 3.3.19 exactly?

hexfusion · 2020-03-25T18:23:40Z

we are forked from upstream regarding the repo. the branch is rebased against the upstream v3.3.19 tag. So we are carrying some patches (not to etcd) on top of v3.3.19.

hexfusion · 2020-03-25T20:57:26Z

/test e2e-gcp-upgrade

hexfusion · 2020-03-25T22:23:50Z

fail [github.com/openshift/origin/test/extended/util/disruption/disruption.go:226]: Mar 25 22:04:35.476: API was unreachable during disruption for at least 8m38s of 34m35s (25%):

.. wow
/test e2e-gcp-upgrade

hexfusion · 2020-03-26T00:22:56Z

/retest

hexfusion · 2020-03-27T01:59:26Z

/hold
wait for GA

hexfusion · 2020-03-29T23:58:58Z

soak

/test all

hexfusion · 2020-04-15T02:11:05Z

/test e2e-aws-upgrade

hexfusion · 2020-04-17T15:04:50Z

this is continued on #45

horkhe and others added 22 commits November 20, 2019 16:45

etcdserver: Remove infinite loop in doSerialize

95095f8

Once chk(ai) fails with auth.ErrAuthOldRevision it will always do, regardless how many times you retry. So the error is better be returned to fail the pending request and make the client re-authenticate.

etcdserver: remove auth validation loop

5a48217

Remove auth validation loop in v3_server.raftRequest(). Re-validation when error ErrAuthOldRevision occurs should be handled on client side.

integration: disable TestV3AuthOldRevConcurrent

e1508f9

Disable TestV3AuthOldRevConcurrent for now. See etcd-io#10468 (comment)

Merge pull request etcd-io#11378 from jingyih/automated-cherry-pick-o…

70853d6

…f-#10218-etcd-io#10468-upstream-release-3.3 Automated cherry pick of etcd-io#10218 etcd-io#10468 on release 3.3

mvcc/backend: Delete orphaned db.tmp files before defrag

b3d9e29

Merge pull request etcd-io#11623 from jpbetz/automated-cherry-pick-of…

b0a4038

…-#11613-origin-release-3.3 Automated cherry pick of etcd-io#11613 to release-3.3

mvcc/backend: check for nil boltOpenOptions

7b1a92c

Check if boltOpenOptions is nil before use it.

Merge pull request etcd-io#11632 from jingyih/automated-cherry-pick-o…

e21e355

…f-#11630-upstream-release-3.3 Automated cherry pick of etcd-io#11630 to release-3.3

etcdctl: fix member add command

c58133b

Use members information from member add response, which is guaranteed to be up to date.

Merge pull request etcd-io#11665 from jingyih/automated-cherry-pick-o…

eb1df6d

…f-#11638-upstream-release-3.3 Automated cherry pick of etcd-io#11638 on release-3.3

proxy/grpcproxy: add return on error for metrics handler

1228d6c

Signed-off-by: Sam Batschelet <sbatsche@redhat.com>

etcdserver/api/etcdhttp: log server-side /health checks

30aaceb

ref. etcd-io#11704 Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

etcdserver/api/v3rpc: handle api version metadata, add metrics

d9027ce

ref. etcd-io#11687 Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

version: 3.3.19

f9c8920

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

words: whitelist "hasleader"

10d50e0

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

Revert "version: 3.3.19"

07562e2

This reverts commit 3f6b978.

version: 3.3.19

acb9746

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

travis.yaml: use Go 1.12.12

5088080

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

Revert "version: 3.3.19"

cd200b4

This reverts commit acb9746.

words: whitelist "racey"

a463bd5

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

version: 3.3.19

67da93f

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

openshift-ci-robot requested review from crawford and smarterclayton March 20, 2020 16:45

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 20, 2020

hexfusion changed the title ~~[openshift-4.4] bump etcd v3.3.19~~ [wip] [openshift-4.4] bump etcd v3.3.19 Mar 20, 2020

openshift-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 20, 2020

deads2k and others added 7 commits March 20, 2020 20:35

add stub discovery-etcd-initial-cluster command

b7641a7

build openshift tools with etcd

02f1808

codify the initial cluster check as golang code

8cf43a7

Archive data-dir if target member is unstarted

8eaefdf

fix removed member name, unmask error

ab0737a

If we weren't able to get client or get target member but memberDir e…

c02fb94

…xists, go ahead and start.

list all peers in initial-cluster

b44f57e

hexfusion force-pushed the openshift-4.4-bump-etcd-v3.3.19 branch from 0430519 to b44f57e Compare March 21, 2020 00:36

openshift-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 27, 2020

hexfusion closed this Apr 17, 2020

openshift-ci-robot mentioned this pull request Jul 10, 2020

Bug 1815637: bump etcd v3.3.22 #47

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[openshift-4.4] Bug 1815637: bump etcd v3.3.19 #40

[openshift-4.4] Bug 1815637: bump etcd v3.3.19 #40

hexfusion commented Mar 20, 2020 •

edited

Loading

openshift-ci-robot commented Mar 20, 2020

openshift-ci-robot commented Mar 20, 2020

openshift-ci-robot commented Mar 20, 2020

hexfusion commented Mar 21, 2020 •

edited

Loading

hexfusion commented Mar 21, 2020

hexfusion commented Mar 21, 2020 •

edited

Loading

hexfusion commented Mar 21, 2020

hexfusion commented Mar 21, 2020

hexfusion commented Mar 21, 2020

hexfusion commented Mar 21, 2020

hexfusion commented Mar 25, 2020

hexfusion commented Mar 25, 2020

hexfusion commented Mar 25, 2020

sdodson commented Mar 25, 2020

hexfusion commented Mar 25, 2020

hexfusion commented Mar 25, 2020

hexfusion commented Mar 25, 2020

hexfusion commented Mar 26, 2020

hexfusion commented Mar 27, 2020

hexfusion commented Mar 29, 2020

hexfusion commented Apr 15, 2020

hexfusion commented Apr 17, 2020

[openshift-4.4] Bug 1815637: bump etcd v3.3.19 #40

[openshift-4.4] Bug 1815637: bump etcd v3.3.19 #40

Conversation

hexfusion commented Mar 20, 2020 • edited Loading

openshift-ci-robot commented Mar 20, 2020

openshift-ci-robot commented Mar 20, 2020

openshift-ci-robot commented Mar 20, 2020

hexfusion commented Mar 21, 2020 • edited Loading

hexfusion commented Mar 21, 2020

hexfusion commented Mar 21, 2020 • edited Loading

hexfusion commented Mar 21, 2020

hexfusion commented Mar 21, 2020

hexfusion commented Mar 21, 2020

hexfusion commented Mar 21, 2020

hexfusion commented Mar 25, 2020

hexfusion commented Mar 25, 2020

hexfusion commented Mar 25, 2020

sdodson commented Mar 25, 2020

hexfusion commented Mar 25, 2020

hexfusion commented Mar 25, 2020

hexfusion commented Mar 25, 2020

hexfusion commented Mar 26, 2020

hexfusion commented Mar 27, 2020

hexfusion commented Mar 29, 2020

hexfusion commented Apr 15, 2020

hexfusion commented Apr 17, 2020

hexfusion commented Mar 20, 2020 •

edited

Loading

hexfusion commented Mar 21, 2020 •

edited

Loading

hexfusion commented Mar 21, 2020 •

edited

Loading