Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Potential data race in TestResumeElection #10697

Closed
mitake opened this issue May 1, 2019 · 4 comments · Fixed by #10699
Closed

Potential data race in TestResumeElection #10697

mitake opened this issue May 1, 2019 · 4 comments · Fixed by #10699
Labels
type/bug
Milestone
etcd-v3.4

Comments

@mitake
Copy link
Contributor

mitake commented May 1, 2019

Found in #9817

TestResumeElection can cause data race like below:

==================
WARNING: DATA RACE
Read at 0x00c00000c418 by goroutine 54:
  go.etcd.io/etcd/mvcc.(*treeIndex).visit()
      /go/src/go.etcd.io/etcd/vendor/github.com/google/btree/btree.go:618 +0x225
  go.etcd.io/etcd/mvcc.(*treeIndex).Revisions()
      /go/src/go.etcd.io/etcd/mvcc/index.go:115 +0x20d
  go.etcd.io/etcd/mvcc.(*storeTxnRead).rangeKeys()
      /go/src/go.etcd.io/etcd/mvcc/kvstore_txn.go:128 +0x37a
  go.etcd.io/etcd/mvcc.(*storeTxnRead).Range()
      /go/src/go.etcd.io/etcd/mvcc/kvstore_txn.go:50 +0xe4
  go.etcd.io/etcd/mvcc.(*txnReadWrite).Range()
      <autogenerated>:1 +0xf4
  go.etcd.io/etcd/mvcc.(*metricsTxnWrite).Range()
      /go/src/go.etcd.io/etcd/mvcc/metrics_txn.go:36 +0x12a
  go.etcd.io/etcd/etcdserver.(*applierV3backend).Range()
      /go/src/go.etcd.io/etcd/etcdserver/apply.go:275 +0x2fc
  go.etcd.io/etcd/etcdserver.(*EtcdServer).Range.func3()
      /go/src/go.etcd.io/etcd/etcdserver/v3_server.go:104 +0x9d
  go.etcd.io/etcd/etcdserver.(*EtcdServer).doSerialize()
      /go/src/go.etcd.io/etcd/etcdserver/v3_server.go:554 +0xad
  go.etcd.io/etcd/etcdserver.(*EtcdServer).Range()
      /go/src/go.etcd.io/etcd/etcdserver/v3_server.go:105 +0x225
  go.etcd.io/etcd/etcdserver/api/v3rpc.(*kvServer).Range()
      /go/src/go.etcd.io/etcd/etcdserver/api/v3rpc/key.go:52 +0xb0
  go.etcd.io/etcd/etcdserver/api/v3rpc.(*quotaKVServer).Range()
      <autogenerated>:1 +0x87
  go.etcd.io/etcd/etcdserver/etcdserverpb._KV_Range_Handler.func1()
      /go/src/go.etcd.io/etcd/etcdserver/etcdserverpb/rpc.pb.go:3481 +0xa1
  go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1()
      /go/src/go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:31 +0x2d3
  go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-prometheus.(*ServerMetrics).UnaryServerInterceptor.func1()
      /go/src/go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-prometheus/server_metrics.go:107 +0xff
  go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1()
      /go/src/go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:34 +0x1d9
  go.etcd.io/etcd/etcdserver/api/v3rpc.newUnaryInterceptor.func1()
      /go/src/go.etcd.io/etcd/etcdserver/api/v3rpc/interceptor.go:60 +0xf0
  go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1()
      /go/src/go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:34 +0x1d9
  go.etcd.io/etcd/etcdserver/api/v3rpc.newLogUnaryInterceptor.func1()
      /go/src/go.etcd.io/etcd/etcdserver/api/v3rpc/interceptor.go:67 +0xda
  go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1()
      /go/src/go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:39 +0x33d
  go.etcd.io/etcd/etcdserver/etcdserverpb._KV_Range_Handler()
      /go/src/go.etcd.io/etcd/etcdserver/etcdserverpb/rpc.pb.go:3483 +0x1d8
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).processUnaryRPC()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:1026 +0xa8b
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).handleStream()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:1252 +0x138d
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).serveStreams.func1.1()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:699 +0xac
Previous write at 0x00c00000c418 by goroutine 121:
  go.etcd.io/etcd/mvcc.(*treeIndex).visit()
      /go/src/go.etcd.io/etcd/vendor/github.com/google/btree/btree.go:620 +0x33b
  go.etcd.io/etcd/mvcc.(*treeIndex).Revisions()
      /go/src/go.etcd.io/etcd/mvcc/index.go:115 +0x20d
  go.etcd.io/etcd/mvcc.(*storeTxnRead).rangeKeys()
      /go/src/go.etcd.io/etcd/mvcc/kvstore_txn.go:128 +0x37a
  go.etcd.io/etcd/mvcc.(*storeTxnRead).Range()
      /go/src/go.etcd.io/etcd/mvcc/kvstore_txn.go:50 +0xe4
  go.etcd.io/etcd/mvcc.(*txnReadWrite).Range()
      <autogenerated>:1 +0xf4
  go.etcd.io/etcd/mvcc.(*metricsTxnWrite).Range()
      /go/src/go.etcd.io/etcd/mvcc/metrics_txn.go:36 +0x12a
  go.etcd.io/etcd/etcdserver.(*applierV3backend).Range()
      /go/src/go.etcd.io/etcd/etcdserver/apply.go:275 +0x2fc
  go.etcd.io/etcd/etcdserver.(*EtcdServer).Range.func3()
      /go/src/go.etcd.io/etcd/etcdserver/v3_server.go:104 +0x9d
  go.etcd.io/etcd/etcdserver.(*EtcdServer).doSerialize()
      /go/src/go.etcd.io/etcd/etcdserver/v3_server.go:554 +0xad
  go.etcd.io/etcd/etcdserver.(*EtcdServer).Range()
      /go/src/go.etcd.io/etcd/etcdserver/v3_server.go:105 +0x225
  go.etcd.io/etcd/etcdserver/api/v3rpc.(*kvServer).Range()
      /go/src/go.etcd.io/etcd/etcdserver/api/v3rpc/key.go:52 +0xb0
  go.etcd.io/etcd/etcdserver/api/v3rpc.(*quotaKVServer).Range()
      <autogenerated>:1 +0x87
  go.etcd.io/etcd/etcdserver/etcdserverpb._KV_Range_Handler.func1()
      /go/src/go.etcd.io/etcd/etcdserver/etcdserverpb/rpc.pb.go:3481 +0xa1
  go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1()
      /go/src/go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:31 +0x2d3
  go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-prometheus.(*ServerMetrics).UnaryServerInterceptor.func1()
      /go/src/go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-prometheus/server_metrics.go:107 +0xff
  go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1()
      /go/src/go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:34 +0x1d9
  go.etcd.io/etcd/etcdserver/api/v3rpc.newUnaryInterceptor.func1()
      /go/src/go.etcd.io/etcd/etcdserver/api/v3rpc/interceptor.go:60 +0xf0
  go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1()
      /go/src/go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:34 +0x1d9
  go.etcd.io/etcd/etcdserver/api/v3rpc.newLogUnaryInterceptor.func1()
      /go/src/go.etcd.io/etcd/etcdserver/api/v3rpc/interceptor.go:67 +0xda
  go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1()
      /go/src/go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:39 +0x33d
  go.etcd.io/etcd/etcdserver/etcdserverpb._KV_Range_Handler()
      /go/src/go.etcd.io/etcd/etcdserver/etcdserverpb/rpc.pb.go:3483 +0x1d8
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).processUnaryRPC()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:1026 +0xa8b
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).handleStream()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:1252 +0x138d
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).serveStreams.func1.1()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:699 +0xac
Goroutine 54 (running) created at:
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).serveStreams.func1()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:697 +0xb8
  go.etcd.io/etcd/vendor/google.golang.org/grpc/internal/transport.(*http2Server).operateHeaders()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/internal/transport/http2_server.go:432 +0x14cb
  go.etcd.io/etcd/vendor/google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/internal/transport/http2_server.go:472 +0x37a
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).serveStreams()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:695 +0x170
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).handleRawConn.func2()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:649 +0x53
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).handleRawConn.func3()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:657 +0x34
Goroutine 121 (finished) created at:
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).serveStreams.func1()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:697 +0xb8
  go.etcd.io/etcd/vendor/google.golang.org/grpc/internal/transport.(*http2Server).operateHeaders()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/internal/transport/http2_server.go:432 +0x14cb
  go.etcd.io/etcd/vendor/google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/internal/transport/http2_server.go:472 +0x37a
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).serveStreams()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:695 +0x170
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).handleRawConn.func2()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:649 +0x53
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).handleRawConn.func3()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:657 +0x34
==================
==================
WARNING: DATA RACE
Read at 0x00c00000e428 by goroutine 54:
  go.etcd.io/etcd/mvcc.(*treeIndex).visit()
      /go/src/go.etcd.io/etcd/vendor/github.com/google/btree/btree.go:618 +0x23e
  go.etcd.io/etcd/mvcc.(*treeIndex).Revisions()
      /go/src/go.etcd.io/etcd/mvcc/index.go:115 +0x20d
  go.etcd.io/etcd/mvcc.(*storeTxnRead).rangeKeys()
      /go/src/go.etcd.io/etcd/mvcc/kvstore_txn.go:128 +0x37a
  go.etcd.io/etcd/mvcc.(*storeTxnRead).Range()
      /go/src/go.etcd.io/etcd/mvcc/kvstore_txn.go:50 +0xe4
  go.etcd.io/etcd/mvcc.(*txnReadWrite).Range()
      <autogenerated>:1 +0xf4
  go.etcd.io/etcd/mvcc.(*metricsTxnWrite).Range()
      /go/src/go.etcd.io/etcd/mvcc/metrics_txn.go:36 +0x12a
  go.etcd.io/etcd/etcdserver.(*applierV3backend).Range()
      /go/src/go.etcd.io/etcd/etcdserver/apply.go:275 +0x2fc
  go.etcd.io/etcd/etcdserver.(*EtcdServer).Range.func3()
      /go/src/go.etcd.io/etcd/etcdserver/v3_server.go:104 +0x9d
  go.etcd.io/etcd/etcdserver.(*EtcdServer).doSerialize()
      /go/src/go.etcd.io/etcd/etcdserver/v3_server.go:554 +0xad
  go.etcd.io/etcd/etcdserver.(*EtcdServer).Range()
      /go/src/go.etcd.io/etcd/etcdserver/v3_server.go:105 +0x225
  go.etcd.io/etcd/etcdserver/api/v3rpc.(*kvServer).Range()
      /go/src/go.etcd.io/etcd/etcdserver/api/v3rpc/key.go:52 +0xb0
  go.etcd.io/etcd/etcdserver/api/v3rpc.(*quotaKVServer).Range()
      <autogenerated>:1 +0x87
  go.etcd.io/etcd/etcdserver/etcdserverpb._KV_Range_Handler.func1()
      /go/src/go.etcd.io/etcd/etcdserver/etcdserverpb/rpc.pb.go:3481 +0xa1
  go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1()
      /go/src/go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:31 +0x2d3
  go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-prometheus.(*ServerMetrics).UnaryServerInterceptor.func1()
      /go/src/go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-prometheus/server_metrics.go:107 +0xff
  go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1()
      /go/src/go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:34 +0x1d9
  go.etcd.io/etcd/etcdserver/api/v3rpc.newUnaryInterceptor.func1()
      /go/src/go.etcd.io/etcd/etcdserver/api/v3rpc/interceptor.go:60 +0xf0
  go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1()
      /go/src/go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:34 +0x1d9
  go.etcd.io/etcd/etcdserver/api/v3rpc.newLogUnaryInterceptor.func1()
      /go/src/go.etcd.io/etcd/etcdserver/api/v3rpc/interceptor.go:67 +0xda
  go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1()
      /go/src/go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:39 +0x33d
  go.etcd.io/etcd/etcdserver/etcdserverpb._KV_Range_Handler()
      /go/src/go.etcd.io/etcd/etcdserver/etcdserverpb/rpc.pb.go:3483 +0x1d8
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).processUnaryRPC()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:1026 +0xa8b
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).handleStream()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:1252 +0x138d
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).serveStreams.func1.1()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:699 +0xac
Previous write at 0x00c00000e428 by goroutine 121:
  go.etcd.io/etcd/mvcc.(*treeIndex).visit()
      /go/src/go.etcd.io/etcd/mvcc/index.go:95 +0x1b7
  go.etcd.io/etcd/mvcc.(*treeIndex).Revisions()
      /go/src/go.etcd.io/etcd/mvcc/index.go:115 +0x20d
  go.etcd.io/etcd/mvcc.(*storeTxnRead).rangeKeys()
      /go/src/go.etcd.io/etcd/mvcc/kvstore_txn.go:128 +0x37a
  go.etcd.io/etcd/mvcc.(*storeTxnRead).Range()
      /go/src/go.etcd.io/etcd/mvcc/kvstore_txn.go:50 +0xe4
  go.etcd.io/etcd/mvcc.(*txnReadWrite).Range()
      <autogenerated>:1 +0xf4
  go.etcd.io/etcd/mvcc.(*metricsTxnWrite).Range()
      /go/src/go.etcd.io/etcd/mvcc/metrics_txn.go:36 +0x12a
  go.etcd.io/etcd/etcdserver.(*applierV3backend).Range()
      /go/src/go.etcd.io/etcd/etcdserver/apply.go:275 +0x2fc
  go.etcd.io/etcd/etcdserver.(*EtcdServer).Range.func3()
      /go/src/go.etcd.io/etcd/etcdserver/v3_server.go:104 +0x9d
  go.etcd.io/etcd/etcdserver.(*EtcdServer).doSerialize()
      /go/src/go.etcd.io/etcd/etcdserver/v3_server.go:554 +0xad
  go.etcd.io/etcd/etcdserver.(*EtcdServer).Range()
      /go/src/go.etcd.io/etcd/etcdserver/v3_server.go:105 +0x225
  go.etcd.io/etcd/etcdserver/api/v3rpc.(*kvServer).Range()
      /go/src/go.etcd.io/etcd/etcdserver/api/v3rpc/key.go:52 +0xb0
  go.etcd.io/etcd/etcdserver/api/v3rpc.(*quotaKVServer).Range()
      <autogenerated>:1 +0x87
  go.etcd.io/etcd/etcdserver/etcdserverpb._KV_Range_Handler.func1()
      /go/src/go.etcd.io/etcd/etcdserver/etcdserverpb/rpc.pb.go:3481 +0xa1
  go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1()
      /go/src/go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:31 +0x2d3
  go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-prometheus.(*ServerMetrics).UnaryServerInterceptor.func1()
      /go/src/go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-prometheus/server_metrics.go:107 +0xff
  go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1()
      /go/src/go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:34 +0x1d9
  go.etcd.io/etcd/etcdserver/api/v3rpc.newUnaryInterceptor.func1()
      /go/src/go.etcd.io/etcd/etcdserver/api/v3rpc/interceptor.go:60 +0xf0
  go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1()
      /go/src/go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:34 +0x1d9
  go.etcd.io/etcd/etcdserver/api/v3rpc.newLogUnaryInterceptor.func1()
      /go/src/go.etcd.io/etcd/etcdserver/api/v3rpc/interceptor.go:67 +0xda
  go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1()
      /go/src/go.etcd.io/etcd/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:39 +0x33d
  go.etcd.io/etcd/etcdserver/etcdserverpb._KV_Range_Handler()
      /go/src/go.etcd.io/etcd/etcdserver/etcdserverpb/rpc.pb.go:3483 +0x1d8
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).processUnaryRPC()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:1026 +0xa8b
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).handleStream()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:1252 +0x138d
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).serveStreams.func1.1()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:699 +0xac
Goroutine 54 (running) created at:
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).serveStreams.func1()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:697 +0xb8
  go.etcd.io/etcd/vendor/google.golang.org/grpc/internal/transport.(*http2Server).operateHeaders()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/internal/transport/http2_server.go:432 +0x14cb
  go.etcd.io/etcd/vendor/google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/internal/transport/http2_server.go:472 +0x37a
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).serveStreams()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:695 +0x170
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).handleRawConn.func2()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:649 +0x53
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).handleRawConn.func3()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:657 +0x34
Goroutine 121 (finished) created at:
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).serveStreams.func1()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:697 +0xb8
  go.etcd.io/etcd/vendor/google.golang.org/grpc/internal/transport.(*http2Server).operateHeaders()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/internal/transport/http2_server.go:432 +0x14cb
  go.etcd.io/etcd/vendor/google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/internal/transport/http2_server.go:472 +0x37a
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).serveStreams()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:695 +0x170
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).handleRawConn.func2()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:649 +0x53
  go.etcd.io/etcd/vendor/google.golang.org/grpc.(*Server).handleRawConn.func3()
      /go/src/go.etcd.io/etcd/vendor/google.golang.org/grpc/server.go:657 +0x34
==================
--- FAIL: TestResumeElection (0.07s)
    testing.go:771: race detected during execution of test
@mitake mitake mentioned this issue May 1, 2019
Merged
@jingyih
Copy link
Contributor

jingyih commented May 1, 2019

I just took a quick look. It might be related to #10526.
cc @xiang90

@mitake
Copy link
Contributor Author

mitake commented May 1, 2019

Thanks @jingyih !

@xiang90
Copy link
Contributor

xiang90 commented May 1, 2019

@jingyih How do you know it is related to #10526? From the stack trace, it seems that the data race happened between two rpc range go routines.

@xiang90
Copy link
Contributor

xiang90 commented May 1, 2019

@jingyih Oh. I see what is going on. It seems that the clone requires a write lock since it is modifying the tree structure. Can you help to change the RLock to Lock?

@xiang90 xiang90 added this to the etcd-v3.4 milestone May 1, 2019
@jingyih jingyih mentioned this issue May 1, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type/bug
Milestone
etcd-v3.4
Development

Successfully merging a pull request may close this issue.

mvcc: protect tree clone with write lock jingyih/etcd
3 participants
@mitake @xiang90 @jingyih