DATA RACE in the executor.(*IndexNestedLoopHashJoin) #49692

hawkingrei · 2023-12-22T00:56:02Z

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

2. What did you expect to see? (Required)

3. What did you see instead (Required)

==================
WARNING: DATA RACE
Read at 0x00c006fd8bd8 by goroutine 47:
  github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).getResultFromChannel()
      pkg/executor/index_lookup_hash_join.go:294 +0x1a4
  github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).runUnordered()
      pkg/executor/index_lookup_hash_join.go:256 +0x88
  github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).Next()
      pkg/executor/index_lookup_hash_join.go:232 +0x1cf
  github.com/pingcap/tidb/pkg/executor/internal/exec.Next()
      pkg/executor/internal/exec/executor.go:309 +0x34b
  github.com/pingcap/tidb/pkg/executor.(*ExecStmt).next()
      pkg/executor/adapter.go:1245 +0x84
  github.com/pingcap/tidb/pkg/executor.(*recordSet).Next()
      pkg/executor/adapter.go:156 +0x19a
  github.com/pingcap/tidb/pkg/session.(*execStmtResult).Next()
      <autogenerated>:1 +0x6c
  github.com/pingcap/tidb/pkg/session.GetRows4Test()
      pkg/session/tidb.go:363 +0xd2
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).QueryToErr()
      pkg/testkit/testkit.go:239 +0x2ea
  pkg/executor/test/jointest/jointest_test.TestIssue30211()
      pkg/executor/test/jointest/join_test.go:780 +0x284
  github.com/pingcap/tidb/pkg/session.runStmt()
      pkg/session/session.go:2386 +0x71a
  github.com/pingcap/tidb/pkg/session.(*session).ExecuteStmt()
      pkg/session/session.go:2228 +0x1af8
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).ExecWithContext()
      pkg/testkit/testkit.go:364 +0x889
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).MustExecWithContext()
      pkg/testkit/testkit.go:136 +0xab
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).MustExec()
      pkg/testkit/testkit.go:131 +0x128
  pkg/executor/test/jointest/jointest_test.TestIssue30211()
      pkg/executor/test/jointest/join_test.go:776 +0x187
  github.com/pingcap/tidb/pkg/domain.(*Domain).LoadSysVarCacheLoop()
      pkg/domain/domain.go:1715 +0x93
  github.com/pingcap/tidb/pkg/session.bootstrapSessionImpl()
      pkg/session/session.go:3350 +0x724
  github.com/pingcap/tidb/pkg/domain.(*Domain).GetSessionCache()
      pkg/domain/sysvar_cache.go:62 +0x50
  github.com/pingcap/tidb/pkg/session.(*session).loadCommonGlobalVariablesIfNeeded()
      pkg/session/session.go:3722 +0xf4
  github.com/pingcap/tidb/pkg/session.(*session).ExecuteStmt()
      pkg/session/session.go:2088 +0x184
  github.com/pingcap/tidb/pkg/session.(*session).ExecuteInternal()
      pkg/session/session.go:1600 +0x3b9
  github.com/pingcap/tidb/pkg/domain.(*Domain).LoadPrivilegeLoop()
      pkg/domain/domain.go:1658 +0x108
  github.com/pingcap/tidb/pkg/session.bootstrapSessionImpl()
      pkg/session/session.go:3343 +0x6ca
  github.com/pingcap/tidb/pkg/session.BootstrapSession()
      pkg/session/session.go:3264 +0xad
  github.com/pingcap/tidb/pkg/testkit.bootstrap()
      pkg/testkit/mockstore.go:227 +0x94
  github.com/pingcap/tidb/pkg/testkit.CreateMockStoreAndDomain()
      pkg/testkit/mockstore.go:200 +0xd5
  github.com/pingcap/tidb/pkg/testkit.CreateMockStore()
      pkg/testkit/mockstore.go:68 +0x3d1
  pkg/executor/test/jointest/jointest_test.TestIssue30211()
      pkg/executor/test/jointest/join_test.go:754 +0x49
  testing.tRunner()
      GOROOT/src/testing/testing.go:1595 +0x261
  testing.(*T).Run.func1()
      GOROOT/src/testing/testing.go:1648 +0x44
Previous write at 0x00c006fd8bd8 by goroutine 42866:
  github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).finishJoinWorkers()
      pkg/executor/index_lookup_hash_join.go:204 +0x33b
  github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).finishJoinWorkers-fm()
      <autogenerated>:1 +0x47
  github.com/pingcap/tidb/pkg/util.WithRecovery.func1()
      pkg/util/misc.go:88 +0x5d
  runtime.gopanic()
      GOROOT/src/runtime/panic.go:920 +0x26f
  github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).newInnerWorker()
      pkg/executor/index_lookup_hash_join.go:469 +0xfca
  github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).startWorkers.func2()
      pkg/executor/index_lookup_hash_join.go:186 +0x58
  github.com/pingcap/tidb/pkg/util.WithRecovery()
      pkg/util/misc.go:96 +0x77
  github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).startWorkers.func4()
      pkg/executor/index_lookup_hash_join.go:186 +0x44
Goroutine 47 (running) created at:
  testing.(*T).Run()
      GOROOT/src/testing/testing.go:1648 +0x845
  testing.runTests.func1()
      GOROOT/src/testing/testing.go:2054 +0x84
  testing.tRunner()
      GOROOT/src/testing/testing.go:1595 +0x261
  testing.runTests()
      GOROOT/src/testing/testing.go:2052 +0x8ad
  testing.(*M).Run()
      GOROOT/src/testing/testing.go:1925 +0xcd7
  go.uber.org/goleak.VerifyTestMain()
      external/org_uber_go_goleak/testmain.go:53 +0x64
  pkg/executor/test/jointest/jointest_test.TestMain()
      pkg/executor/test/jointest/main_test.go:44 +0x676
  main.main()
      bazel-out/k8-fastbuild/bin/pkg/executor/test/jointest/jointest_test_/testmain.go:137 +0x5cb
Goroutine 42866 (finished) created at:
  github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).startWorkers()
      pkg/executor/index_lookup_hash_join.go:186 +0x8dc
  github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).Next()
      pkg/executor/index_lookup_hash_join.go:225 +0x84
  github.com/pingcap/tidb/pkg/executor/internal/exec.Next()
      pkg/executor/internal/exec/executor.go:309 +0x34b
  github.com/pingcap/tidb/pkg/executor.(*ExecStmt).next()
      pkg/executor/adapter.go:1245 +0x84
  github.com/pingcap/tidb/pkg/executor.(*recordSet).Next()
      pkg/executor/adapter.go:156 +0x19a
  github.com/pingcap/tidb/pkg/session.(*execStmtResult).Next()
      <autogenerated>:1 +0x6c
  github.com/pingcap/tidb/pkg/session.GetRows4Test()
      pkg/session/tidb.go:363 +0xd2
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).QueryToErr()
      pkg/testkit/testkit.go:239 +0x2ea
  pkg/executor/test/jointest/jointest_test.TestIssue30211()
      pkg/executor/test/jointest/join_test.go:780 +0x284
  github.com/pingcap/tidb/pkg/session.runStmt()
      pkg/session/session.go:2386 +0x71a
  github.com/pingcap/tidb/pkg/session.(*session).ExecuteStmt()
      pkg/session/session.go:2228 +0x1af8
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).ExecWithContext()
      pkg/testkit/testkit.go:364 +0x889
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).MustExecWithContext()
      pkg/testkit/testkit.go:136 +0xab
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).MustExec()
      pkg/testkit/testkit.go:131 +0x128
  pkg/executor/test/jointest/jointest_test.TestIssue30211()
      pkg/executor/test/jointest/join_test.go:776 +0x187
  github.com/pingcap/tidb/pkg/domain.(*Domain).LoadSysVarCacheLoop()
      pkg/domain/domain.go:1715 +0x93
  github.com/pingcap/tidb/pkg/session.bootstrapSessionImpl()
      pkg/session/session.go:3350 +0x724
  github.com/pingcap/tidb/pkg/domain.(*Domain).GetSessionCache()
      pkg/domain/sysvar_cache.go:62 +0x50
  github.com/pingcap/tidb/pkg/session.(*session).loadCommonGlobalVariablesIfNeeded()
      pkg/session/session.go:3722 +0xf4
  github.com/pingcap/tidb/pkg/session.(*session).ExecuteStmt()
      pkg/session/session.go:2088 +0x184
  github.com/pingcap/tidb/pkg/session.(*session).ExecuteInternal()
      pkg/session/session.go:1600 +0x3b9
  github.com/pingcap/tidb/pkg/domain.(*Domain).LoadPrivilegeLoop()
      pkg/domain/domain.go:1658 +0x108
  github.com/pingcap/tidb/pkg/session.bootstrapSessionImpl()
      pkg/session/session.go:3343 +0x6ca
  github.com/pingcap/tidb/pkg/session.BootstrapSession()
      pkg/session/session.go:3264 +0xad
  github.com/pingcap/tidb/pkg/testkit.bootstrap()
      pkg/testkit/mockstore.go:227 +0x94
  github.com/pingcap/tidb/pkg/testkit.CreateMockStoreAndDomain()
      pkg/testkit/mockstore.go:200 +0xd5
  github.com/pingcap/tidb/pkg/testkit.CreateMockStore()
      pkg/testkit/mockstore.go:68 +0x3d1
  pkg/executor/test/jointest/jointest_test.TestIssue30211()
      pkg/executor/test/jointest/join_test.go:754 +0x49
  testing.tRunner()
      GOROOT/src/testing/testing.go:1595 +0x261
  testing.(*T).Run.func1()
      GOROOT/src/testing/testing.go:1648 +0x44
==================

4. What is your TiDB version? (Required)

The text was updated successfully, but these errors were encountered:

hawkingrei · 2023-12-22T00:56:28Z

https://prow.hawkingrei.com/view/gs/prowpingcap/logs/tidb_race/1737931843371536384#1:build-log.txt%3A2577

qw4990 · 2024-05-20T09:42:53Z

IndexNestedLoopHashJoin.panicErr is not thread-safe:

Transfer this issue to Executor Team.

qw4990 · 2024-05-20T09:43:12Z

PTAL @XuHuaiyu

close #49692

hawkingrei added the type/bug The issue is confirmed as a bug. label Dec 22, 2023

hawkingrei changed the title ~~DATA RACE in the executor.(*IndexNestedLoopHashJoin~~ DATA RACE in the executor.(*IndexNestedLoopHashJoin) Dec 22, 2023

hawkingrei mentioned this issue Dec 22, 2023

flaky test list #41316

Open

seiya-annie added component/test severity/major labels Dec 22, 2023

ti-chi-bot bot added may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-6.1 may-affects-6.5 may-affects-7.1 may-affects-7.5 labels Dec 22, 2023

ti-chi-bot added the affects-8.1 label Apr 9, 2024

jebter added the sig/planner SIG: Planner label Apr 29, 2024

qw4990 assigned hawkingrei May 8, 2024

qw4990 added sig/execution SIG execution and removed sig/planner SIG: Planner labels May 20, 2024

qw4990 unassigned hawkingrei May 20, 2024

solotzg mentioned this issue Sep 3, 2024

executor: fix data race in IndexNestedLoopHashJoin #55824

Merged

13 tasks

windtalker added affects-6.1 affects-6.5 affects-7.1 affects-7.5 and removed may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-6.1 may-affects-6.5 may-affects-7.1 may-affects-7.5 labels Sep 4, 2024

ti-chi-bot bot closed this as completed in #55824 Sep 4, 2024

ti-chi-bot bot closed this as completed in 8cdd449 Sep 4, 2024

ti-chi-bot bot pushed a commit that referenced this issue Sep 9, 2024

executor: fix data race in IndexNestedLoopHashJoin (#55824) (#55909)

58bb25d

close #49692

ti-chi-bot bot pushed a commit that referenced this issue Sep 10, 2024

executor: fix data race in IndexNestedLoopHashJoin (#55824) (#55908)

5665343

close #49692

ti-chi-bot bot pushed a commit that referenced this issue Sep 10, 2024

executor: fix data race in IndexNestedLoopHashJoin (#55824) (#55911)

854bf3c

close #49692

ti-chi-bot bot pushed a commit that referenced this issue Sep 10, 2024

executor: fix data race in IndexNestedLoopHashJoin (#55824) (#55910)

acc2a65

close #49692

ti-chi-bot bot pushed a commit that referenced this issue Sep 10, 2024

executor: fix data race in IndexNestedLoopHashJoin (#55824) (#55907)

92acec0

close #49692

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DATA RACE in the executor.(*IndexNestedLoopHashJoin) #49692

DATA RACE in the executor.(*IndexNestedLoopHashJoin) #49692

hawkingrei commented Dec 22, 2023

hawkingrei commented Dec 22, 2023

qw4990 commented May 20, 2024

qw4990 commented May 20, 2024

DATA RACE in the executor.(*IndexNestedLoopHashJoin) #49692

DATA RACE in the executor.(*IndexNestedLoopHashJoin) #49692

Comments

hawkingrei commented Dec 22, 2023

Bug Report

1. Minimal reproduce step (Required)

2. What did you expect to see? (Required)

3. What did you see instead (Required)

4. What is your TiDB version? (Required)

hawkingrei commented Dec 22, 2023

qw4990 commented May 20, 2024

qw4990 commented May 20, 2024