Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

reduce the execution time of go test -race for package ddl #24459

Closed
4 of 14 tasks
zimulala opened this issue May 7, 2021 · 9 comments
Closed
4 of 14 tasks

reduce the execution time of go test -race for package ddl #24459

zimulala opened this issue May 7, 2021 · 9 comments
Assignees
@zimulala
Labels
component/test type/enhancement The issue or PR belongs to an enhancement.

Comments

@zimulala
Copy link
Contributor

zimulala commented May 7, 2021 •
edited
Loading

Development Task

Update: 2021-05-07 17:35 CST

$ make failpoint-enable
$ cd ddl
$ GO111MODULE=on go test -v -vet=off -p 5 -timeout 20m -race

The execution time of more than 5s test cases in the different test suite are listed as the following:

suiteName.testCaseName execTime
testDDLSerialSuite.TestCancelJob 5.112s
testSerialSuite.TestCreateTableNoBlock 5.217s
testDBSuite2.TestTableLock 5.238s
testColumnSuite.TestColumn 5.524s
testDBSuite4.TestCancelDropPrimaryKey 5.686s
testDBSuite4.TestAnonymousIndex 5.772s
testDBSuite2.TestLock 5.883s
testStateChangeSuite.TestDDLIfExists 6.034s
testStateChangeSuite.TestDropNotNullColumn 6.065s
testIntegrationSuite9.TestMultiRegionGetTableEndHandle 6.171s
testIntegrationSuite2.TestUniqueKeyNullValueClusterIndex 6.200s
testDDLSuite.TestParallelDDL 6.247s
testIntegrationSuite5.TestAlterTableAddPartitionByList 6.483s
testIntegrationSuite1.TestPartitionAddIndex 6.705s
testIntegrationSuite4.TestAlterIndexVisibility 6.737s
testDBSuite1.TestAlterTableAlterPartition 6.739s
testSequenceSuite.TestSequenceAsDefaultValue 6.880s
testIntegrationSuite5.TestAlterTableAddPartition 7.090s
testStateChangeSuite.TestParallelAlterModifyColumnAndAddPK 7.114s
testStateChangeSuite.TestParallelAlterSchemaCharsetAndCollate 7.460s
testDBSuite3.TestCancelDropColumn 7.757s
testIntegrationSuite9.TestAutoRandomChangeFromAutoInc 7.758s
testIntegrationSuite6.TestAddColumnTooMany 8.138s
testDBSuite3.TestCancelDropColumns 8.190s
testDBSuite5.TestCancelTruncateTable 8.214s
testColumnTypeChangeSuite.TestColumnTypeChangeFromDateTimeTypeToOthers 8.248s
testDBSuite4.TestCancelAddIndex1 8.486s
testIntegrationSuite3.TestChangingCharsetToUtf8 9.469s
testIntegrationSuite5.TestAlterTableDropPartition 9.651s
testDBSuite3.TestAddAnonymousIndex 9.661s
testDBSuite5.TestAddIndexForGeneratedColumn 9.766s
testColumnTypeChangeSuite.TestColumnTypeChangeFromStringToOthers 10.079s
testPartitionSuite.TestAddPartitionReplicaBiggerThanTiFlashStores 10.297s
testIntegrationSuite3.TestCreateTableWithAutoIdCache 10.331s
testIntegrationSuite3.TestAlterColumn 10.710s
testDBSuite4.TestComment 10.725s
testColumnTypeChangeSuite.TestColumnTypeChangeFromNumericToOthers 11.209s
testIntegrationSuite5.TestPartitionAddPrimaryKey 11.225s
testDBSuite2.TestAddUniqueIndexRollback 11.411s
testDBSuite2.TestDropIndex 11.670s
testDDLTableSplitSuite.TestTableSplit 11.825s
testDBSuite7.TestAddIndexWithPK 11.935s
testIntegrationSuite3.TestChangeColumnPosition 12.433s
testIntegrationSuite1.TestCreateTableWithListColumnsPartition 12.692s
testIntegrationSuite4.TestExchangePartitionTableCompatiable 12.846s
testDBSuite5.TestAddIndex5 13.004s
testDBSuite1.TestAddPrimaryKeyRollback2 13.150s
testIntegrationSuite3.TestAlterAlgorithm 13.268s
testDBSuite3.TestFKOnGeneratedColumns 13.521s
testIntegrationSuite9.TestPrimaryKey 13.547s
testIntegrationSuite2.TestPartitionDropPrimaryKey 14.480s
testDBSuite5.TestAlterPrimaryKey 15.104s
testDBSuite1.TestCharacterSetInColumns 15.189s
testDBSuite6.TestDropColumn 15.283s
testDBSuite4.TestAddIndex4 15.536s
testStateChangeSuite.TestShowIndex 15.593s
testSerialDBSuite.TestModifyColumnnReorgInfo 15.831s
testIntegrationSuite9.TestInvisibleIndex 16.795s
testDBSuite1.TestAddIndex1 16.838s
testDBSuite2.TestAddIndex2 18.057s
testIntegrationSuite7.TestDuplicateErrorMessage 18.187s
testColumnTypeChangeSuite.TestColumnTypeChangeFromJsonToOthers 18.957s
testSequenceSuite.TestShowCreateSequence 19.774s
testDBSuite3.TestAddIndex3 19.844s
testIntegrationSuite4.TestPartitionCancelAddIndex 20.191s
testDBSuite5.TestAddPrimaryKeyRollback1 20.680s
testDBSuite1.TestColumn 22.858s
testDBSuite4.TestAddPrimaryKey4 23.818s
testSerialDBSuite.TestCommitTxnWithIndexChange 25.053s
testIntegrationSuite4.TestChangingTableCharset 25.201s
testDBSuite1.TestAddPrimaryKey1 28.116s
testIntegrationSuite4.TestAddPartitionTooManyPartitions 30.405s
testDBSuite2.TestAddPrimaryKey2 31.668s
testIntegrationSuite6.TestCreateTableTooLarge 36.040s
testIntegrationSuite1.TestCreateTableWithListPartition 38.099s
testDBSuite3.TestAddPrimaryKey3 38.724s
testIntegrationSuite3.TestMultiRegionGetTableEndHandle 42.431s
testSequenceSuite.TestSequenceFunction 42.713s
testIntegrationSuite2.TestPartitionCancelAddPrimaryKey 48.912s
testDBSuite3.TestCancelAddIndex 52.152s
testDBSuite5.TestCancelAddPrimaryKey 53.672s
testIntegrationSuite5.TestBackwardCompatibility 78.994s
testIntegrationSuite4.TestIndexOnMultipleGeneratedColumn 90.607s
testDBSuite1.TestModifyColumnTime 95.554s
testIntegrationSuite1.TestDisableTablePartition 147.742s

There are some test cases we may refine further:

suiteName.testCaseName execTime ranking creator
  • testDBSuite1.TestModifyColumnTime
95.554s 1 @jianzhiyao
  • testIntegrationSuite4.TestIndexOnMultipleGeneratedColumn
90.607s 1 @jianzhiyao
  • testIntegrationSuite5.TestBackwardCompatibility
78.994s 1 @zimulala
  • testDBSuite5.TestCancelAddPrimaryKey
53.672s 1
  • testDBSuite3.TestCancelAddIndex
52.152s 1
  • testIntegrationSuite2.TestPartitionCancelAddPrimaryKey
48.912s 1
  • testSequenceSuite.TestSequenceFunction
42.713s 1 @AilinKid
  • testIntegrationSuite3.TestMultiRegionGetTableEndHandle
42.431s 1
  • testDBSuite3.TestAddPrimaryKey3
38.724s 2
  • testIntegrationSuite1.TestCreateTableWithListPartition
38.099s 1 @crazycs520
  • testIntegrationSuite6.TestCreateTableTooLarge
36.040s 1
  • testDBSuite2.TestAddPrimaryKey2
31.668s 1
  • testIntegrationSuite4.TestAddPartitionTooManyPartitions
30.405s 1
Archived 2021-04-25 Recently optimized PR:
@zimulala zimulala added type/enhancement The issue or PR belongs to an enhancement. component/test help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. labels May 7, 2021
@jianzhiyao jianzhiyao mentioned this issue May 7, 2021
Merged
@xhebox
Copy link
Contributor

xhebox commented May 7, 2021 •
edited
Loading

@zimulala Actually, I am wonering skip -race for some of the tests. The patch from @jianzhiyao may improve the speed but will consume more memory. TestModifyColumnTime is 8s vs 30s on my PC without race or with race.

@zimulala
Copy link
Contributor Author

zimulala commented May 8, 2021

@xhebox Some tests can skip '-race', and which tests you are working on can be explained here. Avoid duplication of work.

@jianzhiyao jianzhiyao mentioned this issue May 8, 2021
Merged
@AilinKid
Copy link
Contributor

AilinKid commented May 10, 2021 •
edited
Loading

One question: why separate tests into small cases can reduce the execution time/memory usage (run parallelly? / mem resue?).

@xhebox
Copy link
Contributor

xhebox commented May 10, 2021

One question: why separate tests into small cases can reduce the execution time/memory usage (run parallelly? / mem resue?).

Maybe mem will increase, but concurrency does help.

@AilinKid
Copy link
Contributor

One question: why separate tests into small cases can reduce the execution time/memory usage (run parallelly? / mem resue?).

Maybe mem will increase, but concurrency does help.

Seems worthy

@zimulala zimulala mentioned this issue May 18, 2021
Merged
@zimulala zimulala self-assigned this May 18, 2021
@zimulala zimulala removed the help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. label May 18, 2021
@zimulala zimulala mentioned this issue May 19, 2021
Merged
@zimulala
Copy link
Contributor Author

zimulala commented May 20, 2021 •
edited
Loading

Before CI using PingCAP-QE/ci#194.

branch go test ddl package with race( exec go test -v -vet=off -p 5 -timeout 20m -race locally) CI (Race Test Chunk #7 ddl-other)
before this PR(#24704) 566s 6m 13s
after this PR(#24704) 505s 6m33s

@zimulala zimulala mentioned this issue May 20, 2021
Merged
@zimulala
Copy link
Contributor Author

zimulala commented May 20, 2021 •
edited
Loading

After CI using PingCAP-QE/ci#194. Ci (Race Test Chunk #7 ddl-other) is not accurate, so there is no comparison here.

branch CI (Race Test Chunk #7 ddl-other) in local
before this PR(#24704) 375s
after this PR 267s

@zimulala
Copy link
Contributor Author

zimulala commented May 21, 2021 •
edited
Loading

After PR #24780:

branch go test ddl package with race( exec go test -v -vet=off -p 5 -timeout 20m -race locally) CI (Race Test Chunk #7 ddl-other) CI (Race Test Chunk #7 ddl-other) in local
before this PR(#24704) 505s 6m 22s 379s
after this PR (#24780) 405s 2m 45s 199s

@zimulala
Copy link
Contributor Author

The time of CI (Race Test Chunk #7 ddl-other) is reduced to 3min, and this issue can be closed first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@zimulala zimulala

Labels
component/test type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@zimulala @xhebox @AilinKid