Idle TiDB keeps OOM because of AnalyzeColumnExec subMergeWorker

[sticnarf]

Bug Report

1. Minimal reproduce step (Required)

I have a 1 TiDB + 3 TiKV cluster containing only one table. The workload is insert only, each row is about 64 KiB.

Now the table has about 10 million rows. But after I stopped writing, TiDB keeps OOM. The TiDB node has 32 GB memory.

I did a heap profiling and found most memory is used by subMergeWorker.

Heap:
heap.gz

The estimated inuse memory increases like stairs:

2. What did you expect to see? (Required)

When idling, TiDB memory usage should be stable and low.

3. What did you see instead (Required)

The memory usage of TiDB keeps growing and OOM.

4. What is your TiDB version? (Required)

Release Version: v5.2.0-alpha-603-g13f2a661d
Edition: Community
Git Commit Hash: 13f2a661d9be5f9e8a0d8cd592a5f3f5973dee52
Git Branch: master
UTC Build Time: 2021-08-12 11:14:56
GoVersion: go1.16.4
Race Enabled: false
TiKV Min Version: v3.0.0-60965b006877ca7234adaced7890d7b029ed1306
Check Table Before Drop: false

[sticnarf]

Could you take a look at the problem? @winoros

[winoros]

/assign

[winoros]

this is a duplicate issue with #27187
known with the limitation of current sampling algo.
you can set @@tidb_analyze_version=1
@sticnarf

[sticnarf]

@winoros Thanks. I failed to find the duplicated issue because that one does not include the subMergeWorker keywords.

[ti-srebot]

Please edit this comment or add a new comment to complete the following information

Not a bug

1. Remove the 'type/bug' label
2. Add notes to indicate why it is not a bug

Duplicate bug

1. Add the 'type/duplicate' label
2. Add the link to the original bug

Bug

Note: Make Sure that 'component', and 'severity' labels are added
Example for how to fill out the template: #20100

1. Root Cause Analysis (RCA) (optional)

2. Symptom (optional)

3. All Trigger Conditions (optional)

4. Workaround (optional)

5. Affected versions

6. Fixed versions