LOAD DATA behavior change causes upgrade issue

[ghost]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

See my comment here and pingcap/docs#3838

There is an issue in that to restore the previous behavior for LOAD DATA, because tidb_dml_batch_size is only session scope. Therefore, you must SET tidb_dml_batch_size=20000 for each session, but there are some common cases where that is hard to do:

• It is a closed source app (and they can't modify the code)
• The client library handles session scope in a weird way (by default Golang uses a connection pool, and will only provide session scope guarantees with a connection pool).

Because this is a breaking change, I propose allowing GLOBAL | SESSION scope. This will allow an upgrade path, because the TiDB server can be set to the previous behavior.

2. What did you expect to see? (Required)

A small change: tidb_dml_batch_size is SESSION | GLOBAL scope. I prefer the new behavior, I am just concerned about upgrades.

3. What did you see instead (Required)

SESSION scope only

4. What is your TiDB version? (Required)

mysql> select tidb_version()\G
*************************** 1. row ***************************
tidb_version(): Release Version: v4.0.0-beta.2-1142-gebb557386
Edition: Community
Git Commit Hash: ebb55738695d01048f747a43592331e3e2443988
Git Branch: master
UTC Build Time: 2020-09-03 15:56:39
GoVersion: go1.13
Race Enabled: false
TiKV Min Version: v3.0.0-60965b006877ca7234adaced7890d7b029ed1306
Check Table Before Drop: false
1 row in set (0.00 sec)

[ZenoTan]

/label sig/executor

[ti-srebot]

These labels are not found sig/executor.

[ZenoTan]

/label sig/execution

[sre-bot]

Integrity check：
component severity RCA symptom affect_version fix_version fields are empty

Please comment /info to get template

[ti-srebot]

Please edit this comment to complete the following information

Bug

1. Root Cause Analysis (RCA)

The bug is caused due to a change in the behavior of LOAD DATA. The new behavior is more correct but may lead to upgrade issues.

2. Symptom

An error due to transaction size too large.

3. All Trigger Conditions

see in issue body

4. Workaround (optional)

A partial workaround can be achieved by increasing the transaction size. For large LOAD DATA statements, this may not be possible. A shell script can be used to split the file into multiple chunks.

5. Affected versions

unreleased

6. Fixed versions

unplanned