Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

optimize: When the number of primary keys exceeds 1000, use union to concatenate the SQL #6957 #7012

Merged
merged 11 commits into from
Dec 25, 2024

Conversation

remind
Copy link
Contributor

@remind remind commented Nov 18, 2024

…oncatenate the SQL (#6957)

  • I have registered the PR changes.

Ⅰ. Describe what this PR did

When the number of primary keys exceeds 1000, use union to concatenate the SQL

Ⅱ. Does this pull request fix one issue?

fixes #6957

Ⅲ. Why don't you add test cases (unit test/integration test)?

Ⅳ. Describe how to verify it

sql生成有测试用例:
io.seata.rm.datasource.SqlGenerateUtilsTest
此测试类之前已经存在,此次在此基础上进行了修改

其它受影响的地方依赖其它已有测试用例,由于该问题的触发场景比较极端,必须要超过1000个主键,并且该1000的配置是private的,有部分场景没看到之前有编写超过1000时的测试用例,本地通过修改配置值进行了单元测试,不确定集成测试是否有此场景的测试。

Ⅴ. Special notes for reviews

1、io.seata.rm.datasource.undo.AbstractUndoExecutor#queryCurrentRecords 该方法是使用了For Update,之前是一条sql一次性执行完,当主键数量超过1000时现在会拆分成多条sql循环执行,每条执行会有间隔,如果有同样的数据重复处理,并且是不同事务的,可能会导致死锁。

2、由于sql采用了union拼接,影响到了SQLSelectQueryBlock的获取,因此在*SelectForUpdateRecognizer此种类中,如io.seata.sqlparser.druid.mysql.MySQLSelectForUpdateRecognizer#getSelect,修改了获取方式,是否有可能造成影响。

Copy link

codecov bot commented Nov 20, 2024

Codecov Report

Attention: Patch coverage is 94.33962% with 3 lines in your changes missing coverage. Please review.

Project coverage is 52.65%. Comparing base (26950b6) to head (bd674c0).
Report is 2 commits behind head on 2.x.

Files with missing lines Patch % Lines
...qlparser/druid/dm/DmSelectForUpdateRecognizer.java 0.00% 1 Missing ⚠️
.../druid/oracle/OracleSelectForUpdateRecognizer.java 0.00% 1 Missing ⚠️
...ostgresql/PostgresqlSelectForUpdateRecognizer.java 0.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##                2.x    #7012      +/-   ##
============================================
- Coverage     52.66%   52.65%   -0.01%     
- Complexity     6643     6663      +20     
============================================
  Files          1126     1129       +3     
  Lines         40142    40168      +26     
  Branches       4707     4708       +1     
============================================
+ Hits          21139    21151      +12     
- Misses        16990    17000      +10     
- Partials       2013     2017       +4     
Files with missing lines Coverage Δ
...g/apache/seata/rm/datasource/SqlGenerateUtils.java 96.22% <100.00%> (+1.10%) ⬆️
.../rm/datasource/exec/BaseTransactionalExecutor.java 74.12% <100.00%> (+0.12%) ⬆️
.../seata/rm/datasource/exec/MultiUpdateExecutor.java 80.00% <100.00%> (-0.27%) ⬇️
...pache/seata/rm/datasource/exec/UpdateExecutor.java 83.33% <100.00%> (-0.76%) ⬇️
...datasource/exec/mysql/MySQLUpdateJoinExecutor.java 77.56% <100.00%> (-0.70%) ⬇️
...seata/rm/datasource/undo/AbstractUndoExecutor.java 67.53% <100.00%> (+0.86%) ⬆️
...er/druid/mysql/MySQLSelectForUpdateRecognizer.java 78.37% <100.00%> (ø)
...id/sqlserver/SqlServerOperateRecognizerHolder.java 68.42% <100.00%> (ø)
...qlparser/druid/dm/DmSelectForUpdateRecognizer.java 0.00% <0.00%> (ø)
.../druid/oracle/OracleSelectForUpdateRecognizer.java 11.42% <0.00%> (ø)
... and 1 more

... and 8 files with indirect coverage changes

@slievrly slievrly modified the milestones: 2.3.0, 2.4.0 Dec 1, 2024
Copy link
Contributor

@funky-eyes funky-eyes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@funky-eyes funky-eyes merged commit 340ea92 into apache:2.x Dec 25, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
first-time contributor first-time contributor mode: AT AT transaction mode module/rm-datasource rm-datasource module type: optimize
Projects
None yet
Development

Successfully merging this pull request may close these issues.

分支注册,主键超过1000的时候会转为or,但是使用or在mysql索引会失效
4 participants