feature: Integrate RocketMQ into Seata #3974

GasolLY · 2021-08-23T05:28:26Z

Ⅰ. Describe what this PR did

透过Spring的后置代理器实现了RocketMQ Bean的拦截，并返回了静态代理类.

Ⅱ. Does this pull request fix one issue?

fixes #3752.

Ⅲ. Test case

https://github.com/GasolLY/seata-rocketmq-test

Ⅳ. Describe how to verify it

Ⅴ. Special notes for reviews

codecov-commenter · 2021-09-28T01:59:24Z

Codecov Report

Merging #3974 (6a85838) into 2.x (25999c3) will decrease coverage by 0.23%.
The diff coverage is 6.25%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@             Coverage Diff              @@
##                2.x    #3974      +/-   ##
============================================
- Coverage     48.42%   48.20%   -0.23%     
+ Complexity     4171     4168       -3     
============================================
  Files           764      769       +5     
  Lines         26892    26979      +87     
  Branches       3348     3348              
============================================
- Hits          13023    13004      -19     
- Misses        12467    12591     +124     
+ Partials       1402     1384      -18

Impacted Files	Coverage Δ
...ration/tx/api/fence/DefaultCommonFenceHandler.java	`0.00% <0.00%> (ø)`
...gration/tx/api/fence/config/CommonFenceConfig.java	`0.00% <0.00%> (ø)`
...pi/fence/store/db/CommonFenceStoreDataBaseDAO.java	`0.00% <0.00%> (ø)`
...x/api/fence/store/db/sql/CommonFenceStoreSqls.java	`0.00% <0.00%> (ø)`
...erceptor/parser/DefaultResourceRegisterParser.java	`0.00% <0.00%> (ø)`
...ata/integration/tx/api/json/DefaultJsonParser.java	`0.00% <0.00%> (ø)`
.../autoconfigure/SeataRocketMQAutoConfiguration.java	`0.00% <0.00%> (ø)`
...ain/java/io/seata/rm/fence/SpringFenceHandler.java	`0.00% <0.00%> (ø)`
...otation/scannercheckers/PackageScannerChecker.java	`0.00% <0.00%> (ø)`
.../java/io/seata/rm/tcc/rocketmq/RocketMQAspect.java	`0.00% <0.00%> (ø)`
... and 20 more

…re-tcc-rocketmq # Conflicts: # tcc/pom.xml # tcc/src/main/java/io/seata/rm/tcc/rocketmq/TCCRocketMQ.java # tcc/src/main/java/io/seata/rm/tcc/rocketmq/TCCRocketMQImpl.java

…re-tcc-rocketmq # Conflicts: # seata-spring-boot-starter/src/main/java/io/seata/spring/boot/autoconfigure/RocketMQAutoConfiguration.java

seata-spring-boot-starter/pom.xml

...boot-starter/src/main/java/io/seata/spring/boot/autoconfigure/RocketMQAutoConfiguration.java

tcc/pom.xml

funky-eyes · 2021-10-14T10:28:46Z

tcc/src/main/java/io/seata/rm/tcc/rocketmq/TCCRocketMQImpl.java

+        SendResult sendResult = RocketMQUtils.halfSend(defaultMQProducer, message);
+        LOGGER.info("RocketMQ message send prepare, xid = {}, bid = {}", context.getXid(), context.getBranchId());
+        Map<String, Object> params = new HashMap<>(2);
+        params.put("message", message);


没必要传递msg

ok, I will fix it

如果只是为了后面endtransaction，这里是不用保持mesage的，sendresult中的信息足够。但是如果MQ的客户端设置了EndTransactionHook，内部实现回查message中获取TransactionId，如果没有message传入，应该会抛异常出来。

@ShannonDing 这种情况将有后续seata独立的seatadefaultproducer来处理，不会设置EndTransactionHook

# Conflicts: # seata-spring-boot-starter/src/main/resources/META-INF/spring.factories

wxbty · 2021-10-20T02:55:08Z

Aop is based on dynamic proxy, the middleware layer should use the underlying technology

GasolLY · 2021-10-20T11:15:34Z

Aop is based on dynamic proxy, the middleware layer should use the underlying technology
Get it😁

funky-eyes · 2021-10-21T09:24:24Z

再补充一下测试用例

funky-eyes · 2022-05-27T09:44:05Z

tcc/src/main/java/io/seata/rm/tcc/rocketmq/TCCRocketMQImpl.java

+    @Override
+    public SendResult prepare(BusinessActionContext context, Message message)
+        throws MQBrokerException, RemotingException, InterruptedException, MQClientException {
+        SendResult sendResult = RocketMQUtils.halfSend(defaultMQProducer, message);


此处有原子性问题，进入prepare分支已经注册到tc端，发出半消息后，rm宕机，此时sendResult已经不存在了，此时mq消息永久处于不可见状态

有记录transactionID么？是不是可以通过服务端的回查来决定是否commit or rollback？

有记录transactionID么？是不是可以通过服务端的回查来决定是否commit or rollback？

seata这边是回查不了的,所以只能保证commit/rollback主动通知,由于分支注册和发送半消息存在原子性问他,如果发送半消息后rm侧宕机,那么这个半消息回查可能是无法回查到具体的事务状态,所以目前我们打算出现这种原子性导致的低概率事件时,使用无限重试回查来使这个消息ttl到直接删除即可,因为半消息对consumer是不可见的,所以对一致性没影响,只要保证commit能一定通知到rocketmq即可

GasolLY · 2022-05-27T10:00:13Z

再补充一下测试用例

Fixed: https://github.com/GasolLY/seata-rocketmq-test

tcc/src/main/java/io/seata/rm/tcc/rocketmq/RocketMQUtils.java

CLAassistant · 2022-12-12T13:18:47Z

All committers have signed the CLA.

CLAassistant · 2022-12-12T13:18:51Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 4 committers have signed the CLA.

✅ wangliang181230
❌ GasolLY
❌ a364176773
❌ slievrly
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

funky-eyes

1.commit时由seata主动触发（目前pr已实现）
2.事务发生rollback时，由seata主动rollback，如果出现原子性问题导致分支没注册上先发了消息，那么反之这个消息是不可见的就可以不管，回查我们的listener直接unkown就好了（后半部分这块要第三点做出来）
3.需要seata自己的listener不能跟业务的耦合一起，需要重构出一个新的seatadefaultproducer类供用户使用
4.branch type增加rocketmq，这块要重构脱离tcc （重构）

Bughue · 2023-02-02T10:56:58Z

tcc/src/main/java/io/seata/rm/tcc/rocketmq/TCCRocketMQ.java

+     * @throws UnknownHostException
+     * @throws MQBrokerException
+     * @throws RemotingException
+     * @throws NoSuchFieldException


注释的异常不对应代码

Bughue · 2023-02-02T10:57:13Z

tcc/src/main/java/io/seata/rm/tcc/rocketmq/TCCRocketMQ.java

+     * @throws UnknownHostException
+     * @throws MQBrokerException
+     * @throws RemotingException
+     * @throws NoSuchFieldException


注释的异常不对应代码

Bughue · 2023-02-03T06:15:55Z

...a/io/seata/integration/tx/api/interceptor/handler/GlobalTransactionalInterceptorHandler.java

@@ -134,6 +112,25 @@ public GlobalTransactionalInterceptorHandler(FailureHandler failureHandler, Set<
        this.aspectTransactional = aspectTransactional;
    }

+    private void initDefaultGlobalTransactionTimeout() {


这个没必要挪位置吧，看不出来改动了什么

Bughue · 2023-02-03T06:34:46Z

...x-api/src/main/java/io/seata/integration/tx/api/fence/store/db/sql/CommonFenceStoreSqls.java

@@ -24,10 +24,6 @@
 */
 public class CommonFenceStoreSqls {

-    private CommonFenceStoreSqls() {


为什么夹杂这么多tccfence的改动，而且看起来好像都是调了位置而已

Bughue · 2023-02-03T06:44:16Z

tcc/src/main/java/io/seata/rm/tcc/rocketmq/SeataMQProducer.java

+
+    @Override
+    public void shutdown() {
+        target.shutdown();


4.9.3以下的shutdown有内存泄漏问题，这个可能要考虑一下怎样提醒用户

funky-eyes · 2023-11-27T02:56:55Z

Please resolve code conflicts

Bughue · 2024-01-18T11:14:07Z

#6230

feature-透过Spring的后置代理器实现了RocketMQ Bean的拦截，并返回了静态代理类

22f069c

slievrly added type: feature Category issues or prs related to feature request. Summer2021 first-time contributor first-time contributor labels Aug 24, 2021

Merge branch 'develop' into feature-tcc-rocketmq

9250bf4

GasolLY and others added 7 commits September 23, 2021 14:14

feature-初步实现TCC集成RocketMQ的目标

65b1d0d

初步完成tcc 集成 rocketmq

b0503b4

reformat code

44ada44

Merge branch 'develop' into feature-tcc-rocketmq

1a48e4a

添加了RocketMQAspect的自动配置，并对代码进行了优化

d9b01f4

将rocketmq自动配置类从seata-spring模块移动到seata-spring-boot-starter模块

077f473

Merge branch 'develop' into feature-tcc-rocketmq

f18251e

GasolLY added 7 commits September 28, 2021 20:20

代码优化

4c04890

Merge remote-tracking branch 'gitlab/feature-tcc-rocketmq' into featu…

9475bc3

…re-tcc-rocketmq # Conflicts: # tcc/pom.xml # tcc/src/main/java/io/seata/rm/tcc/rocketmq/TCCRocketMQ.java # tcc/src/main/java/io/seata/rm/tcc/rocketmq/TCCRocketMQImpl.java

merge修复

a2a0643

bugfix

e1f1f8a

bugfix

2f878ae

修改seata.rocketmq-enabled默认值为false

618bac2

Merge remote-tracking branch 'origin/feature-tcc-rocketmq' into featu…

0ecce21

…re-tcc-rocketmq # Conflicts: # seata-spring-boot-starter/src/main/java/io/seata/spring/boot/autoconfigure/RocketMQAutoConfiguration.java

slievrly added this to the 2.0.0 milestone Oct 8, 2021

funky-eyes reviewed Oct 14, 2021

View reviewed changes

Merge remote-tracking branch 'origin/develop' into feature-tcc-rocketmq

69909f2

# Conflicts: # seata-spring-boot-starter/src/main/resources/META-INF/spring.factories

使用静态代理重构代码

7528426

Merge branch 'develop' into feature-tcc-rocketmq

62cfc33

GasolLY changed the title ~~[WIP]Integrate RocketMQ into Seata~~ Integrate RocketMQ into Seata Mar 1, 2022

GasolLY changed the title ~~Integrate RocketMQ into Seata~~ feature:Integrate RocketMQ into Seata Mar 1, 2022

funky-eyes mentioned this pull request May 19, 2022

update README-cn.md dtm-labs/dtm#295

Closed

funky-eyes reviewed May 27, 2022

View reviewed changes

bug fix

038e7ea

ShannonDing reviewed Jul 31, 2022

View reviewed changes

tcc/src/main/java/io/seata/rm/tcc/rocketmq/RocketMQUtils.java Outdated Show resolved Hide resolved

funky-eyes changed the title ~~feature:Integrate RocketMQ into Seata~~ feature: Integrate RocketMQ into Seata Jan 30, 2023

funky-eyes changed the base branch from develop to 2.x January 30, 2023 05:29

funky-eyes added 3 commits January 30, 2023 13:30

Merge branch '2.x' into feature-tcc-rocketmq

cc4b655

opt

df29716

opt

74aaf43

funky-eyes requested a review from wangliang181230 February 1, 2023 06:22

opt

c22aea7

funky-eyes reviewed Feb 2, 2023

View reviewed changes

opt

6a85838

Bughue reviewed Feb 3, 2023

View reviewed changes

funky-eyes removed this from the 2.0.0 milestone Nov 4, 2023

funky-eyes added this to the 2.1.0 milestone Nov 27, 2023

Bughue mentioned this pull request Jan 18, 2024

feature : RocketMQ transaction are supported #6230

Merged

1 task

funky-eyes removed this from the 2.1.0 milestone Mar 1, 2024

funky-eyes closed this Mar 1, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feature: Integrate RocketMQ into Seata #3974

feature: Integrate RocketMQ into Seata #3974

GasolLY commented Aug 23, 2021 •

edited

Loading

codecov-commenter commented Sep 28, 2021 •

edited

Loading

funky-eyes Oct 14, 2021

GasolLY Oct 20, 2021

ShannonDing Jul 31, 2022

funky-eyes Feb 2, 2023

wxbty commented Oct 20, 2021

GasolLY commented Oct 20, 2021

funky-eyes commented Oct 21, 2021

funky-eyes May 27, 2022

ShannonDing Jul 31, 2022

funky-eyes Jul 31, 2022

GasolLY commented May 27, 2022

CLAassistant commented Dec 12, 2022 •

edited

Loading

CLAassistant commented Dec 12, 2022

funky-eyes left a comment

Bughue Feb 2, 2023

Bughue Feb 2, 2023

Bughue Feb 3, 2023

Bughue Feb 3, 2023

Bughue Feb 3, 2023

funky-eyes commented Nov 27, 2023

Bughue commented Jan 18, 2024

feature: Integrate RocketMQ into Seata #3974

feature: Integrate RocketMQ into Seata #3974

Conversation

GasolLY commented Aug 23, 2021 • edited Loading

Ⅰ. Describe what this PR did

Ⅱ. Does this pull request fix one issue?

Ⅲ. Test case

Ⅳ. Describe how to verify it

Ⅴ. Special notes for reviews

codecov-commenter commented Sep 28, 2021 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wxbty commented Oct 20, 2021

GasolLY commented Oct 20, 2021

funky-eyes commented Oct 21, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

GasolLY commented May 27, 2022

CLAassistant commented Dec 12, 2022 • edited Loading

CLAassistant commented Dec 12, 2022

funky-eyes left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

funky-eyes commented Nov 27, 2023

Bughue commented Jan 18, 2024

GasolLY commented Aug 23, 2021 •

edited

Loading

codecov-commenter commented Sep 28, 2021 •

edited

Loading

CLAassistant commented Dec 12, 2022 •

edited

Loading