Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ISSUE #8698] Remove batch write in kv cq store and update rocksdb cq check tool #8739

Merged
merged 13 commits into from
Sep 25, 2024

Conversation

LetLetMe
Copy link
Contributor

@LetLetMe LetLetMe commented Sep 24, 2024

Which Issue(s) This PR Fixes

  1. Fixes [Bug] rocksdbcq batch dispatch should be removed #8698
  2. edit checkRocksdbCqWriteProgress admin tool

Brief Description

How Did You Test This Change?

@LetLetMe
Copy link
Contributor Author

In order to minimize changes, we have opted to set the batch_size to 1 this time. However, this solution is not perfect, and we will need to use a more rigorous approach in the future to completely resolve this issue.

为了尽量少的变更,我们此次采用了batch_size改为1的方式,但这个方案不完美,后续还是要用更严谨的方式,彻底修复这个问题

@LetLetMe
Copy link
Contributor Author

To avoid changing the default logic of the currently deployed cluster, we will make the batch_size configurable and consider a complete optimization later. @RongtongJin

为了不改变现有已部署集群的默认逻辑,我们把batch_size改为可配置,后续在考虑彻底优化 @RongtongJin

@codecov-commenter
Copy link

codecov-commenter commented Sep 24, 2024

Codecov Report

Attention: Patch coverage is 34.61538% with 17 lines in your changes missing coverage. Please review.

Project coverage is 47.34%. Comparing base (525f877) to head (3556074).
Report is 12 commits behind head on develop.

Files with missing lines Patch % Lines
...ocketmq/broker/processor/AdminBrokerProcessor.java 0.00% 11 Missing ⚠️
...mand/queue/CheckRocksdbCqWriteProgressCommand.java 0.00% 3 Missing ⚠️
...ache/rocketmq/store/config/MessageStoreConfig.java 66.66% 1 Missing ⚠️
...ketmq/store/plugin/AbstractPluginMessageStore.java 0.00% 1 Missing ⚠️
.../apache/rocketmq/tools/command/MQAdminStartup.java 0.00% 1 Missing ⚠️
Additional details and impacted files
@@              Coverage Diff              @@
##             develop    #8739      +/-   ##
=============================================
- Coverage      47.48%   47.34%   -0.15%     
+ Complexity     11571    11549      -22     
=============================================
  Files           1282     1282              
  Lines          89813    89848      +35     
  Branches       11553    11557       +4     
=============================================
- Hits           42652    42536     -116     
- Misses         41928    42063     +135     
- Partials        5233     5249      +16     
Flag Coverage Δ
47.34% <34.61%> (-0.15%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@LetLetMe
Copy link
Contributor Author

image
image
I made some modifications to the tool, and the actual effect of the tool is shown in the image.
修改了下工具,工具实际效果如图所示

@LetLetMe LetLetMe changed the title [Enhancement] Remove kvCQ batch write [Enhancement] Remove kvCQ batch write and edit checkRocksdbCqWriteProgress admin tool Sep 25, 2024
@lizhimins lizhimins changed the title [Enhancement] Remove kvCQ batch write and edit checkRocksdbCqWriteProgress admin tool [ISSUE #8698] Remove batch write in kv cq store and update rocksdb cq check tool Sep 25, 2024
@lizhimins lizhimins merged commit 3b5cbf8 into apache:develop Sep 25, 2024
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] rocksdbcq batch dispatch should be removed
5 participants