-
Notifications
You must be signed in to change notification settings - Fork 642
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ISSUE #4852] Support Jraft algorithm as meta storage #4862
Conversation
Please be noted that you're linking an irrelavant issue. |
May you please introduce briefly that how JRaft stores metadata and how to retrieve them (such as subscription info) from clients (such as EventMesh Dashboard or user)? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall good job.
...tmesh-meta/eventmesh-meta-raft/src/main/java/org/apache/eventmesh/meta/raft/MetaService.java
Outdated
Show resolved
Hide resolved
# metaStorage plugin: raft | ||
#eventMesh.metaStorage.raft.dataPath=/tmp/server1 | ||
#eventMesh.metaStorage.raft.self=127.0.0.1:9091 | ||
#eventMesh.metaStorage.raft.members=127.0.0.1:9092,127.0.0.1:9093 | ||
#eventMesh.metaStorage.raft.electionTimeout=5 | ||
#eventMesh.metaStorage.raft.snapshotInterval=30 | ||
#eventMesh.metaStorage.raft.refreshLeaderInterval=3 | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
May you please leave a usage document/comment/URL that explains the meaning of these properties?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
eventMesh.metaStorage.raft.dataPath=/tmp/server1 // Path to local snapshot storage for raft data
Is server1
the recommended path of JRaft? Maybe /tmp/eventmesh-meta-raft
would be better to understand.
eventMesh.metaStorage.raft.members=127.0.0.1:9092,127.0.0.1:9093 // Raft members node ip and port
My guess is that the IP addresses are different for different nodes, but the port should be the same to allow scaling. The example here requires changing the IP address and maintaining the default port number.
May you please add these comments into eventmesh.properties
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, this example was configured locally for ease of testing, and it should be more appropriate to keep the same port numbers
My understanding, Jraft is to synchronise the data to the cluster internal machine, each machine above the data is consistent, this PR is to save the data in memory, every once in a while will snapshot the data to the local disk, start the server will snapshot the data loaded into memory, EventMesh Runtime needs to provide restful api in order to allow EventMesh Dashboard to get the data saved by the jraft! |
May you please indicate which class to interact with in order to get JRaft data? |
You can get MetaStorage in the Runtime, and then get MetaService, which is the implementation class of RaftMetaService. |
Please add known dependencies~ |
Implemented JRaft Feature in this PRIn the current implementation of this PR, if a new node is added to the cluster, users need to add the new node to the Hopefully you can implement After it's implemented, the newly added node can connect to any online node in the cluster. If connected to a follower, the current leader will be returned to new node by the follower. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
If you get bugs during use, I will continue to improve it later, thanks! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@karsonto Please fix conflicts
@mxsm Done,please review. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Fixes #4852
Motivation
Support Jraft algorithm as meta storage
Modifications
Removed unused variable responseEventMeshCommand in SendSyncMessageProcessor#processRequest
Documentation
New Configuration Information Explanation