Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[distributed storage] support re-syncing counters after a peer is partitioned. #343

Merged
merged 2 commits into from
May 24, 2024

Conversation

chirino
Copy link
Contributor

@chirino chirino commented May 23, 2024

This likely needs more thought put into it. A peer only resyncs when it goes from having zero active connections to 1 active connection.

@chirino chirino changed the title [distributed storage] support re-syncing counters after a peer is par… [distributed storage] support re-syncing counters after a peer is partitioned. May 23, 2024
@chirino chirino force-pushed the distributed-re-sync branch from db635a9 to 9e88f5b Compare May 23, 2024 19:55
@chirino
Copy link
Contributor Author

chirino commented May 23, 2024

Trying to avoid doing too many re-sync on every connect to a peer, but I think we could end up in scenarios where you have peers A, B, C, D and
peer A and B start up and connect and sync,
then C and D start up and connect and sync,
then C,D and A,B connect and don't sync anymore as they thought they were already synced with the cluster.

…titioned.

Signed-off-by: Hiram Chirino <hiram@hiramchirino.com>
@chirino chirino force-pushed the distributed-re-sync branch from 9e88f5b to 0f45043 Compare May 23, 2024 20:33
@chirino
Copy link
Contributor Author

chirino commented May 23, 2024

Switched to a more simplified strategy of syncing with every peer session. It should work better in the face of complex network partitions.

limitador/src/storage/distributed/mod.rs Outdated Show resolved Hide resolved
limitador/src/storage/distributed/mod.rs Show resolved Hide resolved
limitador/src/storage/distributed/mod.rs Show resolved Hide resolved
limitador/src/storage/distributed/mod.rs Outdated Show resolved Hide resolved
* rewrote the inter_ourselfs_inner, we now avoid a full clone
* remove unneeded clone of the limits.
* avoid sending a re-sync counter if it has expired.
Copy link
Member

@alexsnaps alexsnaps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I'll try to wrap my head around all of this "better" next week.
But there is no world in which all of this PR isn't making this better already!

limitador/src/storage/distributed/grpc/mod.rs Show resolved Hide resolved
limitador/src/storage/distributed/mod.rs Show resolved Hide resolved
@chirino chirino merged commit c88628f into Kuadrant:main May 24, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants