Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement ShardedPeer #763

Open
aeplay opened this issue Nov 15, 2024 — with Linear · 0 comments
Open

Implement ShardedPeer #763

aeplay opened this issue Nov 15, 2024 — with Linear · 0 comments

Comments

Copy link
Contributor

aeplay commented Nov 15, 2024

For edge nodes of a sync server cluster, simple load balancing is enough to distribute load across cores & machines. But so far, these edge nodes had to rely on a single core server (and single CPU core) as their upstream. And that core server had to process updates to all CoValues, making it a bottleneck.

In order to parallelise the core server and start introducing redundancy, we need to shard it by CoValue ID.

To achieve this, we need to implement a ShardedPeer which consists of multiple other Peer definitions ("sub-peers"). It should multiplex outgoing messages to the correct peer by using Rendezvous Hashing - picking the sub-peer with the smallest value of hash(coValueID, peerName).

It should support dynamic changes to the existence/availability of sub-peers.

Incoming messages from all sub-peers should be coalesced to give the appearance of ShardedPeer being a single peer.

Note that a shard becoming unavailable will cause CoValue unavailability issues, because we only sync each CoValue with a single shard in this initial naive setup. In this case, the known-state handling of the ShardedPeer as implemented in the SyncManager also becomes slightly wrong for affected CoValues, because it reflects the state of the now-unavailable shard.

This is however already a marked improvement to the single-core-server situation and a first step to implement smarter behaviour in the future (such as picking two or three peers for each CoValue ID, which Rendezvous Hashing lets us gracefully upgrade to later)

Setting up the shard peers correctly and making sure the shard peers know about each other, too, is left as an exercise to the cluster deployer. This is important, because CoValue dependencies (such as value-owned-by-group, account-member-of-group, or group-extended-by-group) need to be resolved correctly, and depended-on CoValues might live on another shard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants