You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
After a few experiments, we can introduce delta-based quicksync.
From the client side, the process of quicksync would look like:
IF the DB exists, just download batches and restore batch after batch
IF DB does not exist, download normal quicksync (full db state)
Process of batch downloading:
each batch is produced with start_layer,end_layer,start_layer_minus_one_db_hash. The layer hash NEEDS TO BE COMPARED before restoring with the local state
IF hash is different then proceed to PREVIOUS batch till the hash matches
after restoring from partial you can continue without layer checks.
The server side publishes partials in predefined sizes OR times so do not hardcode the start-end span. All values in metadata.csv are sorted by start_layer.
Restoring same partial twice should be noop and is considered safe. BE WARNED that there are NO deletes during the restore process (that's why absolutely make sure that you compare the layer_hash).
It is safe to stop / restart / retry at any stage after restoring from any number of partials
Objects that are not tied to time or layer are remembered on the server side and added to the first batch when they appeared in the DB.
The process should be integrated into quicksync-rs and the initial integration was provided in spacemeshos/quicksync-rs#49
The text was updated successfully, but these errors were encountered:
After a few experiments, we can introduce delta-based quicksync.
From the client side, the process of quicksync would look like:
Process of batch downloading:
start_layer,end_layer,start_layer_minus_one_db_hash
. The layer hash NEEDS TO BE COMPARED before restoring with the local stateThe server side publishes partials in predefined sizes OR times so do not hardcode the start-end span. All values in
metadata.csv
are sorted by start_layer.Objects that are not tied to time or layer are remembered on the server side and added to the first batch when they appeared in the DB.
The process should be integrated into
quicksync-rs
and the initial integration was provided in spacemeshos/quicksync-rs#49The text was updated successfully, but these errors were encountered: