Data columns initial sync: Refactor #14522
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull requests rework the way data columns sidecars are retrieved from peers.
Before this PR:
If in
bwb
we needed to retrieve columns[1, 3]
for block[12, 13]
and columns[8,9]
for block20
, then, we sent a data column sidecar by range request corresponding to columns[1, 3, 8, 9]
for blocks[12, 13, 14, 15, 16, 17, 18, 19, 20]
.So,
[8, 9]
for blocks[12, 13]
,[1, 3]
for blocks20
, and[1, 3, 8, 9]
for blocks[14, 15, 16, 17, 18, 19]
were returned by our peer for strictly no usage at all.
==> This occurs a useless bandwidth usage and a higher initial sync time (especially in small devnets) due to our internal rate limiting.
With this PR:
We carefully craft columns by range requests to ask peers to return exactly the columns we need.
(We take into account missed blocks and blocks without commitments, to minimise the number of generated requests.)
Remaining tasks regarding initial syncing: