PeerDAS: Batch columns verifications #14559
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nalepae
force-pushed
the
peerDAS-initial-sync-batch
branch
2 times, most recently
from
c3eb8d1 to
1efb15c
Compare
nalepae
force-pushed
the
peerDAS-initial-sync-batch
branch
2 times, most recently
from
f65e1a6 to
aebc3a0
Compare
nalepae
requested review from
potuz,
terencechain and
james-prysm
and removed request for
a team
nisdas
reviewed
Oct 22, 2024
| @@ -69,26 +68,32 @@ var ( | |||
| ErrColumnIndexInvalid = errors.New("incorrect column sidecar index") | |||
| ) | |||
|
|
|||
| type RODataColumnVerifier struct { | |||
| type RODataColumnsVerifier struct { | |||
There was a problem hiding this comment.
Why is this being changed ? We already have a batch verifier for the cases where we need to modify multiple columns at once
| @@ -538,9 +538,11 @@ func verifyColumn( | |||
| return false | |||
| } | |||
|
|
|||
| vf := columnVerifier(roDataColumn, verification.SamplingColumnSidecarRequirements) | |||
| // TODO: Once peer sampling is used, we should verify all sampled data columns in a single batch. | |||
| verifier := dataColumnsVerifier([]blocks.RODataColumn{roDataColumn}, verification.SamplingColumnSidecarRequirements) | |||
There was a problem hiding this comment.
This is a nit, but in go its always more preferable for shorter variable names (vf vs verifier). This was done in a few places in this PR
nalepae
force-pushed
the
peerDAS-initial-sync-batch
branch
2 times, most recently
from
416bddb to
e6038a3
Compare
Only `DataColumnsVerifier` (with `s`) on columns remains. It is the responsability of the function which receive the data column (either by gossip, by range request or by root request) to verify the data column wrt. corresponding checks.
nalepae
force-pushed
the
peerDAS-initial-sync-batch
branch
from
e6038a3 to
9490e32
Compare
nisdas
reviewed
Oct 23, 2024
| if err != nil { | ||
| return pubsub.ValidationReject, err | ||
| } | ||
|
|
||
| msg.ValidatorData = verifiedRODataColumn | ||
| if len(verifiedRODataColumns) != 1 { |
There was a problem hiding this comment.
This should be an error logged since its an impossible condition
nalepae
added a commit
that referenced
this pull request
Oct 23, 2024
* `ColumnAlignsWithBlock`: Split lines. * Data columns verifications: Batch * Remove completely `DataColumnBatchVerifier`. Only `DataColumnsVerifier` (with `s`) on columns remains. It is the responsability of the function which receive the data column (either by gossip, by range request or by root request) to verify the data column wrt. corresponding checks. * Fix Nishant's comment.
nalepae
added a commit
that referenced
this pull request
Oct 23, 2024
* `ColumnAlignsWithBlock`: Split lines. * Data columns verifications: Batch * Remove completely `DataColumnBatchVerifier`. Only `DataColumnsVerifier` (with `s`) on columns remains. It is the responsability of the function which receive the data column (either by gossip, by range request or by root request) to verify the data column wrt. corresponding checks. * Fix Nishant's comment.
nalepae
added a commit
that referenced
this pull request
Oct 23, 2024
* `ColumnAlignsWithBlock`: Split lines. * Data columns verifications: Batch * Remove completely `DataColumnBatchVerifier`. Only `DataColumnsVerifier` (with `s`) on columns remains. It is the responsability of the function which receive the data column (either by gossip, by range request or by root request) to verify the data column wrt. corresponding checks. * Fix Nishant's comment.
nalepae
added a commit
that referenced
this pull request
Oct 24, 2024
* `ColumnAlignsWithBlock`: Split lines. * Data columns verifications: Batch * Remove completely `DataColumnBatchVerifier`. Only `DataColumnsVerifier` (with `s`) on columns remains. It is the responsability of the function which receive the data column (either by gossip, by range request or by root request) to verify the data column wrt. corresponding checks. * Fix Nishant's comment.
nalepae
added a commit
that referenced
this pull request
Nov 20, 2024
* `ColumnAlignsWithBlock`: Split lines. * Data columns verifications: Batch * Remove completely `DataColumnBatchVerifier`. Only `DataColumnsVerifier` (with `s`) on columns remains. It is the responsability of the function which receive the data column (either by gossip, by range request or by root request) to verify the data column wrt. corresponding checks. * Fix Nishant's comment.
nalepae
added a commit
that referenced
this pull request
Nov 22, 2024
* `ColumnAlignsWithBlock`: Split lines. * Data columns verifications: Batch * Remove completely `DataColumnBatchVerifier`. Only `DataColumnsVerifier` (with `s`) on columns remains. It is the responsability of the function which receive the data column (either by gossip, by range request or by root request) to verify the data column wrt. corresponding checks. * Fix Nishant's comment.
nalepae
added a commit
that referenced
this pull request
Nov 22, 2024
* `ColumnAlignsWithBlock`: Split lines. * Data columns verifications: Batch * Remove completely `DataColumnBatchVerifier`. Only `DataColumnsVerifier` (with `s`) on columns remains. It is the responsability of the function which receive the data column (either by gossip, by range request or by root request) to verify the data column wrt. corresponding checks. * Fix Nishant's comment.
nalepae
added a commit
that referenced
this pull request
Nov 25, 2024
* `ColumnAlignsWithBlock`: Split lines. * Data columns verifications: Batch * Remove completely `DataColumnBatchVerifier`. Only `DataColumnsVerifier` (with `s`) on columns remains. It is the responsability of the function which receive the data column (either by gossip, by range request or by root request) to verify the data column wrt. corresponding checks. * Fix Nishant's comment.
nalepae
added a commit
that referenced
this pull request
Nov 25, 2024
* `ColumnAlignsWithBlock`: Split lines. * Data columns verifications: Batch * Remove completely `DataColumnBatchVerifier`. Only `DataColumnsVerifier` (with `s`) on columns remains. It is the responsability of the function which receive the data column (either by gossip, by range request or by root request) to verify the data column wrt. corresponding checks. * Fix Nishant's comment.
nalepae
added a commit
that referenced
this pull request
Nov 27, 2024
* `ColumnAlignsWithBlock`: Split lines. * Data columns verifications: Batch * Remove completely `DataColumnBatchVerifier`. Only `DataColumnsVerifier` (with `s`) on columns remains. It is the responsability of the function which receive the data column (either by gossip, by range request or by root request) to verify the data column wrt. corresponding checks. * Fix Nishant's comment.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR transforms
NewColumnVerifier(1 column) intoNewDataColumnsVerifier(multiple columns,Datahas been added for consistency).The purpose of this PR is to batch data columns verification, and especially
VerifyDataColumnsSidecarKZGProofs, which is much more efficient when called by batch than when called columns by columns.On my computer (ARM M-2), duration of KZG verification assuming a constant blob count per block, for 32 blocks:
~12-13 s~700 ms==>
x18(!) improvement.peerdas-devnet-3peerdas-devnet-3(in progress)