-
Notifications
You must be signed in to change notification settings - Fork 81
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Handle out of order
shares in the data square
#191
Comments
Is this not contradictory to this comment from @adlerjohn? Verification logic doesn't happen in rsmt2d.
|
@musalbas From what I understand, this issue is not about implementing the verification logic in the rsmt2d, rather it is about adding support for detecting situations where the root construction has failed due to unordered leaves and returning an error with enough information e.g., the index of unordered shares (column and row) to the caller, likely as part of the ErrByzantineData. Then the caller will take care of crafting a proper proof. |
I don't think that can be handled directly in rsmt2d, because rsmt2d does not assume that the square uses a namespaced Merkle tree. I also don't know if that's what's being requested, but we should clarify. cc @adlerjohn |
As far as I can say, it has access to the errors returned by the underlying tree construction function, based on which can populate the ErrByzantineData. Update: and those functions actually are able to detect unordered leaves. Below are two approaches that I see to this issue (considering that what I have described in this comment is what this issue is aimed for).
|
I don't think it can be done in As for (1), are you proposing to add a new error field to |
Am I correct that in the case of out-of-order shares in the particular row/col, we will not be able to get the subtree roots and build the proof against this particular row/col? If so, then I guess in the node we can add a statement, during BEFP construction, that checks the subtree roots and re-requests the orthogonal axis. |
The issue is that you might not be able to get the subtree root not only if the shares are out of order, but if the row that the BEFP for is simply erasure coded incorrectly, and thus does not match the subtree root in the block header. In that case, you need to use the inclusion proofs that the node already downloaded via sampling. |
But does not ErrByzantineData store only sampled shares and not the rebuilt ones? |
I believe ErrByzantineData returns the shares of the "last known good" square, which can include rebuilt rows and columns. This is because you may need to use shares from rebuilt rows and columns in the BEFP. Basically, I believe the rule for choosing what proof should be used for a BEFP share should be:
|
I have read the comments and will get back soon to this thread. |
I think it might be easier if we discuss our ideas using an example:
s1-s16 are the shares On the rsmt2d side, we want to return an error similar to the
Based on the answer to the above questions, we can then proceed with whether the current fields in @musalbas @vgonkivs @adlerjohn Please let me know your thoughts |
AFAIK, we do not need to modify any fields in The only thing needed, is to see that the row root of the recomputed row does not match the row root that was committed to in the block header, because when we recompute the row root with the nmt library, it should return an See this comment for an explanation of why:
Implementation wise, the only thing I'm unsure of is if |
Agreed, the error already contains all the necessary information from rsmt2d that node needs.
Good catch, and I think that's the missing piece. IMO rsmt2d should return an |
This is the correct intuition: either the shares in the |
Agree with Mustafa and John that we do not need to modify fields in |
Also, agreed with the point that
Also, agree with this. |
FYI: I'm moving forward with the implementation of the following solution in rsmt2d: if the tree construction fails for a row or column, during the repair process (this includes any inner function calls as well), an |
Related, potentially the same issue: #178 |
Thanks for mentioning the #178. |
…hares in tree construction (#242) ## Overview Closes #191 ## Checklist - [x] New and updated code has appropriate documentation - [x] New and updated code has new and/or updated testing - [x] Required CI checks are passing - [x] Visual proof for any user facing features like CLI or documentation updates - [x] Linked issues closed with keywords
Based on this we may run in a situation when shares will be out of order in the dataSquare. In this case, we will not be able to compute the Merkle root of row or col and reconstruction fails. To handle such error properly we need to get these shares+indexes of Merkel Roots of the opposite axis(to build a Merkle Proof against them). If Merkle Roots for both row/col can't be computed, then we should receive an error that data is not available.
The text was updated successfully, but these errors were encountered: