Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Metachain notarizez based on equivalent proofs #6513

Open
wants to merge 17 commits into
base: feat/equivalent-messages
Choose a base branch
from
Open

Metachain notarizez based on equivalent proofs #6513

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
1eae0e9
now metachain notarizez based on equivalent proofs
sstanculeanu Oct 2, 2024
4197d2b
Merge branch 'feat/equivalent-messages' into adapt_notarization
sstanculeanu Oct 3, 2024
71187d1
Merge branch 'feat/equivalent-messages' into adapt_notarization
sstanculeanu Oct 3, 2024
367d1d1
Merge branch 'feat/equivalent-messages' of https://github.com/multive…
sstanculeanu Oct 10, 2024
3df3761
Merge remote-tracking branch 'origin/adapt_notarization' into adapt_n…
sstanculeanu Oct 10, 2024
8236823
fixes after review
sstanculeanu Oct 15, 2024
d96462a
fixed more tests
sstanculeanu Oct 17, 2024
e970b06
Merge branch 'feat/equivalent-messages' of https://github.com/multive…
sstanculeanu Oct 17, 2024
2e1b145
fix after merge
sstanculeanu Oct 17, 2024
f34b5cf
Merge branch 'feat/equivalent-messages' into adapt_notarization
sstanculeanu Oct 17, 2024
6aad53b
fix tests
sstanculeanu Oct 17, 2024
50ba8a1
subround block fixes
sstanculeanu Oct 22, 2024
3fb045b
removed tests for after equivalent messages as they are not valid any…
sstanculeanu Oct 22, 2024
93afc4e
fix linter
sstanculeanu Oct 22, 2024
1158d33
remove one more test
sstanculeanu Oct 22, 2024
450d335
proper proofs verification on notarization
sstanculeanu Oct 23, 2024
644d8be
fixes after review
sstanculeanu Nov 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 20 additions & 3 deletions process/block/baseProcess.go
View file
Open in desktop
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adapt also sortHeadersForCurrentBlockByNonce to have hasProof check?

Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ type nonceAndHashInfo struct {

type hdrInfo struct {
usedInBlock bool
hasProof bool
hdr data.HeaderHandler
}

Expand Down Expand Up @@ -123,6 +124,8 @@ type baseProcessor struct {
mutNonceOfFirstCommittedBlock sync.RWMutex
nonceOfFirstCommittedBlock core.OptionalUint64
extraDelayRequestBlockInfo time.Duration

proofsPool dataRetriever.ProofsPool
}

type bootStorerDataArgs struct {
Expand Down Expand Up @@ -639,7 +642,7 @@ func (bp *baseProcessor) sortHeaderHashesForCurrentBlockByNonce(usedInBlock bool

bp.hdrsForCurrBlock.mutHdrsForBlock.RLock()
for metaBlockHash, headerInfo := range bp.hdrsForCurrBlock.hdrHashAndInfo {
if headerInfo.usedInBlock != usedInBlock {
if headerInfo.usedInBlock != usedInBlock || !headerInfo.hasProof {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that if a header is used in block but has no valid proof, then this is an error case
Could you add the condition separately and log an error for that?
in case we end up in this case we need to find out how it happened and fix.
When picking up a header to notarize it, and adding it to usedInBlock then it should have already been checked that there is a valid proof for it.

Copy link
Collaborator Author

@sstanculeanu sstanculeanu Oct 10, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

usedInBlock field is set to true also when shard headers are requested.. it is basically set in advance to true, even though the header is missing.. so maybe the proof would be received later.. not sure if this is possible though

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check for enabled flag here?

continue
}

Expand Down Expand Up @@ -978,8 +981,8 @@ func (bp *baseProcessor) cleanupPools(headerHandler data.HeaderHandler) {

err := bp.dataPool.Proofs().CleanupProofsBehindNonce(bp.shardCoordinator.SelfId(), highestPrevFinalBlockNonce)
if err != nil {
log.Warn("%w: failed to cleanup notarized proofs behind nonce %d on shardID %d",
err, noncesToPrevFinal, bp.shardCoordinator.SelfId())
log.Warn(fmt.Sprintf("%s: failed to cleanup notarized proofs behind nonce %d on shardID %d",
err.Error(), noncesToPrevFinal, bp.shardCoordinator.SelfId()))
}

if bp.shardCoordinator.SelfId() == core.MetachainShardId {
Expand Down Expand Up @@ -2167,3 +2170,17 @@ func (bp *baseProcessor) checkSentSignaturesAtCommitTime(header data.HeaderHandl

return nil
}

func (bp *baseProcessor) isFirstBlockAfterEquivalentMessagesFlag(header data.HeaderHandler) bool {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe rename this - from the name it is a bit unclear if it refers to the start of epoch block when EquivalentMessages activate or the block after that - FirstBlockAfter

maybe rename to: isEpochChangeBlockForEquivalentMessagesActivation

isStartOfEpochBlock := header.IsStartOfEpochBlock()
isBlockInActivationEpoch := header.GetEpoch() == bp.enableEpochsHandler.GetCurrentEpoch()

return isBlockInActivationEpoch && isStartOfEpochBlock
}

func (bp *baseProcessor) shouldConsiderProofsForNotarization(header data.HeaderHandler) bool {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should consider proofs for notarization I think from the block following the equivalents messages activation block.
The change of epoch block would have a small consensus group as it was proposed by a smaller set of validators.

Not to forget that the first block (which follows the new consensus) if it is considered as a confirmation block then it needs itself to have a valid proof.

isEquivalentMessagesFlagEnabledForHeader := bp.enableEpochsHandler.IsFlagEnabledInEpoch(common.EquivalentMessagesFlag, header.GetEpoch())
isFirstBlockAfterEquivalentMessagesFlag := bp.isFirstBlockAfterEquivalentMessagesFlag(header)

return isEquivalentMessagesFlagEnabledForHeader && !isFirstBlockAfterEquivalentMessagesFlag
}
6 changes: 3 additions & 3 deletions process/block/export_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ func NewShardProcessorEmptyWith3shards(
return shardProc, err
}

func (mp *metaProcessor) RequestBlockHeaders(header *block.MetaBlock) (uint32, uint32) {
func (mp *metaProcessor) RequestBlockHeaders(header *block.MetaBlock) (uint32, uint32, uint32) {
return mp.requestShardHeaders(header)
}

Expand Down Expand Up @@ -582,12 +582,12 @@ func (mp *metaProcessor) ChannelReceiveAllHeaders() chan bool {
}

// ComputeExistingAndRequestMissingShardHeaders -
func (mp *metaProcessor) ComputeExistingAndRequestMissingShardHeaders(metaBlock *block.MetaBlock) (uint32, uint32) {
func (mp *metaProcessor) ComputeExistingAndRequestMissingShardHeaders(metaBlock *block.MetaBlock) (uint32, uint32, uint32) {
return mp.computeExistingAndRequestMissingShardHeaders(metaBlock)
}

// ComputeExistingAndRequestMissingMetaHeaders -
func (sp *shardProcessor) ComputeExistingAndRequestMissingMetaHeaders(header data.ShardHeaderHandler) (uint32, uint32) {
func (sp *shardProcessor) ComputeExistingAndRequestMissingMetaHeaders(header data.ShardHeaderHandler) (uint32, uint32, uint32) {
return sp.computeExistingAndRequestMissingMetaHeaders(header)
}

Expand Down
1 change: 1 addition & 0 deletions process/block/hdrForBlock.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import (
type hdrForBlock struct {
missingHdrs uint32
missingFinalityAttestingHdrs uint32
missingHdrsProofs uint32
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i might help to rename here to indicate that it's num missing

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed it

highestHdrNonce map[uint32]uint64
mutHdrsForBlock sync.RWMutex
hdrHashAndInfo map[string]*hdrInfo
Expand Down
2 changes: 2 additions & 0 deletions process/block/headerValidator.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,8 @@ func (h *headerValidator) IsHeaderConstructionValid(currHeader, prevHeader data.
return process.ErrRandSeedDoesNotMatch
}

// TODO: check here if proof from currHeader is valid for prevHeader

return nil
}

Expand Down
113 changes: 98 additions & 15 deletions process/block/metablock.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,7 @@ func NewMetaProcessor(arguments ArgMetaProcessor) (*metaProcessor, error) {
managedPeersHolder: arguments.ManagedPeersHolder,
sentSignaturesTracker: arguments.SentSignaturesTracker,
extraDelayRequestBlockInfo: time.Duration(arguments.Config.EpochStartConfig.ExtraDelayForRequestBlockInfoInMilliseconds) * time.Millisecond,
proofsPool: arguments.DataComponents.Datapool().Proofs(),
}

mp := metaProcessor{
Expand Down Expand Up @@ -297,7 +298,7 @@ func (mp *metaProcessor) ProcessBlock(
}

mp.txCoordinator.RequestBlockTransactions(body)
requestedShardHdrs, requestedFinalityAttestingShardHdrs := mp.requestShardHeaders(header)
requestedShardHdrs, requestedFinalityAttestingShardHdrs, missingShardHdrProofs := mp.requestShardHeaders(header)

if haveTime() < 0 {
return process.ErrTimeIsOut
Expand All @@ -308,7 +309,9 @@ func (mp *metaProcessor) ProcessBlock(
return err
}

haveMissingShardHeaders := requestedShardHdrs > 0 || requestedFinalityAttestingShardHdrs > 0
haveMissingShardHeaders := requestedShardHdrs > 0 ||
requestedFinalityAttestingShardHdrs > 0 ||
missingShardHdrProofs > 0
if haveMissingShardHeaders {
if requestedShardHdrs > 0 {
log.Debug("requested missing shard headers",
Expand All @@ -320,8 +323,13 @@ func (mp *metaProcessor) ProcessBlock(
"num finality shard headers", requestedFinalityAttestingShardHdrs,
)
}
if missingShardHdrProofs > 0 {
log.Debug("missing shard header proofs",
"num", missingShardHdrProofs,
)
}

err = mp.waitForBlockHeaders(haveTime())
err = mp.waitForBlockHeadersAndProofs(haveTime())

mp.hdrsForCurrBlock.mutHdrsForBlock.RLock()
missingShardHdrs := mp.hdrsForCurrBlock.missingHdrs
Expand Down Expand Up @@ -1081,8 +1089,18 @@ func (mp *metaProcessor) createAndProcessCrossMiniBlocksDstMe(
continue
}

shouldCheckProof := mp.enableEpochsHandler.IsFlagEnabledInEpoch(common.EquivalentMessagesFlag, currShardHdr.GetEpoch())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the computeLongestChainFromLastNotarized would return I think the chain of headers without the last one (which is considered the confirmation for the last returned one).

Was computeLongestChainFromLastNotarized updated for equivalent proofs?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated the blockProcessor as well

hasProofForHdr := mp.proofsPool.HasProof(currShardHdr.GetShardID(), orderedHdrsHashes[i])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think here we should check if HasProof only if shouldCheckProof is true

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correct, optimization.. implemented it

if shouldCheckProof && !hasProofForHdr {
log.Trace("no proof for shard header",
"shard", currShardHdr.GetShardID(),
"hash", logger.DisplayByteSlice(orderedHdrsHashes[i]),
)
continue
}

if len(currShardHdr.GetMiniBlockHeadersWithDst(mp.shardCoordinator.SelfId())) == 0 {
mp.hdrsForCurrBlock.hdrHashAndInfo[string(orderedHdrsHashes[i])] = &hdrInfo{hdr: currShardHdr, usedInBlock: true}
mp.hdrsForCurrBlock.hdrHashAndInfo[string(orderedHdrsHashes[i])] = &hdrInfo{hdr: currShardHdr, usedInBlock: true, hasProof: true}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

split on multiple lines for better visibility? same below

hdrsAdded++
hdrsAddedForShard[currShardHdr.GetShardID()]++
lastShardHdr[currShardHdr.GetShardID()] = currShardHdr
Expand Down Expand Up @@ -1121,7 +1139,7 @@ func (mp *metaProcessor) createAndProcessCrossMiniBlocksDstMe(
miniBlocks = append(miniBlocks, currMBProcessed...)
txsAdded += currTxsAdded

mp.hdrsForCurrBlock.hdrHashAndInfo[string(orderedHdrsHashes[i])] = &hdrInfo{hdr: currShardHdr, usedInBlock: true}
mp.hdrsForCurrBlock.hdrHashAndInfo[string(orderedHdrsHashes[i])] = &hdrInfo{hdr: currShardHdr, usedInBlock: true, hasProof: true}
hdrsAdded++
hdrsAddedForShard[currShardHdr.GetShardID()]++

Expand Down Expand Up @@ -1284,6 +1302,17 @@ func (mp *metaProcessor) CommitBlock(
mp.lastRestartNonce = header.GetNonce()
}

isBlockAfterEquivalentMessagesFlag := !check.IfNil(lastMetaBlock) &&
mp.enableEpochsHandler.IsFlagEnabledInEpoch(common.EquivalentMessagesFlag, lastMetaBlock.GetEpoch())
if isBlockAfterEquivalentMessagesFlag {
// for the first block we need to update both the state of the previous one and for current
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and for all future blocks, updateState should be done only for current, not lastMetaBlock.

I would use a different variable e.g finalMetaBlock and not overwrite the lastMetaBlock (that should still point to the last committed meta block not the one currently being committed).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should work as described, updated the variable name for clarity

if mp.isFirstBlockAfterEquivalentMessagesFlag(header) {
mp.updateState(lastMetaBlock, lastMetaBlockHash)
}

lastMetaBlock = header
lastMetaBlockHash = headerHash
}
mp.updateState(lastMetaBlock, lastMetaBlockHash)

committedRootHash, err := mp.accountsDB[state.UserAccountsState].RootHash()
Expand Down Expand Up @@ -1838,6 +1867,20 @@ func (mp *metaProcessor) checkShardHeadersFinality(
continue
}

if mp.enableEpochsHandler.IsFlagEnabledInEpoch(common.EquivalentMessagesFlag, lastVerifiedHdr.GetEpoch()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this loop is correct, as it does not get to iterate over all shard headers and verify.
it returns either a nil or an error in the first iteration.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

true, moved this if into the next for loop and switched the return nil into a break

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does it fits better to check proof in checkShardHeadersValidity which is called just before instead of here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should stay here, as here is the place where we check for finality

marshalledHeader, err := mp.marshalizer.Marshal(lastVerifiedHdr)
if err != nil {
return err
}

headerHash := mp.hasher.Compute(string(marshalledHeader))
if !mp.proofsPool.HasProof(shardId, headerHash) {
return process.ErrHeaderNotFinal
}

return nil
}

// verify if there are "K" block after current to make this one final
nextBlocksVerified := uint32(0)
for _, shardHdr := range finalityAttestingShardHdrs[shardId] {
Expand Down Expand Up @@ -1886,7 +1929,9 @@ func (mp *metaProcessor) receivedShardHeader(headerHandler data.HeaderHandler, s

mp.hdrsForCurrBlock.mutHdrsForBlock.Lock()

haveMissingShardHeaders := mp.hdrsForCurrBlock.missingHdrs > 0 || mp.hdrsForCurrBlock.missingFinalityAttestingHdrs > 0
haveMissingShardHeaders := mp.hdrsForCurrBlock.missingHdrs > 0 ||
mp.hdrsForCurrBlock.missingFinalityAttestingHdrs > 0 ||
mp.hdrsForCurrBlock.missingHdrsProofs > 0
if haveMissingShardHeaders {
hdrInfoForHash := mp.hdrsForCurrBlock.hdrHashAndInfo[string(shardHeaderHash)]
headerInfoIsNotNil := hdrInfoForHash != nil
Expand All @@ -1900,18 +1945,36 @@ func (mp *metaProcessor) receivedShardHeader(headerHandler data.HeaderHandler, s
}
}

if hdrInfoForHash != nil && !hdrInfoForHash.hasProof {
hasProof := mp.proofsPool.HasProof(shardHeader.GetShardID(), shardHeaderHash)
hdrInfoForHash.hasProof = hasProof
if hasProof {
mp.hdrsForCurrBlock.missingHdrsProofs--
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missingHdrsProofs should be decreased if that proof was missing previously and still missing at this step only

e.g you receive a shard hdr which is not part of the metablock (an older block maybe), so here you would end up decreasing the missing headers proofs.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated the condition, it was not working as expected

}
}

if mp.hdrsForCurrBlock.missingHdrs == 0 {
mp.hdrsForCurrBlock.missingFinalityAttestingHdrs = mp.requestMissingFinalityAttestingShardHeaders()
if !mp.shouldConsiderProofsForNotarization(shardHeader) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on the else, you should check that there is a valid proof for this header, otherwise add it to the missing proofs.

mp.hdrsForCurrBlock.missingFinalityAttestingHdrs = mp.requestMissingFinalityAttestingShardHeaders()
}

if mp.hdrsForCurrBlock.missingFinalityAttestingHdrs == 0 {
log.Debug("received all missing finality attesting shard headers")
}

if mp.hdrsForCurrBlock.missingHdrsProofs == 0 {
log.Debug("received all shard header proofs")
}
}

missingShardHdrs := mp.hdrsForCurrBlock.missingHdrs
missingFinalityAttestingShardHdrs := mp.hdrsForCurrBlock.missingFinalityAttestingHdrs
missingHdrsProofs := mp.hdrsForCurrBlock.missingHdrsProofs
mp.hdrsForCurrBlock.mutHdrsForBlock.Unlock()

allMissingShardHeadersReceived := missingShardHdrs == 0 && missingFinalityAttestingShardHdrs == 0
allMissingShardHeadersReceived := missingShardHdrs == 0 &&
missingFinalityAttestingShardHdrs == 0 &&
missingHdrsProofs == 0
if allMissingShardHeadersReceived {
mp.chRcvAllHdrs <- true
}
Expand Down Expand Up @@ -1941,20 +2004,21 @@ func (mp *metaProcessor) requestMissingFinalityAttestingShardHeaders() uint32 {
return missingFinalityAttestingShardHeaders
}

func (mp *metaProcessor) requestShardHeaders(metaBlock *block.MetaBlock) (uint32, uint32) {
func (mp *metaProcessor) requestShardHeaders(metaBlock *block.MetaBlock) (uint32, uint32, uint32) {
_ = core.EmptyChannel(mp.chRcvAllHdrs)

if len(metaBlock.ShardInfo) == 0 {
return 0, 0
return 0, 0, 0
}

return mp.computeExistingAndRequestMissingShardHeaders(metaBlock)
}

func (mp *metaProcessor) computeExistingAndRequestMissingShardHeaders(metaBlock *block.MetaBlock) (uint32, uint32) {
func (mp *metaProcessor) computeExistingAndRequestMissingShardHeaders(metaBlock *block.MetaBlock) (uint32, uint32, uint32) {
mp.hdrsForCurrBlock.mutHdrsForBlock.Lock()
defer mp.hdrsForCurrBlock.mutHdrsForBlock.Unlock()

notarizedShardHdrsBasedOnProofs := 0
for _, shardData := range metaBlock.ShardInfo {
if shardData.Nonce == mp.genesisNonce {
lastCrossNotarizedHeaderForShard, hash, err := mp.blockTracker.GetLastCrossNotarizedHeader(shardData.ShardID)
Expand Down Expand Up @@ -1982,6 +2046,7 @@ func (mp *metaProcessor) computeExistingAndRequestMissingShardHeaders(metaBlock
hdr: nil,
usedInBlock: true,
}

go mp.requestHandler.RequestShardHeader(shardData.ShardID, shardData.HeaderHash)
continue
}
Expand All @@ -1994,13 +2059,24 @@ func (mp *metaProcessor) computeExistingAndRequestMissingShardHeaders(metaBlock
if hdr.GetNonce() > mp.hdrsForCurrBlock.highestHdrNonce[shardData.ShardID] {
mp.hdrsForCurrBlock.highestHdrNonce[shardData.ShardID] = hdr.GetNonce()
}

if mp.shouldConsiderProofsForNotarization(hdr) {
notarizedShardHdrsBasedOnProofs++
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of incrementing the missing headers proofs, maybe use a map for the missing header proofs, with key the header hash

then you could correctly check which proofs are still missing.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated


hasProofForShardHdr := mp.proofsPool.HasProof(shardData.ShardID, shardData.HeaderHash)
mp.hdrsForCurrBlock.hdrHashAndInfo[string(shardData.HeaderHash)].hasProof = hasProofForShardHdr
if !hasProofForShardHdr {
mp.hdrsForCurrBlock.missingHdrsProofs++
}
}
}

if mp.hdrsForCurrBlock.missingHdrs == 0 {
shouldRequestMissingFinalityAttestingShardHeaders := notarizedShardHdrsBasedOnProofs != len(metaBlock.ShardInfo)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check this only if equivalent flag enabled?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not needed, here notarizedShardHdrsBasedOnProofs would be 0, thus the second condition will always be true

if mp.hdrsForCurrBlock.missingHdrs == 0 && shouldRequestMissingFinalityAttestingShardHeaders {
mp.hdrsForCurrBlock.missingFinalityAttestingHdrs = mp.requestMissingFinalityAttestingShardHeaders()
}

return mp.hdrsForCurrBlock.missingHdrs, mp.hdrsForCurrBlock.missingFinalityAttestingHdrs
return mp.hdrsForCurrBlock.missingHdrs, mp.hdrsForCurrBlock.missingFinalityAttestingHdrs, mp.hdrsForCurrBlock.missingHdrsProofs
}

func (mp *metaProcessor) createShardInfo() ([]data.ShardDataHandler, error) {
Expand All @@ -2017,6 +2093,13 @@ func (mp *metaProcessor) createShardInfo() ([]data.ShardDataHandler, error) {
continue
}

isBlockAfterEquivalentMessagesFlag := check.IfNil(headerInfo.hdr) &&
mp.enableEpochsHandler.IsFlagEnabledInEpoch(common.EquivalentMessagesFlag, headerInfo.hdr.GetEpoch())
hasMissingProof := isBlockAfterEquivalentMessagesFlag && !headerInfo.hasProof
if hasMissingProof {
continue
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is an error if a header was used in block but has no proof, should not continue but return an error.
If we get such errors, then we need to fix in the place where the headers are notarized, as there should be the check that the notarized headers have the proper proof.

Please check that the shardInfo (createShardInfo) contains the proof for the shard header itself, along with the proof for the previous block.
This means that the shardInfo would change as well to include 2 proofs.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated to return error

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated also to use both proofs

}

shardHdr, ok := headerInfo.hdr.(data.ShardHeaderHandler)
if !ok {
return nil, process.ErrWrongTypeAssertion
Expand Down Expand Up @@ -2274,7 +2357,7 @@ func (mp *metaProcessor) prepareBlockHeaderInternalMapForValidatorProcessor() {
}

mp.hdrsForCurrBlock.mutHdrsForBlock.Lock()
mp.hdrsForCurrBlock.hdrHashAndInfo[string(currentBlockHeaderHash)] = &hdrInfo{false, currentBlockHeader}
mp.hdrsForCurrBlock.hdrHashAndInfo[string(currentBlockHeaderHash)] = &hdrInfo{false, false, currentBlockHeader}
mp.hdrsForCurrBlock.mutHdrsForBlock.Unlock()
}

Expand All @@ -2301,7 +2384,7 @@ func (mp *metaProcessor) verifyValidatorStatisticsRootHash(header *block.MetaBlo
return nil
}

func (mp *metaProcessor) waitForBlockHeaders(waitTime time.Duration) error {
func (mp *metaProcessor) waitForBlockHeadersAndProofs(waitTime time.Duration) error {
select {
case <-mp.chRcvAllHdrs:
return nil
Expand Down
Loading
Loading