Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core: fix chain indexer reorg bug #19748

Merged
merged 2 commits into from
Jul 2, 2019
Merged

core: fix chain indexer reorg bug #19748

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
48c9d28
core: fix chain indexer reorg bug
zsfelfoldi Jun 20, 2019
7f782b1
core: prevent reverting valid section when reorg happens
rjl493456442 Jun 27, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 18 additions & 3 deletions core/chain_indexer.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -243,7 +243,7 @@ func (c *ChainIndexer) newHead(head uint64, reorg bool) {
// If a reorg happened, invalidate all sections until that point
if reorg {
// Revert the known section number to the reorg point
known := head / c.sectionSize
known := (head + 1) / c.sectionSize
stored := known
if known < c.checkpointSections {
known = 0
Expand Down Expand Up @@ -323,6 +323,7 @@ func (c *ChainIndexer) updateLoop() {
updated = time.Now()
}
// Cache the current section count and head to allow unlocking the mutex
c.verifyLastHead()
section := c.storedSections
var oldHead common.Hash
if section > 0 {
Expand All @@ -342,8 +343,8 @@ func (c *ChainIndexer) updateLoop() {
}
c.lock.Lock()

// If processing succeeded and no reorgs occcurred, mark the section completed
if err == nil && oldHead == c.SectionHead(section-1) {
// If processing succeeded and no reorgs occurred, mark the section completed
if err == nil && (section == 0 || oldHead == c.SectionHead(section-1)) {
c.setSectionHead(section, newHead)
c.setValidSections(section + 1)
if c.storedSections == c.knownSections && updating {
Expand All @@ -358,6 +359,7 @@ func (c *ChainIndexer) updateLoop() {
} else {
// If processing failed, don't retry until further notification
c.log.Debug("Chain index processing failed", "section", section, "err", err)
c.verifyLastHead()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this necessary to put the statement here? Since we will verify the validity of stored sections before processing anyway.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, it does not matter really but still I would prefer not leaving the indexer in a bad state after detecting an error.

c.knownSections = c.storedSections
}
}
Expand Down Expand Up @@ -411,13 +413,26 @@ func (c *ChainIndexer) processSection(section uint64, lastHead common.Hash) (com
return lastHead, nil
}

// verifyLastHead compares last stored section head with the corresponding block hash in the
// actual canonical chain and rolls back reorged sections if necessary to ensure that stored
// sections are all valid
func (c *ChainIndexer) verifyLastHead() {
for c.storedSections > 0 {
if c.SectionHead(c.storedSections-1) == rawdb.ReadCanonicalHash(c.chainDb, c.storedSections*c.sectionSize-1) {
return
}
c.setValidSections(c.storedSections - 1)
}
}

// Sections returns the number of processed sections maintained by the indexer
// and also the information about the last header indexed for potential canonical
// verifications.
func (c *ChainIndexer) Sections() (uint64, uint64, common.Hash) {
c.lock.Lock()
defer c.lock.Unlock()

c.verifyLastHead()
return c.storedSections, c.storedSections*c.sectionSize - 1, c.SectionHead(c.storedSections - 1)
}

Expand Down