Skip to content

Commit

Permalink
tns: checkpoint wal periodically while backfilling
Browse files Browse the repository at this point in the history
Witout this, we end up with a massive .wal file that needs to be
checkpointed on first startup (which takes a few minutes) - it's much
more efficient to do smaller checkpoints, it turns out.
  • Loading branch information
arnetheduck committed Mar 18, 2022
1 parent 12dc427 commit 0f704f6
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 0 deletions.
1 change: 1 addition & 0 deletions beacon_chain/consensus_object_pools/blockchain_dag.nim
Original file line number Diff line number Diff line change
Expand Up @@ -1836,6 +1836,7 @@ proc rebuildIndex*(dag: ChainDAGRef) =

withState(state[]):
dag.db.putState(state)
dag.db.checkpoint()

state_root = state.root

Expand Down
5 changes: 5 additions & 0 deletions beacon_chain/trusted_node_sync.nim
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,8 @@ proc doTrustedNodeSync*(

let
db = BeaconChainDB.new(databaseDir, inMemory = false)
defer:
db.close()

var
dbCache = DbCache(summaries: db.loadSummaries())
Expand Down Expand Up @@ -391,6 +393,9 @@ proc doTrustedNodeSync*(
continue

gets[int(i mod gets.lenu64)] = downloadBlock(slot)

if i mod 1024 == 0:
db.checkpoint() # Transfer stuff from wal periodically
true
else:
notice "Database initialized, historical blocks will be backfilled when starting the node",
Expand Down

0 comments on commit 0f704f6

Please sign in to comment.