Improve genesis state performance at startup #6977
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
8 tasks
|
Hey, I tested the current code; here are some records:
Before your refactor, the first start of Besu needed only 7 mins and 186 GB of memory allocated. The test record is in the jfr record file; you can load it directly using IntelliJ IDEA: https://github.com/lyfsn/besu-starter/blob/main/g-1.jfr |
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
…u into refactor-genesis-file-options
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
fab-10
force-pushed
the
refactor-genesis-file-options
branch
from
08ab687
to
0889ee4
Compare
# Conflicts: # acceptance-tests/dsl/src/main/java/org/hyperledger/besu/tests/acceptance/dsl/node/ThreadBesuNodeRunner.java # besu/src/main/java/org/hyperledger/besu/cli/BesuCommand.java # besu/src/main/java/org/hyperledger/besu/cli/config/EthNetworkConfig.java # besu/src/main/java/org/hyperledger/besu/controller/BesuController.java # besu/src/main/java/org/hyperledger/besu/controller/BesuControllerBuilder.java # besu/src/main/java/org/hyperledger/besu/controller/TransitionBesuControllerBuilder.java # besu/src/test/java/org/hyperledger/besu/PrivacyReorgTest.java # besu/src/test/java/org/hyperledger/besu/RunnerTest.java # besu/src/test/java/org/hyperledger/besu/cli/BesuCommandTest.java # besu/src/test/java/org/hyperledger/besu/cli/CascadingDefaultProviderTest.java # besu/src/test/java/org/hyperledger/besu/cli/config/EthNetworkConfigTest.java # besu/src/test/java/org/hyperledger/besu/controller/BesuControllerTest.java # config/src/main/java/org/hyperledger/besu/config/GenesisConfigFile.java # config/src/test/java/org/hyperledger/besu/config/JsonUtilTest.java # consensus/merge/src/test/java/org/hyperledger/besu/consensus/merge/blockcreation/MergeGenesisConfigHelper.java # consensus/qbft/src/integration-test/java/org/hyperledger/besu/consensus/qbft/support/TestContextBuilder.java # ethereum/core/src/test/java/org/hyperledger/besu/ethereum/trie/diffbased/bonsai/AbstractIsolationTests.java
fab-10
force-pushed
the
refactor-genesis-file-options
branch
from
9ca104c
to
b3a8978
Compare
# Conflicts: # config/src/main/java/org/hyperledger/besu/config/JsonUtil.java
fab-10
force-pushed
the
refactor-genesis-file-options
branch
from
829b437
to
07ab31c
Compare
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
fab-10
force-pushed
the
refactor-genesis-file-options
branch
from
07ab31c
to
a662895
Compare
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: ahamlat <ameziane.hamlat@consensys.net>
macfarla
approved these changes
Jun 11, 2024
ahamlat
reviewed
Jun 11, 2024
ahamlat
approved these changes
Jun 11, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR description
This PR takes inspiration from #6952 to improve Besu starts after the first one, specifically when the database is present, when
genesis-state-hash-cache-enabled=true, since it completely avoid to reprocess the genesis state, and in case of large genesis files, with may allocations, the startup time drops from many minutes to few seconds, and in any case the genesis allocations are not kept line in memory.The core of the change is to make the loading of the genesis allocations lazy, so they are only parsed from file when needed, and switching to a streaming parser so they are not all loaded and kept in memory.
Fixed Issue(s)
Thanks for sending a pull request! Have you done the following?
doc-change-requiredlabel to this PR if updates are required.Locally, you can run these tests to catch failures early:
./gradlew build./gradlew acceptanceTest./gradlew integrationTest./gradlew ethereum:referenceTests:referenceTests