Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

If a PoS block creation repetition takes less than a configurable dur… #5048

Merged
merged 7 commits into from
Feb 14, 2023

Conversation

fab-10
Copy link
Contributor

@fab-10 fab-10 commented Feb 3, 2023

…ation, then waits before next repetition

Signed-off-by: Fabio Di Fabio fabio.difabio@consensys.net

PR description

For PoS networks, block creation is repeated until the CL asks for the payload or a timeout occurs, there is no wait between repetitions and if the single block creation job is very fast, it means that there could be hundreds of similar repetitions, since only new addition to the tx pool could create a different block.
This PR introduce a (configurable) minimal duration for the block creation repetition, and make the block production wait to match it, in case the single block creation job is less than that duration.

Fixed Issue(s)

Documentation

  • I thought about documentation and added the doc-change-required label to this PR if
    updates are required.

Changelog

…ation, then waits before next repetition

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
@fab-10 fab-10 marked this pull request as ready for review February 3, 2023 13:25
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Copy link
Contributor

@macfarla macfarla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me. Have you tested it out on a running node @fab-10 ?

fab-10 and others added 2 commits February 9, 2023 13:18
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
…iningOptions.java

Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
@fab-10 fab-10 force-pushed the block-creation-min-duration branch 2 times, most recently from 95ab6a6 to 7913748 Compare February 9, 2023 12:22
@fab-10
Copy link
Contributor Author

fab-10 commented Feb 9, 2023

Makes sense to me. Have you tested it out on a running node @fab-10 ?

Tested on local devnets, still need to test it on some validators on official testnets, will merge only after these tests

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
@fab-10 fab-10 force-pushed the block-creation-min-duration branch from 7913748 to 29f4887 Compare February 9, 2023 13:44
@fab-10
Copy link
Contributor Author

fab-10 commented Feb 14, 2023

Successfully tested on Sepolia and Goerli

@fab-10 fab-10 merged commit 97edc67 into hyperledger:main Feb 14, 2023
@fab-10 fab-10 deleted the block-creation-min-duration branch February 14, 2023 11:26
jframe pushed a commit to jframe/besu that referenced this pull request Feb 17, 2023
hyperledger#5048)

* If a PoS block creation repetition takes less than a configurable duration, then waits before next repetition

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update CHANGELOG

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Add unit test

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update besu/src/main/java/org/hyperledger/besu/cli/BesuCommand.java

Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update besu/src/main/java/org/hyperledger/besu/cli/options/unstable/MiningOptions.java

Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

---------

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
jframe pushed a commit to jframe/besu that referenced this pull request Feb 17, 2023
hyperledger#5048)

* If a PoS block creation repetition takes less than a configurable duration, then waits before next repetition

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update CHANGELOG

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Add unit test

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update besu/src/main/java/org/hyperledger/besu/cli/BesuCommand.java

Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update besu/src/main/java/org/hyperledger/besu/cli/options/unstable/MiningOptions.java

Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

---------

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
jframe added a commit that referenced this pull request Feb 17, 2023
* Prepare for version 23.1.1-SNAPSHOT (#5067)

Signed-off-by: Simon Dudley <simon.dudley@consensys.net>

* Add getPayloadBodiesByRangeV1 and getPayloadBodiesByHash engine methods (#4980)

* Add engine get payload body methods and test

Signed-off-by: Zhenyang Shi <wcgcyx@gmail.com>
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Add header

Signed-off-by: Zhenyang Shi <wcgcyx@gmail.com>

* Update result struct & add test

Signed-off-by: Zhenyang Shi <wcgcyx@gmail.com>

* Change constant to use upper case

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Add PayloadBody class and withdrawals to response of methods

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Add unit tests

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Add changelog

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* spotless

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Add check to prevent returning trailing null results past the head

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Add test to check trailing null post head scenario

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Split tests into pre and post shanghai

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* spotless

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Rename methods

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Use getName() to log method name

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* spotless

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Rename variable

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Call constructor directly

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Fix ByHash json parsing
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>

* Fix json parsing again
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>

* Add check to prevent unnecessary queries

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Refactor method

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Add new error code for EngineGetPayloadBodies methods

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Add return error for request above the API limit

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Add constructor for empty response

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* add check for number of blocks requested and for requests of post head

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Add test to check error code when request exceeds API limits

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* add constant for max blocks allowed per request

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* spotless

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Fix some nits

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Add invalid params check

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Add tests for invalid params check

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Fix test and spotless

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Revert "Fix json parsing again"

This reverts commit 558d325.

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Revert "Fix ByHash json parsing Signed-off-by: Simon Dudley <simon.dudley@consensys.net>"

This reverts commit 663e11e

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Use UnsignedLongParameter to cast params of the request

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Add optional withdrawals to the NewPayload log (#5021)

Signed-off-by: Simon Dudley <simon.dudley@consensys.net>

* kubernetes and errorprone - update versions (#5013)

* update errorprone and kubernetes versions
* fixed errorprone issues in prod cod
* fixed errorprone issues in test code

---------

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

* Rename JsonRpcService to EngineJsonRpcService (#5036)

* rename JsonRpcService to EngineJsonRpcService

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Params should be single item of array type, not outer array of strings (#5037)

Signed-off-by: Simon Dudley <simon.dudley@consensys.net>

* Add EIP-2537 (BLS12-381 curve precompiles) to Cancun (#5017)

Add the BLS curve precompiles into the registry for cancun.  All of the
curve precompiles have been here since berlin, so this is just wiring
them in.

Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>

* Use UnsignedLongParameter to cast params of the request

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Add optional withdrawals to the NewPayload log (#5021)

Signed-off-by: Simon Dudley <simon.dudley@consensys.net>

* kubernetes and errorprone - update versions (#5013)

* update errorprone and kubernetes versions
* fixed errorprone issues in prod cod
* fixed errorprone issues in test code

---------

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

* Rename JsonRpcService to EngineJsonRpcService (#5036)

* rename JsonRpcService to EngineJsonRpcService

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Params should be single item of array type, not outer array of strings (#5037)

Signed-off-by: Simon Dudley <simon.dudley@consensys.net>

* Add EIP-2537 (BLS12-381 curve precompiles) to Cancun (#5017)

Add the BLS curve precompiles into the registry for cancun.  All of the
curve precompiles have been here since berlin, so this is just wiring
them in.

Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>

* Convert start and count from hex to match JSON-RPC Spec standard

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

---------

Signed-off-by: Zhenyang Shi <wcgcyx@gmail.com>
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Co-authored-by: Zhenyang Shi <wcgcyx@gmail.com>
Co-authored-by: Simon Dudley <simon.dudley@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Co-authored-by: Daniel Lehrner <daniel.lehrner@consensys.net>
Co-authored-by: Danno Ferrin <danno.ferrin@swirldslabs.com>

* Revert "Keep Worldstate Storage open for Bonsai archive latest layer (#5039)" (#5073)

This reverts commit e715010.

Signed-off-by: Simon Dudley <simon.dudley@consensys.net>

* Support post merge forks at genesis for hive tests (#5019)

Signed-off-by: Jason Frame <jason.frame@consensys.net>

* Fix manifest docker not skipping interim builds for RCs (#5068)

Signed-off-by: Jason Frame <jason.frame@consensys.net>

* Fix PoS checkpoint validation (#5081)

* change validation to lessThan

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Change exception message to PoS instead of Near head

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Add unit test and fix message of existing unit test

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

---------

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* moves check for init code length before balance check (#5077)

Signed-off-by: Justin Florentine <justin+github@florentine.us>

* bump revision for 23.1.x release branch

Signed-off-by: garyschulte <garyschulte@gmail.com>

* Burn in build of 23.1.0 (#5093)

* rebase off of burn-in release, remove unreleased 23.1.0-RC2 from CHANGELOG

Signed-off-by: garyschulte <garyschulte@gmail.com>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>

* re-default global max rpc batch size to 1k (#5104) (#5105)

* default global max rpc batch size to 1000 for now

Signed-off-by: garyschulte <garyschulte@gmail.com>

* Fix Layered World State issue (#5076)

* Use the copy during prepareTrieLog instead of saveTrieLog
* add final flag on BonsaiWorldStateUpdater
* Use a copy of BonsaiInMemoryWorldState inside prepareTrieLog
* add link to persisted worldstate storage
* fix tests
* Make a copy of the worldstate after committing changes to the trielog
* spotless + remove maybeUnSubscribe in setNextWorldView
* subscribe storage on layered worldstate
* fix null issue
* not close layered worldstate during getAccount
* clean code
* Add changelog entry

---------

Signed-off-by: ahamlat <ameziane.hamlat@consensys.net>
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
Co-authored-by: Karim TAAM <karim.t2am@gmail.com>

* Add shanghaiTime to sepolia (#5088)

Update and fix forkId tests

Move timestamp forks from getForkBlockNumbers to getForkBlockTimestamps in JsonGenesisConfigOptions - this ultimately gets used to popoulate the ForkIdManager which handles lists of blocks and timestamps the same way so this hasn't changed any actual behaviour, but rather supports the test fixes.

Implement TransitionProtocolSchedule.streamMilestoneBlocks as a concatenation of blockNumbers++blockTimestamps. This may have been a latent bug since it's used to update the node record when a fork transition occurs.

Signed-off-by: Simon Dudley <simon.dudley@consensys.net>

* If a PoS block creation repetition takes less than a configurable dur… (#5048)

* If a PoS block creation repetition takes less than a configurable duration, then waits before next repetition

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update CHANGELOG

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Add unit test

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update besu/src/main/java/org/hyperledger/besu/cli/BesuCommand.java

Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update besu/src/main/java/org/hyperledger/besu/cli/options/unstable/MiningOptions.java

Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

---------

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>

* Allow dashes in ethstats password (#5090)

Signed-off-by: Simon Dudley <simon.dudley@consensys.net>

* reintroduce checking of block height for certain tasks when we are not PoS (Revert PR#3911) (#5083)

* reintroduce checking of block height for certain tasks when we are not PoS (Revert PR#3911)

Signed-off-by: Stefan Pingel <16143240+pinges@users.noreply.github.com>

* Allow other users to read the /opt/besu dir when using docker (#5092)

Signed-off-by: Rafael Matias <rafael@skyle.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>

* Only use MAINNET version of KZG (#5095)

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* add more context to exception messages and debug logging (#5066)

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

* Fix block value calculation (#5100)

* Add gasUsed calculation

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Fix unit test

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Fix typo

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

---------

Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

* Add 23.1.1-RC1 changelog

Signed-off-by: Jason Frame <jason.frame@consensys.net>

* Change version to 23.1.1-RC1

Signed-off-by: Jason Frame <jason.frame@consensys.net>

---------

Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Zhenyang Shi <wcgcyx@gmail.com>
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Signed-off-by: Jason Frame <jason.frame@consensys.net>
Signed-off-by: Justin Florentine <justin+github@florentine.us>
Signed-off-by: garyschulte <garyschulte@gmail.com>
Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
Signed-off-by: ahamlat <ameziane.hamlat@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Stefan Pingel <16143240+pinges@users.noreply.github.com>
Signed-off-by: Rafael Matias <rafael@skyle.net>
Co-authored-by: Simon Dudley <simon.dudley@consensys.net>
Co-authored-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
Co-authored-by: Zhenyang Shi <wcgcyx@gmail.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Co-authored-by: Daniel Lehrner <daniel.lehrner@consensys.net>
Co-authored-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Co-authored-by: Justin Florentine <justin+github@florentine.us>
Co-authored-by: garyschulte <garyschulte@gmail.com>
Co-authored-by: ahamlat <ameziane.hamlat@consensys.net>
Co-authored-by: Karim TAAM <karim.t2am@gmail.com>
Co-authored-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Co-authored-by: Stefan Pingel <16143240+pinges@users.noreply.github.com>
Co-authored-by: Rafael Matias <rafael@skyle.net>
ensi321 pushed a commit to ensi321/besu that referenced this pull request Feb 19, 2023
hyperledger#5048)

* If a PoS block creation repetition takes less than a configurable duration, then waits before next repetition

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update CHANGELOG

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Add unit test

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update besu/src/main/java/org/hyperledger/besu/cli/BesuCommand.java

Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update besu/src/main/java/org/hyperledger/besu/cli/options/unstable/MiningOptions.java

Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

---------

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
elenduuche pushed a commit to elenduuche/besu that referenced this pull request Aug 16, 2023
hyperledger#5048)

* If a PoS block creation repetition takes less than a configurable duration, then waits before next repetition

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update CHANGELOG

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Add unit test

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update besu/src/main/java/org/hyperledger/besu/cli/BesuCommand.java

Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update besu/src/main/java/org/hyperledger/besu/cli/options/unstable/MiningOptions.java

Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

---------

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
eum602 pushed a commit to lacchain/besu that referenced this pull request Nov 3, 2023
hyperledger#5048)

* If a PoS block creation repetition takes less than a configurable duration, then waits before next repetition

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update CHANGELOG

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Add unit test

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update besu/src/main/java/org/hyperledger/besu/cli/BesuCommand.java

Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update besu/src/main/java/org/hyperledger/besu/cli/options/unstable/MiningOptions.java

Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

---------

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants