Signed-off-by: Karim TAAM <karim.t2am@gmail.com>

… than its parent (#4066)

* After merge add a rule to check that the current block is more recent than its parent

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update CHANGELOG

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Unit test

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix to return pre-merge protocol schedule when block difficulty is less that TTD

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* adds interfaces for tracking merge state and forchoices
* after 2 finalizations from fcu, disconnect any peers sending new blocks, or connecting with td > ttd
* tests for preventing pow peers from joining
* refactored to separate out merge logic

Signed-off-by: Justin Florentine <justin+github@florentine.us>

* if block did not descend from the terminal block add it to the bad blocks

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* adds test coverage for bug
* corrects root issue and covers the bug

Signed-off-by: Justin Florentine <justin+github@florentine.us>

…4026)" (#4093)

This reverts commit 27fc468.

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Remove backward sync exception recursive nesting

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>

#4098)

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

…at should not be the case (#4102)

* ignore 2 tests that assume that the system language is English, if that should not be the case

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

…tion post-merge sync. (#4104)

Signed-off-by: Justin Florentine <justin+github@florentine.us>

Co-authored-by: garyschulte <garyschulte@gmail.com>

* Add experimental config option to enable v5 discovery

Signed-off-by: Gabriel Trintinalia <gabriel.trintinalia@consensys.net>

Co-authored-by: Gabriel Trintinalia <gabriel.trintinalia@consensys.net>
Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>

Move Taccat Isid from Active to Emeritus Maintainer.

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>

Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* Remove hash to append from the queue only if the step succeeds

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Create backward sync retries on demand

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

Signed-off-by: Karim TAAM <karim.t2am@gmail.com>

* add a merge-specific definiton of bestPeer and the supporting plumbing

Signed-off-by: garyschulte <garyschulte@gmail.com>

* set reached TTD when finishing a fast sync if appropriate

Signed-off-by: garyschulte <garyschulte@gmail.com>

* spdx header

Signed-off-by: garyschulte <garyschulte@gmail.com>

* fix BetterSyncTargetEvaluatorTest tests

Signed-off-by: garyschulte <garyschulte@gmail.com>

…#4122)

* check status of block propagation manager before starting or stopping

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* candidate fix for missing parent worldstate
* add rollback if failure
* fix persist trie log issue
* add trielog manager
* fix inmemory issue for bonsai

Signed-off-by: Karim TAAM <karim.t2am@gmail.com>

Co-authored-by: garyschulte <garyschulte@gmail.com>
Co-authored-by: Justin Florentine <justin+github@florentine.us>

Signed-off-by: Diego López León <dieguitoll@gmail.com>

* Update RocksDB default block cache size.

Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>

* add malloc arena params to besu startup
* add jemalloc preload to besu startup
* add jemalloc lib download to docker jvm images
* use 5.* for jemalloc-dev version
* add jemalloc to circleci containers for acceptance tests

Signed-off-by: garyschulte <garyschulte@gmail.com>

* useful response improves reputation

Signed-off-by: Stefan <stefan.pingel@consensys.net>

* remove the limit for the incoming message size. The size is limited on the RLPx layer (16.7MB)

Signed-off-by: Stefan <stefan.pingel@consensys.net>

Signed-off-by: garyschulte <garyschulte@gmail.com>

update changelog for release

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>

Release 22.7.0-RC2 of Hyperledger Besu

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>

Prep snapshot of 22.7.0

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>

* Allow creating Trie Log Layer objects outside of its package

Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>

* Make access public so that I can use the class outside of besu

Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>

Sonar has been removed from the build workflow. This PR removes all
configuration and code markings that were used to facilitate it.

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>

* log connected/disconnected state of RemotelyInitiatedConnections
* separate class for logging process message

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* logging changes for peering

Signed-off-by: Stefan <stefan.pingel@consensys.net>

Signed-off-by: Stefan <stefan.pingel@consensys.net>

…#4155)

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* fix infinite loop if a reorg contain a bad block
* add cache for latest valid ancestors for bad blocks

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

…ds (#4168)

* change expiration for JWT authentification of engine port to 60 seconds

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

Signed-off-by: garyschulte <garyschulte@gmail.com>

* add sepolia paris / mergeNetSplit block
* fix test looking for old premerge fork alias, add sepolia terminal conditions test
* rename to mergeNetSplit, ditch the terminalBlockHash and terminalBlockNumber configs for sepolia

Signed-off-by: garyschulte <garyschulte@gmail.com>

* limit engine-api info logging
* use a common constant for engine api logging threshold

Signed-off-by: garyschulte <garyschulte@gmail.com>

Signed-off-by: Roman Vaseev <4833306+Filter94@users.noreply.github.com>

Co-authored-by: Justin Florentine <justin+github@florentine.us>

* reimplemented with simple loop instead of recursing

Signed-off-by: Justin Florentine <justin+github@florentine.us>

…on workflow dispatch on demand (#4165)

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* Add mechanism to retrieve missing blocks that caused BlockPropagationManager to get stuck on pending blocks
* Remove threshold and request the lowest block parent everytime a block is saved for future

Signed-off-by: Gabriel Trintinalia <gabriel.trintinalia@consensys.net>

Co-authored-by: Gabriel Trintinalia <gabriel.trintinalia@consensys.net>
Co-authored-by: garyschulte <garyschulte@gmail.com>
Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>

Signed-off-by: garyschulte <garyschulte@gmail.com>

Signed-off-by: garyschulte <garyschulte@gmail.com>

Signed-off-by: matt-nelson-csi <matt.nelson@consensys.net>

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* print warning for deprecated testnets

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

If a response to the get header P2P request only returns the header that
is the start of the range we may need to trim it to an empty response,
this is breaking the validation response. One client has started
returning only the range header start in some circumstances (which is a
valid response per spec). Because we sometimes request an overlapping
header this results in an empty stream once we cut the duplicates.

fixes #3336

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* add oldState to MergeStateHandler, move Pandas to Transition watcher

Signed-off-by: garyschulte <garyschulte@gmail.com>

* on block propagation stop, only kill the new block and new block hash handlers, leave all others (especially txs) alone

Signed-off-by: Justin Florentine <justin+github@florentine.us>

* initial impl of engine-api override:
* remove reliance on isMergeEnabled in pre-merge block header validation rules
* create a merge-specific block header validtion stack rather than leveraging mainnet
* re-add --engine-api-enabled cli param
* make engine api usable without a merge config.
leaves the door open to supporting the engine api with other MiningCoordinators, but disables all but engine_exchangeTransitionConfiguration in the absence of mergeCoordinator

Signed-off-by: garyschulte <garyschulte@gmail.com>

……changeTransitionConfiguration (#4205)

Signed-off-by: garyschulte <garyschulte@gmail.com>

* applies merge filtering to peers after 2 finalized
* fixes logical error in handling status messages

Signed-off-by: Justin Florentine <justin+github@florentine.us>

* DNS peers handled the same as boot nodes
* make sure that non bonded peers can be used as initial peers
* try to connect to DNS nodes

Signed-off-by: Stefan <stefan.pingel@consensys.net>

Co-authored-by: Justin Florentine <justin+github@florentine.us>

* fix enr request
* add tests
* removed errant debug
* adds pr feedback and test to ensure ENR responses never go out to incomplete ping/pong handshakes
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
Co-authored-by: Justin Florentine <justin+github@florentine.us>

Signed-off-by: garyschulte <garyschulte@gmail.com>

Release Besu 22.7.0

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>

Ser versions for next release - 22.7.1-SNAPSHOT

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>

…trics after sync. (#4166)

Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>

Co-authored-by: Fabio Di Fabio <fabio.difabio@consensys.net>

Signed-off-by: Diego López León <dieguitoll@gmail.com>

Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* Upgrade web3j dependencies to latest versions

Signed-off-by: Diego López León <dieguitoll@gmail.com>
Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
Co-authored-by: Antony Denyer <git@antonydenyer.co.uk>
Co-authored-by: Miguel Rojo <miguelangel.rojofernandez@mastercard.com>
Co-authored-by: Miguel Angel Rojo <freemanz1486@gmail.com>

…4206)

* Handle ConcurrentModificationException on ReattemptPendingPeerRequests
* Replace removeIf with Iterator

Signed-off-by: Gabriel Trintinalia <gabriel.trintinalia@consensys.net>

Co-authored-by: mark-terry <36909937+mark-terry@users.noreply.github.com>
Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>

Signed-off-by: Simon Dudley <simon.dudley@consensys.net>

* correct fallback ttd to correct value

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>

…#4086)

* removed constraint with the length of the privacy public key

* refactor to include tessera ec encryptor

* added EC snippet to the tessera json config, still need to replace base64string from web3j

* acceptance tests working after modifying the web3j library to allow secp256r1 keys

* using NACL encryptor by default

* using web3j v4.9.4 and web3j-quorum v4.9.0

Signed-off-by: Miguel Rojo <miguelangel.rojofernandez@mastercard.com>

Co-authored-by: Antony Denyer <git@antonydenyer.co.uk>

* introducing a lower bound on number of peers
* added lower bound CLI option

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
Signed-off-by: Roman Vaseev <4833306+Filter94@users.noreply.github.com>
Co-authored-by: Roman Vaseev <4833306+Filter94@users.noreply.github.com>
Co-authored-by: Justin Florentine <justin+github@florentine.us>

…o execute a reorg around an already finalized block. (#4240)

Signed-off-by: garyschulte <garyschulte@gmail.com>

* use attached/detached rule difference to prevent PoW rules from influencing fast sync around TTD

Signed-off-by: garyschulte <garyschulte@gmail.com>

* quieten DynamicPivotBlockManager.  Only change pivot block if it is different than existing.  move logging to debug

Signed-off-by: garyschulte <garyschulte@gmail.com>

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Add more log to retrieve parent method
* Request the lowest pending ancestor when saving a block
* Replace recursive implementation with iterative when getting pending ancestors of Block
* Decrease scope of synchronized block to reflect only the event of adding pending block to the list
* Add fork to the chain so test is more representative

Signed-off-by: Gabriel Trintinalia <gabriel.trintinalia@consensys.net>

Signed-off-by: Gabriel Trintinalia <gabriel.trintinalia@consensys.net>
Co-authored-by: Gabriel Trintinalia <gabriel.trintinalia@consensys.net>

Make the ethereum/trie and services/kvstore modules  as slim as possible
by dropping dependencies.

* The ':util' api dependency is removed from kvstore  and added to
   eth module
* Move tries's kvstore dependency to test
* Other dependencies are removed as unused from trie and kvstore

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* getProof encoding fix for 4249

Signed-off-by: garyschulte <garyschulte@gmail.com>

* start filtering peers after 1 finalized instead of 2
* stops counting finalized, and starts filtering on first finalized
* DefaultSynchronizer now listens to Forkhoice messages so it can stop block propagation at finalization, as opposed to TTD (previous behavior)

Signed-off-by: Justin Florentine <justin+github@florentine.us>

…aces (#4268)

Signed-off-by: Stefan <stefan.pingel@consensys.net>
Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>

Cleanup repository references and limit scope of non-mavenCentral
repositories.

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>

* if merge enabled, wrap two clique rules in composed Attached rule to enable fast-sync to proceed normally for post-merge networks
* move BlockPropagationManager warning to debug until #4274

Signed-off-by: garyschulte <garyschulte@gmail.com>

* breaks pandas up, test coverage

Signed-off-by: Justin Florentine <justin+github@florentine.us>

* implement tentative mainnet TTD
* fix unit test breakage
* add to change log

Signed-off-by: garyschulte <garyschulte@gmail.com>

* Refactor retrying peer task switching peers at every try

RetryingGetBlockFromPeersTask had a problem that prevented to complete
when all the peers were tried without success, and that also had the
consequence to not removing the failed requested block for the internal
caches in BlockPropagationManager, that could cause a stall since that
block will to be tried to be retrieved again.

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

Paris block definition was set up prior to gray glacier.  Update the
base to be gray glacier.

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>

The four instances of gas in the CALL operations are all implemented
identically.  Refactor the method into the abstract parent.

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>

* upreved to new snapshot
* download links / shas in changelog

Signed-off-by: Justin Florentine <justin+github@florentine.us>

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Co-authored-by: Antoine Toulme <antoine@lunar-ocean.com>