Releases: Consensys/teku
22.12.0
This is an optional update that primarily updates dependent libraries.
Downloads
- Available as
22.12.0
on Dockerhub - Download the binary distribution:
Breaking Changes
- Dropped support for custom network configurations that do not use the current preset format. Presets have been in use since before the MainNet beaconchain launch so this is not expected to affect any users.
Additions and Improvements
- Updated third party libraries
- Early access: Support for additional DoS protection by using a separate sentry beacon node to publish blocks and attestations.
- Tightened validation of the fork for gossip messages
22.11.0
This is a recommended update containing security updates. It is a required update for Gnosis chain nodes.
Downloads
- Available as
22.11.0
on Dockerhub - Download the binary distribution:
Breaking Changes
- The logic with which the default configuration is applied when using
validators-proposer-config
has been updated.
The logic is now more intuitive but a given proposer config file may result in a different configuration compared with the previous Teku version.
(Refer to #6325 (comment) for more information).
Additions and Improvements
- Updated protobuf used by libp2p library to resolve a potential DoS vector
- Added merge configuration for Gnosis chain. Note that all Gnosis nodes will need to be made merge-ready
- Improved discv5 compliance
- Changed the builder
is online\is offline
logs tois available\is not available
- Added
/eth/v1/beacon/states/{state_id}/randao
to beacon-api - Block timing tracking is now enabled by default. The
--Xmetrics-block-timing-tracking-enabled
option has been renamed to--metrics-block-timing-tracking-enabled
.
Bug Fixes
- Fixed missing status filters (active, pending, exited, withdrawal) for the
/eth/v1/beacon/states/{state_id}/validators
endpoint - Fixed issue which could lead to duplicate processing of some events after gossip is stopped and restarted.
- Fixed issue which could cause sync committee duties to be calculated too early, potentially causing duties to be missed if the scheduling was changed by a reorg.
22.10.2
Downloads
- Available as 22.10.2 on Dockerhub
- Download the binary distribution:
Breaking Changes
There are no breaking changes in this release.
Additions and Improvements
- Enabled new framework for REST API with better performance and less memory usage. The new framework does not cause any breaking changes, however if needed the old version can be restored with
--Xrest-api-migrated-enabled=false
- Updated jackson-databind library to version 2.13.4.2 addressing CVE-2022-42003
Bug Fixes
- Fix issue where /readiness endpoint returned 200 when Execution Client was not available.
Upcoming Breaking Changes
- The
validator_beacon_node_published_attestation_total
,validator_beacon_node_published_aggregate_total
,
validator_beacon_node_send_sync_committee_messages_total
,validator_beacon_node_send_sync_committee_contributions_total
andvalidator_beacon_node_published_block_total
metrics have been deprecated in favour of the newvalidator_beacon_node_requests_total
metric.
The old metrics will be removed in a future release. An update to the Teku Dashboard that uses the new metric is available. - The
/eth/v1/debug/beacon/states/:state_id
endpoint has been deprecated in favor of the v2 Altair endpoint/eth/v2/debug/beacon/states/:state_id
- The
/eth/v1/beacon/blocks/:block_id
endpoint has been deprecated in favor of the v2 Altair endpoint/eth/v2/beacon/blocks/:block_id
- The
/eth/v1/validator/blocks/:slot
endpoint has been deprecated in favor of the v2 Altair endpoint/eth/v2/validator/blocks/:slot
- The
/eth/v1/debug/beacon/heads
endpoint has been deprecated in favor of the v2 Bellatrix endpoint/eth/v2/debug/beacon/heads
- The commandline option
--network
of thevalidator-client
subcommand has been undeprecated and can be used to select a network for standalone validator clients. When set toauto
, it automatically
fetches network configuration information from the configured beacon node endpoint.
22.10.1
This is a recommended upgrade containing bug fixes, optimisations and voluntary-exit command improvements
Downloads
- Available as
22.10.1
on Dockerhub - Download the binary distribution:
Additions and Improvements
- Improve Execution Layer error logging
- Add new validator client metric
validator_remote_beacon_nodes_requests_total
which tracks the requests made to remote beacon nodes (useful when there are failovers configured) - The
voluntary-exit
subcommand can restrict the exit to a specific list of validators public keys using the option--validator-public-keys
.
Example:teku voluntary-exit --beacon-node-api-endpoint=<ENDPOINT>[,<ENDPOINT>...]... --data-validator-path=<PATH> --include-keymanager-keys=<BOOLEAN> --validator-keys=<KEY_DIR>:<PASS_DIR> | <KEY_FILE>:<PASS_FILE> --validator-public-keys=<PUBKEY>[,<PUBKEY>...]...
To include validator keys managed via keymanager APIs, the option--include-keymanager-keys
could be set totrue
(The default value is set tofalse
) - Throttle signing of validator registrations when using an external signer
Bug Fixes
- Filter out unknown validators when sending validator registrations to the builder network
- Fix issue which could cause locally produced aggregates to not be gossiped
- Fix issue where the sync module could cause
Unexpected rejected execution due to full task queue in nioEventLoopGroup
log messages and high CPU usage
Upcoming Breaking Changes
- The
validator_beacon_node_published_attestation_total
,validator_beacon_node_published_aggregate_total
,
validator_beacon_node_send_sync_committee_messages_total
,validator_beacon_node_send_sync_committee_contributions_total
andvalidator_beacon_node_published_block_total
metrics have been deprecated in favour of the newvalidator_beacon_node_requests_total
metric.
The old metrics will be removed in a future release. An update to the Teku Dashboard that uses the new metric is available. - The
/eth/v1/debug/beacon/states/:state_id
endpoint has been deprecated in favor of the v2 Altair endpoint/eth/v2/debug/beacon/states/:state_id
- The
/eth/v1/beacon/blocks/:block_id
endpoint has been deprecated in favor of the v2 Altair endpoint/eth/v2/beacon/blocks/:block_id
- The
/eth/v1/validator/blocks/:slot
endpoint has been deprecated in favor of the v2 Altair endpoint/eth/v2/validator/blocks/:slot
- The
/eth/v1/debug/beacon/heads
endpoint has been deprecated in favor of the v2 Bellatrix endpoint/eth/v2/debug/beacon/heads
- The commandline option
--network
of thevalidator-client
subcommand has been undeprecated and can be used to select a network for standalone validator clients. When set toauto
, it automatically
fetches network configuration information from the configured beacon node endpoint.
22.9.1
This is a recommended update containing improvements to post-merge performance issues on MainNet.
There are no changes from the previous 22.9.1-RC1 so users who are running the RC do not need to upgrade.
Downloads
- Available as
22.9.1
on Dockerhub - Download the binary distribution:
Breaking Changes
- The
--initial-state
and--eth1-deposit-contract-address
options has been removed from thevalidator-client
subcommand. They have been ignored for some time but are now completely removed. - LevelDB configuration has been changed to increase the maximum number of files it may open at a time. Systems with low open file handle limits may encounter issues.
We recommend at least 2048 file handles or preferably unlimited. Theulimit
command can be used to check and configure file handle limits.
Performance Improvements
This release significantly improves Teku's performance post-merge, but we will continue to investigate further improvements.
Please note that if the execution client is slow to respond, particularly when importing new blocks, that will increase the rate of incorrect head votes and increase the likelihood of attestations not being included.
To see more information on where time is being spent when importing blocks, users can enable late block alerts with the --Xmetrics-block-timing-tracking-enabled
option. This option will log a warning when a block import completes more than 4 seconds into the slot (after attestations will be created) with a break down of where time was spent. For example:
Late Block Import *** Block: e83a322a714b1b364ab59b2cb8666076fa015a8a5b4f1a8b9829306fc249f787 (4718961) proposer 425317 arrival 681ms, pre-state_retrieved +11ms, processed +306ms, execution_payload_result_received +3173ms, begin_importing +0ms, transaction_prepared +0ms, transaction_committed +1ms, completed +15ms
The labels used for timings are somewhat Teku developer focussed, but some key steps are:
arrival
- the time since the start of the slot that the block was first received. Large values here usually indicate that the block was produced late.state_retrieved
- the time it took to prepare the pre-state to apply the block to.processed
- the time it took Teku to process the block.execution_payload_result_received
- the time Teku spent waiting for the execution client to finish importing the block.
The remaining entries are Teku processing time to complete importing the block, updating fork choice records and storing the data to disk.
So in the example above the block was late to import because of a slow response from the execution client.
Additions and Improvements
- Enables asynchronous database updates by default. This ensures slow disk access or LevelDB compactions don't cause delays in the beacon node
- Increased leveldb open files to 1000 files by default. If teku fails to start, update maximum open files to 2048 or unlimited
- Reduced number of blocks held in in-memory cache to reduce need for old-gen GC with the larger post-merge blocks
- Make Validator Client connect to a failover event stream (if failovers are configured) when the current Beacon Node is not synced
- Detect Lodestar clients in
libp2p_connected_peers_current
metrics - Reduce CPU and Memory consumption in shuffling, which will improve epoch transition performance
- Faster peer discovery on startup
Bug Fixes
- Fixed an issue where epoch cache priming sometimes happened synchronously, causing delays in time tick processing in the beacon node
- Fix
/eth/v1/validator/register_validator
responding with a 400 status code and a misleading error message in case of exceptions - Update snakeyaml dependency to resolve cve-2022-25857 which could result in excessive memory usage when parsing YAML content
- Fixed an issue where the range requested for deposit logs was not reduced when using only
--ee-endpoint
leading to persistent timeouts with execution clients
Upcoming Breaking Changes
- The
validator_beacon_node_published_attestation_total
,validator_beacon_node_published_aggregate_total
,
validator_beacon_node_send_sync_committee_messages_total
,validator_beacon_node_send_sync_committee_contributions_total
andvalidator_beacon_node_published_block_total
metrics have been deprecated in favour of the newvalidator_beacon_node_requests_total
metric.
The old metrics will be removed in a future release. An update to the Teku Dashboard that uses the new metric is available. - The
/eth/v1/debug/beacon/states/:state_id
endpoint has been deprecated in favor of the v2 Altair endpoint/eth/v2/debug/beacon/states/:state_id
- The
/eth/v1/beacon/blocks/:block_id
endpoint has been deprecated in favor of the v2 Altair endpoint/eth/v2/beacon/blocks/:block_id
- The
/eth/v1/validator/blocks/:slot
endpoint has been deprecated in favor of the v2 Altair endpoint/eth/v2/validator/blocks/:slot
- The
/eth/v1/debug/beacon/heads
endpoint has been deprecated in favor of the v2 Bellatrix endpoint/eth/v2/debug/beacon/heads
22.9.1-RC1
This is a release candidate containing improvements to post-merge performance issues on MainNet that has received less testing than our normal releases.
MainNet users should only upgrade if they are confident in their ability to rollback if required. We encourage all testnet users to upgrade to assist with testing.
Downloads
- Available as
22.9.1-RC1
on Dockerhub - Download the binary distribution:
Breaking Changes
- The
--initial-state
and--eth1-deposit-contract-address
options has been removed from thevalidator-client
subcommand. They have been ignored for some time but are now completely removed. - LevelDB configuration has been changed to increase the maximum number of files it may open at a time. Systems with low open file handle limits may encounter issues.
We recommend at least 2048 file handles or preferably unlimited. Theulimit
command can be used to check and configure file handle limits.
Additions and Improvements
- Enables asynchronous database updates by default. This ensures slow disk access or LevelDB compactions don't cause delays in the beacon node. Can be disabled if needed via
--Xdata-storage-async-enabled=false
- Increased leveldb open files to 1000 files by default. If teku fails to start, update maximum open files to 2048 or unlimited
- Reduced number of blocks held in in-memory cache to reduce need for old-gen GC with the larger post-merge blocks
- Make Validator Client connect to a failover event stream (if failovers are configured) when the current Beacon Node is not synced
- Detect Lodestar clients in
libp2p_connected_peers_current
metrics - Reduce CPU and Memory consumption in shuffling, which will improve epoch transition performance
- Faster peer discovery on startup
Bug Fixes
- Fixed an issue where epoch cache priming sometimes happened synchronously, causing delays in time tick processing in the beacon node
- Fix
/eth/v1/validator/register_validator
responding with a 400 status code and a misleading error message in case of exceptions - Fix a
NullPointerException
for the gas limit when a proposer config is used and builder is enabled - Update snakeyaml dependency to resolve cve-2022-25857 which could result in excessive memory usage when parsing YAML content
- Fixed an issue where the range requested for deposit logs was not reduced when using only
--ee-endpoint
leading to persistent timeouts with execution clients
Upcoming Breaking Changes
- The
validator_beacon_node_published_attestation_total
,validator_beacon_node_published_aggregate_total
,
validator_beacon_node_send_sync_committee_messages_total
,validator_beacon_node_send_sync_committee_contributions_total
andvalidator_beacon_node_published_block_total
metrics have been deprecated in favour of the newvalidator_beacon_node_requests_total
metric.
The old metrics will be removed in a future release. An update to the Teku Dashboard that uses the new metric is available. - The
/eth/v1/debug/beacon/states/:state_id
endpoint has been deprecated in favor of the v2 Altair endpoint/eth/v2/debug/beacon/states/:state_id
- The
/eth/v1/beacon/blocks/:block_id
endpoint has been deprecated in favor of the v2 Altair endpoint/eth/v2/beacon/blocks/:block_id
- The
/eth/v1/validator/blocks/:slot
endpoint has been deprecated in favor of the v2 Altair endpoint/eth/v2/validator/blocks/:slot
- The
/eth/v1/debug/beacon/heads
endpoint has been deprecated in favor of the v2 Bellatrix endpoint/eth/v2/debug/beacon/heads
22.9.0
This is a required update. All users should upgrade as soon as possible.
IMPORTANT: Mainnet nodes must be made "merge ready"
All Mainnet nodes will need to prepare for the merge, including:
- Connecting to a local execution client's engine API with the
--ee-endpoint
option. - Validators will need to set a default fee recipient with
--validators-proposer-default-fee-recipient
.
Both Teku and your execution client will need to be updated to prepare for the merge. You should not receive any warning or error messages about the execution client being unavailable.
See the Teku guide on preparing for the merge for more information.
Downloads
- Available as
22.9.0
on Dockerhub - Download the binary distribution:
Bug Fixes
- Resolves an issue with public key validation.
Upcoming Breaking Changes
- The
validator_beacon_node_published_attestation_total
,validator_beacon_node_published_aggregate_total
,
validator_beacon_node_send_sync_committee_messages_total
,validator_beacon_node_send_sync_committee_contributions_total
andvalidator_beacon_node_published_block_total
metrics have been deprecated in favour of the newvalidator_beacon_node_requests_total
metric.
The old metrics will be removed in a future release. An update to the Teku Dashboard that uses the new metric is available. - The
/eth/v1/debug/beacon/states/:state_id
endpoint has been deprecated in favor of the v2 Altair endpoint/eth/v2/debug/beacon/states/:state_id
- The
/eth/v1/beacon/blocks/:block_id
endpoint has been deprecated in favor of the v2 Altair endpoint/eth/v2/beacon/blocks/:block_id
- The
/eth/v1/validator/blocks/:slot
endpoint has been deprecated in favor of the v2 Altair endpoint/eth/v2/validator/blocks/:slot
- The
/eth/v1/debug/beacon/heads
endpoint has been deprecated in favor of the v2 Bellatrix endpoint/eth/v2/debug/beacon/heads
22.8.2
This is a strongly recommended update that fixes a corner case which could lead to producing invalid blocks.
IMPORTANT: Mainnet nodes must be made "merge ready"
All Mainnet nodes will need to prepare for the merge, including:
- Connecting to a local execution client's engine API with the
--ee-endpoint
option. - Validators will need to set a default fee recipient with
--validators-proposer-default-fee-recipient
.
Both Teku and your execution client will need to be updated to prepare for the merge. You should not receive any warning or error messages about the execution client being unavailable.
See the Teku guide on preparing for the merge for more information.
Downloads
- Available as
22.8.2
on Dockerhub - Download the binary distribution:
Additions and Improvements
- Added Gas Limit APIs (GET/POST/DELETE)
- Skip finding the PoW block that first satisfies the minimum genesis time condition when the genesis state is already known. Fixes an incompatibility with Nethermind's backwards sync for historic blocks.
- Circuit breaker logic added when interacting with Builder endpoint
- VC using failover nodes for retrieving the network spec when
--network=auto
and multiple beacon nodes are configured - Teku now confirms the first successful connection to the execution client by logging
Execution Client is online
.
Bug Fixes
- Fixed a corner case that could lead to producing invalid blocks
- Fixed
io.libp2p.core.InternalErrorException: [peerHandler] not initialized yet
error from jvm-libp2p - Fixed VC incompatibility with Nimbus BN for the
/eth/v1/validator/beacon_committee_subscriptions
endpoint
Upcoming Breaking Changes
- The
validator_beacon_node_published_attestation_total
,validator_beacon_node_published_aggregate_total
,
validator_beacon_node_send_sync_committee_messages_total
,validator_beacon_node_send_sync_committee_contributions_total
andvalidator_beacon_node_published_block_total
metrics have been deprecated in favour of the newvalidator_beacon_node_requests_total
metric.
The old metrics will be removed in a future release. An update to the Teku Dashboard that uses the new metric is available. - The
/eth/v1/debug/beacon/states/:state_id
endpoint has been deprecated in favor of the v2 Altair endpoint/eth/v2/debug/beacon/states/:state_id
- The
/eth/v1/beacon/blocks/:block_id
endpoint has been deprecated in favor of the v2 Altair endpoint/eth/v2/beacon/blocks/:block_id
- The
/eth/v1/validator/blocks/:slot
endpoint has been deprecated in favor of the v2 Altair endpoint/eth/v2/validator/blocks/:slot
- The
/eth/v1/debug/beacon/heads
endpoint has been deprecated in favor of the v2 Bellatrix endpoint/eth/v2/debug/beacon/heads
22.8.1
This is a required update that include the Bellatrix network upgrade and merge transition configuration. All Mainnet users must upgrade before September 6.
IMPORTANT: Mainnet nodes must be made "merge ready"
All Mainnet nodes will need to prepare for the merge, including:
- Connecting to a local execution client's engine API with the
--ee-endpoint
option. - Validators will need to set a default fee recipient with
--validators-proposer-default-fee-recipient
.
Both Teku and your execution client will need to be updated to prepare for the merge. Until the execution client is updated Teku will regularly report an error like:
Merge *** Transition Configuration error: local TerminalTotalDifficulty and TerminalBlockHash not matching remote Execution Client values
local configuration: TransitionConfiguration{terminalTotalDifficulty=0x000000000000000000000000000000000000000000000c70d808a128d7380000, terminalBlockHash=0x0000000000000000000000000000000000000000000000000000000000000000, terminalBlockNumber=0}
remote configuration: TransitionConfiguration{terminalTotalDifficulty=0xfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffbff, terminalBlockHash=0x0000000000000000000000000000000000000000000000000000000000000000, terminalBlockNumber=0}
or like:
Execution Client is offline java.io.IOException: -32000: invalid ttd: execution <nil> consensus 0xc70d808a128d7380000 (See log file for full stack trace)
Once the execution client has been updated, you should not continue receiving these errors or other warnings about the execution client being unavailable.
Please continue to monitor for further Teku updates prior to the merge and be ready to upgrade if necessary.
See the Teku guide on preparing for the merge for more information.
Downloads
- Available as
22.8.1
on Dockerhub - Download the binary distribution:
Additions and Improvements
- Sets the Bellatrix fork epoch and TTD for The Merge on MainNet
- Automatically add
/eth/v2/debug/beacon/states/finalized
to the initial state file url (--initial-state
CLI option) when the provided url doesn't work and doesn't contain any path.
This simplifies using the standard REST API to retrieve the initial state as just the base URL can be specified (e.g.--initial-state https://<credentials@eth2-beacon-mainnet.infura.io
) - Ability to configure multiple beacon nodes for a single validator client using
--beacon-node-api-endpoints
CLI option - Primed cache for new justified checkpoints to reduce time required to run fork choice immediately after justification
- Remain in optimistic mode when there are no viable branches in the block tree because blocks from every branch were marked INVALID during optimistic sync (ethereum/consensus-specs#2955)
Bug Fixes
- Fixed
NullPointerException
when checking for the terminal PoW block while the EL was syncing - Changed log level of
Ignoring calculated payload
message to debug level. - Fixed repeated timeout exceptions when requesting deposit logs from the
--ee-endpoint
Upcoming Breaking Changes
- The
validator_beacon_node_published_attestation_total
,validator_beacon_node_published_aggregate_total
,
validator_beacon_node_send_sync_committee_messages_total
,validator_beacon_node_send_sync_committee_contributions_total
andvalidator_beacon_node_published_block_total
metrics have been deprecated in favour of the newvalidator_beacon_node_requests_total
metric.
The old metrics will be removed in a future release. An update to the Teku Dashboard that uses the new metric is available. - The
/eth/v1/debug/beacon/states/:state_id
endpoint has been deprecated in favor of the v2 Altair endpoint/eth/v2/debug/beacon/states/:state_id
- The
/eth/v1/beacon/blocks/:block_id
endpoint has been deprecated in favor of the v2 Altair endpoint/eth/v2/beacon/blocks/:block_id
- The
/eth/v1/validator/blocks/:slot
endpoint has been deprecated in favor of the v2 Altair endpoint/eth/v2/validator/blocks/:slot
- The
/eth/v1/debug/beacon/heads
endpoint has been deprecated in favor of the v2 Bellatrix endpoint/eth/v2/debug/beacon/heads
22.8.0
This is an optional update containing bug fixes, optimizations and minor new features.
Downloads
- Available as
22.8.0
on Dockerhub - Download the binary distribution:
IMPORTANT: Prater nodes must be made "merge ready"
See the Teku guide on preparing for the merge for more information.
Additions and Improvements
- Support for the external builder API (e.g. mev-boost)
- Enables fork choice before block proposals by default on MainNet (previously on by default on testnets only)
- Optimisations in jvm-libp2p to reduce CPU usage
- Updated Sepolia bootnodes
- Enabled progressive balance tracking optimisation on MainNet
- Replaced separate metrics for each beacon node request with one metric
beacon_node_requests_total
. Some old metrics were kept for backwards compatibility, but will be removed in future versions.
Bug Fixes
--ee-endpoint
option was not used to retrieve deposits for networks where Bellatrix was not yet scheduled- Fix
latestValidHash
with invalid Execution Payload in response from execution engine didn't trigger appropriate ForkChoice changes - Remove incorrect error about potentially finalizing an invalid execution payload when importing a block with an invalid payload
Upcoming Breaking Changes
- The
validator_beacon_node_published_attestation_total
,validator_beacon_node_published_aggregate_total
,
validator_beacon_node_send_sync_committee_messages_total
,validator_beacon_node_send_sync_committee_contributions_total
andvalidator_beacon_node_published_block_total
metrics have been deprecated in favour of the newvalidator_beacon_node_requests_total
metric.
The old metrics will be removed in a future release. An update to the Teku Dashboard that uses the new metric is available. - The
/eth/v1/debug/beacon/states/:state_id
endpoint has been deprecated in favor of the v2 Altair endpoint/eth/v2/debug/beacon/states/:state_id
- The
/eth/v1/beacon/blocks/:block_id
endpoint has been deprecated in favor of the v2 Altair endpoint/eth/v2/beacon/blocks/:block_id
- The
/eth/v1/validator/blocks/:slot
endpoint has been deprecated in favor of the v2 Altair endpoint/eth/v2/validator/blocks/:slot
- The
/eth/v1/debug/beacon/heads
endpoint has been deprecated in favor of the v2 Bellatrix endpoint/eth/v2/debug/beacon/heads