op-stack v1.9.5 - Batcher Sequencer Throttling

❗ It is strongly recommended for all chain operators to upgrade to this release.

Highlights

Batcher Sequencer Throttling (ethereum-optimism/op-geth#421 #12735)

This release contains a new batcher-driven sequencer-throttling control loop. This is to avoid sudden spikes in L1 DA-usage consuming too much available gas and causing a backlog in batcher transactions. The batcher can throttle the sequencer’s data throughput instantly when it sees too much batcher data built up.

🎛️ There are two throttling knobs:

• transaction L1 data throttling, which skips individual transactions whose estimated compressed L1 DA usage goes over a certain threshold, and
• block L1 data throttling, which caps a block’s estimated total L1 DA usage and leads to not including transactions during block building that would move the block’s L1 DA usage past a certain threshold.

Upgrade path

This new feature is enabled by default and requires running op-geth version v1.101411.1 or later. It can be disabled by setting --throttle-interval to 0. The sequencer's op-geth node has to be updated first, before updating the batcher, so that the new required RPC is available at the time of the batcher restart.

op-conductor

It is required to upgrade to op-conductor/v0.2.0 if you are using conductor’s leader-aware rpc proxy feature. This conductor release includes support for the miner_setMaxDASize op-geth rpc proxy.

Configuration

Batcher

⚠️ Note that this feature requires the batcher to correctly follow the sequencer at all times, or it would set throttling parameters on a non-sequencer EL client. That means, active sequencer follow mode has to be enabled correctly by listing all the possible sequencers in the L2 rollup and EL endpoint flags.

The batcher can be configures with the following new flags and default parameters:

• interval at which throttling operations happen (besides when loading an L2 block in the batcher) via --throttle-interval: 2s (env var OP_BATCHER_THROTTLE_INTERVAL)
  • This can be set to zero to completely disable this feature. Since it’s set to 2s by default, the feature is enabled by default.
• backlog of pending block bytes beyond which the batcher will enable throttling on the sequencer via --throttle-threshold: 1_000_000 (batcher backlog of 1MB of data to batch)
  • env var OP_BATCHER_THROTTLE_THRESHOLD
• individual tx size throttling via --throttle-tx-size: 300 (estimated compressed bytes)
  • env var OP_BATCHER_THROTTLE_TX_SIZE
• block size throttling via --throttle-block-size: 21_000 (estimated total compressed bytes, at least 70 transactions per block of up to 300 compressed bytes each)
  • env var OP_BATCHER_THROTTLE_BLOCK_SIZE
• block size throttling that’s always active via --throttle-always-block-size: 130_000
  • env var OP_BATCHER_THROTTLE_ALWAYS_BLOCK_SIZE
  • This block size limit is enforced on the sequencer at all times, even if there isn’t any backlog in the batcher. Normal network usage shouldn’t be impacted by this. This is to prevent a too fast build up of data to batch.

If the batcher at startup has throttling enabled and the sequencer’s op-geth node to which it’s talking hasn’t the miner_setMaxDASize RPC enabled, it will fail with an error message like

lvl=warn msg="Served miner_setMaxDASize" reqid=1 duration=11.22µs err="the method miner_setMaxDASize does not exist/is not available"

In this case, make sure the miner API namespace is enabled for the correct transport protocol (HTTP or WS), see next paragraph.

Sequencer

The new RPC miner_setMaxDASize is available in op-geth since v1.101411.1. It has to be enabled by adding the miner namespace to the correct API flags, like

• GETH_HTTP_API: web3,debug,eth,txpool,net,miner
• GETH_WS_API: debug,eth,txpool,net,miner

It is recommended to add it to both, HTTP and WS.

Other

• A bug in the sequencer is fixed that may cause derivation to stall on flaky L1 connections (#12258).
• This release also contains many incremental changes regarding Holocene and Interop feature development.

What's Changed

• Interop: Update Inputs (rebased) by @axelKingsley in #12204
• [batcher] derive.ChannelOut factory by @mdehoog in #12344
• op-service: Add optional headers to the signer client by @trianglesphere in #12407
• [batcher] Cleanup batcher channel inclusion block logic by @mdehoog in #12363
• [batcher] Rename channel receiver by @mdehoog in #12453
• op-node/rollup/derive: Add Holocene Channel Assembler by @sebastianst in #12334
• op-node/rollup/sequencing: Fix temporary engine error handling by @sebastianst in #12258
• feat(op-batcher): wait for pre-genesis RPC request loop by @jsvisa in #12383
• op-node: Alphabetize CLI network lists by @mbaxter in #12522
• Holocene: initial op-node support for configurable eip-1559 params by @roberto-bayardo in #12497
• Holocene: SystemConfig contract changes for updating EIP-1559 params by @roberto-bayardo in #12498
• op-node/rollup/derive: Holocene Batch Stage by @sebastianst in #12417
• Holocene: e2e test for Holocene's configurable eip-1559 params by @roberto-bayardo in #12499
• rollup-client: Increase call timeouts in CI by @ajsutton in #12561
• op-service: Add omitempty to EIP1559Params by @mslipper in #12582
• chore(op-service): Add missing fields to blob API types by @clabby in #12602
• op-service/client: rm unnecessary type infer arguments by @jsvisa in #12651
• Interop: Fix DerivedFrom Client Signature by @axelKingsley in #12661
• op-node: disable finality based on local-safe when interop is active by @protolambda in #12690
• op-node/rollup/derive: Implement pipeline stage multiplexing by @sebastianst in #12506
• op-node/rollup: Implement Holocene invalid payload attributes handling by @sebastianst in #12621
• op-node/rollup: Promote all attributes to safe post-Holocene by @sebastianst in #12724
• Interop: Add error case for parent of start of database by @axelKingsley in #12818
• interop: Handle first L1 Parent Ref in CandidateCrossSafe by @axelKingsley in #12830
• interop: correct error in prior WithParent commit by @axelKingsley in #12832
• interop: Handle ErrParentToFirst from PreviousDerivedFrom by @axelKingsley in #12833
• op-batcher: Increase the pending_blocks metric when a channel times out by @geoknee in #12810
• op-batcher: control loop to throttle DA usage by @roberto-bayardo in #12735
• op-batcher: Enable throttling by default, shutdown on broken RPC by @sebastianst in #12840
• interop: Fix AtLeastAsSafe ; Geth Mempool Filter E2E Test by @axelKingsley in #12823
• [Batcher] Fix memory leak, reverse prepended blocks by @mdehoog in #12874
• op-conductor,op-node: allow system to select port, make op-node wait for conductor endpoint by @protolambda in #12863

New Contributors (full monorepo)

• @qizhou made their first contribution in #12310
• @gotzenx made their first contribution in #12514
• @meyer9 made their first contribution in #12576
• @klein818 made their first contribution in #11812
• @ashutoshvarma made their first contribution in #12679
• @ControlCplusControlV made their first contribution in #12727
• @sigma made their first contribution in #12856

Full Changelog: v1.9.4...v1.9.5

🚢 Docker Images:

https://us-docker.pkg.dev/oplabs-tools-artifacts/images/op-node:v1.9.5
https://us-docker.pkg.dev/oplabs-tools-artifacts/images/op-batcher:v1.9.5
https://us-docker.pkg.dev/oplabs-tools-artifacts/images/op-proposer:v1.9.5

OP Contracts v1.8.0-rc.1

Overview

This release is suitable for the L1 contracts as part of the Holocene network upgrade.

Full Contract Set

A chain using this contracts release must be using the following contracts at the specified semvers.

The following contracts are included in this release:

• MIPS: 1.2.1
• SystemConfig: 2.3.0

Additional Information

There is no finalized v1.7.0 release. The release candidate for v1.7.0 was for generating L2 genesis.

op-dispute-mon/v1.2.0

Overview

This is a minor release of op-dispute-mon that includes minor new features and bug fixes. It is an optional upgrade for all users.

What's Changed

• Use game data from previous update cycle if fetching fails. This ensures aggregate metrics remain accurate when there are intermittent errors from source nodes. (#12481)
• Add op_dispute_mon_oldest_game_update_time metric to report the timestamp of the least recently updated game.

Full Changelog: op-dispute-mon/v1.1.0...op-dispute-mon/v1.2.0

🚢 Docker Image:

• https://us-docker.pkg.dev/oplabs-tools-artifacts/images/op-dispute-mon/v1.2.0

op-challenger/v1.2.0

Overview

This is a minor release of op-challenger that includes minor new features and bug fixes. It is an optional upgrade for all users.

What's Changed

• Allow op-challenger to startup when an enabled trace type does not have a game implementation set on chain (#12539)
• Use binary snapshots with Asterisc (#12586)
• Introduce --prestates-url option to specify a URL to download prestates for all trace types from. Can be overridden for specific VM types with --cannon-prestates-url, --asterisc-prestates-url and --asterisc-kona-prestates-url (#12440)
• All prestates URL options now accept file URLs in addition to http and https (#12441)
• op-program log level is now automatically set to match op-challenger log level (#12379)

Full Changelog: op-challenger/v1.1.2...op-challenger/v1.2.0

🚢 Docker Image:

• https://us-docker.pkg.dev/oplabs-tools-artifacts/images/op-challenger:v1.2.0

op-deployer/v0.0.3

Changelog

• 96468b7 op-deployer: Add Docker builds (#12660)
• d73a4bc op-service, op-deployer: Support marshaling systemconfig in pre-Holocene format (#12619)

op-deployer/v0.0.2

Changelog

• e7fbaec Holocene: initial op-node support for configurable eip-1559 params (#12497)
• c9e3ad6 op-deployer: Add deploy config inspect command (#12570)
• bb5ab2e op-deployer: Add support for inspecting l2 semvers (#12577)
• 9483e3b op-deployer: Fix standard artifacts urls (#12575)
• d035096 op-deployer: Remove binary that got committed (#12555)
• 3b83710 op-deployer: Run validations over L2 genesis (#12560)
• c6c9c4b op-deployer: Support L1 alloc deployments (#12517)
• d8012e0 op-deployer: bugfix: Chain ownership roles (#12556)

op-deployer/v0.0.1-rc.0

Changelog

• f58f1f5 op-deployer: Marshal tag-based ArtifactsLocator with correct scheme (#12494)
• da6625f op-deployer: Most implementation addresses not set in state.json when standard release tag used (#12434)
• 1f106f7 op-deployer: Package for production (#12491)
• 5f7f44c op-deployer: Remove irrelevant flags (#12525)
• 05961cc op-deployer: error handling bugs (#12507)
• 5a37546 op-deployer: update default gas limit values. (#12524)
• ceafacf op-deployer: updating opcm proxies for mainnet and sepolia (#12527)
• 1bf64d9 op-e2e: Increase timeout on output submitter test (#12531)
• c05aca3 opcm: exposing preimage oracle and mips singletons via DeployOPChain.s.sol (#12521)

op-stack v1.9.4

Overview

⬆️ This release updates the op-geth dependency to v1.101411.0, which includes the upstream go-ethereum changes until v1.14.11.

Other changes are related to ongoing development of Interop and Holocene, and other minor fixes.

What's Changed

• op-batcher: use deterministic seed for test data by @geoknee in #12129
• op-batcher,op-e2e: replace magic numbers like 6 with consts, eg MaxBlobsPerBlobTx by @zhiqiangxu in #11842
• txmgr: call EstimateGas correctly for blob tx by @zhiqiangxu in #12086
• Use DA /put path from spec by @karlb in #12081
• op-node/rollup/derive: Implement Holocene Frame Queue by @sebastianst in #12069
• Holocene: extensions to L1Block.sol by @roberto-bayardo in #12096
• Revert "Holocene extensions to L1Block.sol (#12096)" by @roberto-bayardo in #12236
• feat(sequencer): Origin Selector asynchronously prefetches the next origin from events by @BrianBland in #12134
• Eth: BlockRef by @axelKingsley in #12251
• batcher: use abstract Queue type for blocks state by @geoknee in #12180
• Update op-geth depdency to 1.14.11 base by @0x00101010 in #12262
• Remove AddSingularBatch from ChannelOut interface (prefer AddBlock) by @mdehoog in #12079
• batcher: Increase metric properly when blocks are requeued by @geoknee in #12326
• txmgr: reinstate fee metrics by @geoknee in #12369

New Contributors

• @tsnob made their first contribution in #12164
• @raffaelemazzitelli made their first contribution in #12288
• @pauldowman made their first contribution in #12362

Full Changelog: v1.9.3...v1.9.4

🚢 Docker Images:

https://us-docker.pkg.dev/oplabs-tools-artifacts/images/op-node:v1.9.4
https://us-docker.pkg.dev/oplabs-tools-artifacts/images/op-batcher:v1.9.4
https://us-docker.pkg.dev/oplabs-tools-artifacts/images/op-proposer:v1.9.4

op-challenger v1.1.2

Overview

This is a minor release of op-challenger with minor new features. It is an optional upgrade for all users.

What's Changed

• Skip prestate verifications for the permissioned game. (#12140)

Full Changelog: op-challenger/v1.1.0...op-challenger/v1.1.2

🚢 Docker Image:

• https://us-docker.pkg.dev/oplabs-tools-artifacts/images/op-challenger:v1.1.2

op-stack v1.9.3

Overview

This release improves the automatic data-availability switching behavior of op-batcher. It will now assess the optimal configuration (i.e. to use calldata or blobs) based on market conditions immediately before it is due to send an L1 transaction (instead of when it starts to build a channel) (#12002 #12115). This release also includes minor bug fixes for and improvements to op-node and op-batcher.

Partial Changelog

• op-node: op-node interop block verification (devnet-1 scope) by @protolambda in #11611
• op-node: sanity-check the forkchoice updates by @protolambda in #11780
• Stop using global logger in data source by @joshklop in #11756
• Catch 'header not found' string in L2BlockRefByLabel by @emhane in #11759
• fix(op-batcher): initAltDA before initChannelConfig by @emilianobonassi in #11816
• op-node: read DACommitmentType from scr in LoadOPStackRollupConfig by @bitwiseguy in #11830
• op-node: find sync-start, discard candidate when ahead by @zhiqiangxu in #11874
• op-node: handle crit error events, lift event-system out of driver by @protolambda in #11932
• op-batcher: Add String methods to txRef by @sebastianst in #11904
• op-node: lazy-dial the supervisor RPC by @protolambda in #11970
• chore(op-service): use errors.New to replace fmt.Errorf with no parameters by @yukionfire in #11796
• show block number when "L2 safe head ahead of L2 unsafe head" happens by @zhiqiangxu in #12030
• Batcher: support the txmgr.TxManager interface rather than SimpleTxManager implementation by @mdehoog in #12040
• Batcher: simplify ChannelOut initialization by @mdehoog in #12045
• op-service: add configurable client timeout by @bitwiseguy in #12074
• op-service: remove test print statement by @bitwiseguy in #12083
• op-node: Remove unused field in ChannelBank by @palango in #12001
• op-batcher: Move decision about data availability type to channel submission time by @geoknee in #12002
• op-batcher: prevent over-assessment of DA type by @geoknee in #12115

Full Changelog: v1.9.2...v1.9.3

🚢 Docker Images:

• https://us-docker.pkg.dev/oplabs-tools-artifacts/images/op-node:v1.9.3
• https://us-docker.pkg.dev/oplabs-tools-artifacts/images/op-batcher:v1.9.3
• https://us-docker.pkg.dev/oplabs-tools-artifacts/images/op-proposer:v1.9.3