feat(state-keeper): Parallel l2 block sealing #1801

perekopskiy · 2024-04-25T19:42:10Z

What ❔

L2 block data (tx results, storage logs, events, l2 to l1 logs, factory deps, tokens) is sealed in parallel using concurrent DB connections. L2 block header is inserted after all l2 block data is inserted and serves as a marker that l2 block is sealed. DB queries are adjusted so L2 block is not accessed before block is sealed. Note, that some of the queries are only queried for resolved block range, i.e. caller never tries to access "pending" data. I haven't adjusted such queries on purpose to keep their performance the same.

Why ❔

Improves L2 block sealing performance

Checklist

PR title corresponds to the body of PR (we generate changelog entries from PRs).
Tests for the changes have been added / updated.
Documentation comments have been added / updated.
Code has been formatted via zk fmt and zk lint.
Spellcheck has been run via zk spellcheck.
Linkcheck has been run via zk linkcheck.

core/lib/dal/src/models/storage_sync.rs

core/lib/zksync_core/src/state_keeper/io/seal_logic/mod.rs

core/lib/dal/src/factory_deps_dal.rs

slowli

Looking at all additional filtering needed to beI wonder whether a potential alternative with primitive L2 block lifecycle would be easier. That is:

Insert an L2 block header before other data (meaning that foreign keys don't need to be removed)
...but mark the L2 block as incomplete until all other data is inserted. Having a new miniblocks field looks like an overkill / impractical; another way is to record the latest finished L2 block in a singleton table.

On the one hand, this would allow retaining FK constraints and make filtering out incomplete L2 blocks more explicit. OTOH, it'd require filtering out throughout all queries touching L2 blocks, which is easy to screw up.

core/lib/dal/src/blocks_web3_dal.rs

core/lib/dal/src/storage_logs_dal.rs

core/lib/dal/src/tokens_dal.rs

core/lib/dal/src/storage_web3_dal.rs

core/lib/zksync_core/src/state_keeper/io/seal_logic/l2_block_seal_subtasks.rs

core/lib/zksync_core/src/sync_layer/external_io.rs

RomanBrodetski · 2024-04-30T09:15:30Z

Didn't have a deep loop but wanted to respond to @slowli 's comment on

Insert an L2 block header before other data (meaning that foreign keys don't need to be removed)

IMO it might be even beneficial to remote foreign keys - this will make insertion even faster.

perekopskiy · 2024-04-30T09:57:48Z

My thoughts on alternative approach with inserting header first:
IMO foreign keys don't add much value. It's more important to keep data flow more convenient to use. As @slowli said, if we insert header first then we would need to add filters to (almost?) any query that reads from miniblocks. So more queries should be adjusted in addition to those touched in this PR

core/lib/dal/src/storage_web3_dal.rs

core/lib/zksync_core/src/api_server/tx_sender/proxy.rs

core/lib/zksync_core/src/state_keeper/io/common/mod.rs

core/lib/zksync_core/src/state_keeper/io/seal_logic/l2_block_seal_subtasks.rs

core/lib/zksync_core/src/state_keeper/io/seal_logic/mod.rs

## What ❔ Removes pre_insert_txs l2 block seal step. Instead `insert_txs` flag should be passed to `mark_txs_as_executed_in_l2_block`. ## Why ❔ Inserting txs with block data right away is faster then insert + updating block data. ## Checklist   - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `zk spellcheck`. - [x] Linkcheck has been run via `zk linkcheck`.

github-actions · 2024-05-15T07:54:06Z

No performance difference detected (anymore)

🤖 I have created a release *beep* *boop* --- ## [24.3.0](core-v24.2.0...core-v24.3.0) (2024-05-16) ### Features * Added support for making EN a (non-leader) consensus validator (BFT-426) ([#1905](#1905)) ([9973629](9973629)) * **configs:** Extract secrets to an additional config ([#1956](#1956)) ([bab4d65](bab4d65)) * **en:** Fetch L1 batch root hashes from main node ([#1923](#1923)) ([72a3571](72a3571)) * **eth-client:** Generalize RPC client ([#1898](#1898)) ([a4e099f](a4e099f)) * **Prover CLI:** `requeue` cmd ([#1719](#1719)) ([f722df7](f722df7)) * **Prover CLI:** `status batch --verbose` ([#1899](#1899)) ([cf80184](cf80184)) * **pruning:** Vacuum freeze started daily ([#1929](#1929)) ([5c85e9f](5c85e9f)) * Remove metrics crate ([#1902](#1902)) ([5f7bda7](5f7bda7)) * **state-keeper:** Parallel l2 block sealing ([#1801](#1801)) ([9b06dd8](9b06dd8)) * tee_verifier_input_producer ([#1860](#1860)) ([fea7f16](fea7f16)) * **vm-runner:** implement output handler for VM runner ([#1856](#1856)) ([1e4aeb5](1e4aeb5)) ### Bug Fixes * **basic_types:** bincode deserialization for `web3::Bytes` ([#1928](#1928)) ([406ec8c](406ec8c)) * **config:** Fix data-handler-config ([#1919](#1919)) ([b6bb041](b6bb041)) * **en:** Delete old txs by (init_addr, nonce) ([#1942](#1942)) ([fa5f4a7](fa5f4a7)) * **en:** Fix reorg detector logic for dealing with last L1 batch ([#1906](#1906)) ([3af5f5b](3af5f5b)) * parentHash in websocket blocks subscription is shown as 0x0 ([#1946](#1946)) ([fc2efad](fc2efad)) * **Prover CLI:** `status batch` bugs ([#1865](#1865)) ([09682f2](09682f2)) * **utils:** bincode ser-/deserialization for `BytesToHexSerde` ([#1947](#1947)) ([a75b917](a75b917)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). --------- Co-authored-by: zksync-era-bot <zksync-era-bot@users.noreply.github.com>

🤖 I have created a release *beep* *boop* --- ## [14.2.0](prover-v14.1.1...prover-v14.2.0) (2024-05-17) ### Features * Added support for making EN a (non-leader) consensus validator (BFT-426) ([#1905](#1905)) ([9973629](9973629)) * **configs:** Extract secrets to an additional config ([#1956](#1956)) ([bab4d65](bab4d65)) * **eth-client:** Generalize RPC client ([#1898](#1898)) ([a4e099f](a4e099f)) * **Prover CLI:** `delete` cmd ([#1802](#1802)) ([6e4a92e](6e4a92e)) * **Prover CLI:** `requeue` cmd ([#1719](#1719)) ([f722df7](f722df7)) * **Prover CLI:** `status batch --verbose` ([#1899](#1899)) ([cf80184](cf80184)) * **state-keeper:** Parallel l2 block sealing ([#1801](#1801)) ([9b06dd8](9b06dd8)) * tee_verifier_input_producer ([#1860](#1860)) ([fea7f16](fea7f16)) ### Bug Fixes * **Prover CLI:** `status batch` bugs ([#1865](#1865)) ([09682f2](09682f2)) * **prover:** Bump Cargo.lock and update VKs ([#1959](#1959)) ([367baad](367baad)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

perekopskiy added 5 commits April 25, 2024 22:35

Parallel miniblock sealing

186d04f

Resolve conflicts

a5aa211

Fix spelling

3eb87cb

Fix get_traces_for_l2_block

68d7f07

fmt

ac0ed80

perekopskiy commented Apr 26, 2024

View reviewed changes

core/lib/dal/src/models/storage_sync.rs Show resolved Hide resolved

core/lib/zksync_core/src/state_keeper/io/seal_logic/mod.rs Outdated Show resolved Hide resolved

lint

7612b1a

perekopskiy marked this pull request as ready for review April 26, 2024 07:49

perekopskiy requested review from RomanBrodetski, itegulov and slowli April 26, 2024 07:49

montekki reviewed Apr 26, 2024

View reviewed changes

core/lib/dal/src/factory_deps_dal.rs Outdated Show resolved Hide resolved

slowli reviewed Apr 29, 2024

View reviewed changes

perekopskiy added 6 commits April 30, 2024 17:06

Resolve conflict

652f9b2

Apply suggestion to DAL

cd8dff1

spawn -> run; nits

d2b98d0

fix contracts

d3f278a

Resolve conflict; wait for l1 batch in nonce sweeper

3f92d24

fmt

13d8f24

perekopskiy requested review from slowli and montekki May 1, 2024 12:28

perekopskiy and others added 3 commits May 3, 2024 14:22

clear pending data in persistance; improve sql queries

3fa70d6

Merge branch 'main' into parallel-miniblock-seal

3dc2897

Merge branch 'main' into parallel-miniblock-seal

49c5592

slowli reviewed May 7, 2024

View reviewed changes

perekopskiy added 3 commits May 9, 2024 15:32

Seal strategy; get rid of Arc

f0303d1

Resolve conflicts

435fb76

fmt

aed270a

github-merge-queue bot removed this pull request from the merge queue due to failed status checks May 14, 2024