[Epic]: ✂️ Sequencer-prover separation

[spalladino]

We want to split up sequencers and provers, as they are two different entities in the system. With the changes to block building, sequencers no longer need to upload a proof along their block to L1. Instead, provers group together multiple (32) blocks and submit a proof for several of them.

More context and discussion in this hackmd.

Implementation

Beta Give feedback

1. [prover] Update the orchestrator to handle epochs #8210
   +0
   
2. [prover] Update L1 contract to handle epoch proofs #8716
   +0
   
3. [prover] Update prover-node to schedule epoch proofs
4. [prover] Update e2e tests to prove epochs instead of blocks
5. [prover] Handle epochs with 0-1 blocks
6. [prover] Remove the flow for block proofs
7. [prover] Sort proving jobs by discrete priorities #7925
   S-needs-discussion +1
8. [prover] Split orchestrator and broker #8489
   +0
9. [prover] Handle connection between orchestrator and broker with multiple pending requests #8490
   +0
10. [prover] Persist and rehydrate state of the prover node and orchestrators #8491
    +0
11. [prover] Upload proving payloads in a storage shared between orchestrator and agents #8493
    +0

Tests

Beta Give feedback

1. Prover node can orchestrate proving of one epoch of blocks at 1 TPS (proving disabled first, enabled then)
2. Prover node can orchestrate concurrent proving of N=10 epochs of blocks at 1 TPS (proving disabled first, enabled then)
3. Prover node can recover from a crash in each of the four scenarios above without losing executed proving work

Attic

Beta Give feedback

1. Remove the proof and aggregation object from the L1 contract process method.
2. Have the L1 contract accept proofs in a separate method. Uploading a proof emits a ProofSubmitted event.
3. Remove the prover from the sequencer.
4. Have the L2BlockSource monitor ProofSubmitted events and track the last proven block number.
5. Tweak the ServerWorldStateSynchronizer so it can be configured to only process proven blocks, and to optionally fast-forward to a given block.
6. Tweak the TxPool so that, when it sees a tx included in a block, it persists it to a store so it can be later consumed from the prover node.
7. Create a new prover-node package that references the prover-client, world-state, and the public tx processor.
8. Implement the prove(l2blockrange) method on the prover-node, which should get the tx hashes to prove from an L2BlockSource, the individual tx objects from a tx db backed by a modified TxPool, process them using a PublicProcessor backed by a world-state pointed to the first block in the range, sends the jobs to the prover-client, and uploads them to L1.
9. Allow the prover-node to handle multiple simultaneous proving jobs by introducing a queue. The node should move onto the next job once the current one has finished tx processing.
10. Add a switch to the prover-node so it continuously proves unproven blocks in batches of N blocks.
11. Update the merge rollup circuits so they can accept txs from more than one block.
12. Update the merge and root rollup circuits so they output the block range proven.

[spalladino]

Definitely not completed!

[rahul-kothari]

woops unsure how a release PR closed it sorry!