Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Merge provernet to master #8373

Merged
merged 61 commits into from
Sep 4, 2024
Merged

chore: Merge provernet to master #8373

merged 61 commits into from
Sep 4, 2024

Commits on Aug 15, 2024

  1. fix: Log err stack traces

    spalladino committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    ec6b366 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    5166d05 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    aa31996 View commit details
    Browse the repository at this point in the history
  4. fix: Publish proof log

    spalladino committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    efcd9ed View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2024

  1. Configuration menu
    Copy the full SHA
    ac55ae3 View commit details
    Browse the repository at this point in the history
  2. feat: Max pending jobs in prover node (#8045)

    Adds a max number of pending block-proving jobs to the prover node. When
    reached, the prover node stops proving, and resumes once a job finishes.
    Also adds error handling to the creation of a new proving job, so if a
    block-proving-job fails to start, it is skipped instead of blocking the
    prover node.
    spalladino committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    ce35cad View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    1753e18 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    62ded62 View commit details
    Browse the repository at this point in the history
  5. TF fix

    PhilWindle committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    9fa2f89 View commit details
    Browse the repository at this point in the history
  6. Updated deployment config

    PhilWindle committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    84444cf View commit details
    Browse the repository at this point in the history
  7. Destroy PXE datastore

    PhilWindle committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    aeb84f4 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    2f0e148 View commit details
    Browse the repository at this point in the history
  9. Increased bot tx timeout

    PhilWindle committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    1902209 View commit details
    Browse the repository at this point in the history

Commits on Aug 19, 2024

  1. Configuration menu
    Copy the full SHA
    c721280 View commit details
    Browse the repository at this point in the history
  2. Bot fixes

    PhilWindle committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    bed7bf9 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    677ebd0 View commit details
    Browse the repository at this point in the history
  4. Re-instate efs destruction

    PhilWindle committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    37bd2a3 View commit details
    Browse the repository at this point in the history
  5. fix: Deploy L1 rollup contract using salt if provided

    Fixes a bad merge that disabled the usage of deployment salt for rollup
    contract.
    spalladino committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    d024eb5 View commit details
    Browse the repository at this point in the history
  6. Provernet bot doesn't wait

    PhilWindle committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    7af8a20 View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2024

  1. fix: Deploy L1 rollup contract using salt if provided (#8074)

    Fixes a bad merge that disabled the usage of deployment salt for rollup
    contract.
    spalladino committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    1c7d3da View commit details
    Browse the repository at this point in the history
  2. fix: Add DEBUG and LOG_LEVEL to valid env vars (#8094)

    Otherwise they don't get forwarded to the container if run with the
    aztec script.
    spalladino committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    9e11baf View commit details
    Browse the repository at this point in the history
  3. feat: track world state metrics (#8109)

    Extract relevant stats from world state:
    - sync duration (by synch type)
    - fork duration
    - db size (estimate)
    - individual tree sizes (ie. how many leaves are filled)
    alexghr authored and spalladino committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    636e4ef View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    4f25cde View commit details
    Browse the repository at this point in the history
  5. feat: JSON logging (#8095)

    Adds a LOG_JSON env var that disables the foundation logger and replaces
    it with a winston json logger.
    
    ```
    {"level":"info","message":"Set block interval to 12","module":"aztec:sandbox","timestamp":"2024-08-21T02:15:29.509Z"}
    {"level":"info","message":"Deploying contracts from 0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266...","module":"aztec:sandbox","timestamp":"2024-08-21T02:15:29.509Z"}
    {"level":"info","message":"Deployed Registry at 0x3aa5ebb10dc797cac828524e59a333d0a371443c","module":"aztec:sandbox","timestamp":"2024-08-21T02:15:29.534Z"}
    {"level":"info","message":"Deployed AvailabilityOracle at 0xc6e7df5e7b4f2a278906862b61205850344d4e7d","module":"aztec:sandbox","timestamp":"2024-08-21T02:15:29.551Z"}
    {"level":"info","message":"Deployed Fee Juice at 0x59b670e9fa9d0a427751af201d676719a970857b","module":"aztec:sandbox","timestamp":"2024-08-21T02:15:29.563Z"}
    {"level":"info","message":"Deployed Gas Portal at 0x4ed7c70f96b99c776995fb64377f0d4ab3b0e1c1","module":"aztec:sandbox","timestamp":"2024-08-21T02:15:29.573Z"}
    {"level":"info","message":"Funded fee juice portal contract with Fee Juice","module":"aztec:sandbox","timestamp":"2024-08-21T02:15:29.582Z"}
    {"level":"info","message":"Initialized Gas Portal at 0x4ed7c70f96b99c776995fb64377f0d4ab3b0e1c1 to bridge between L1 0x59b670e9fa9d0a427751af201d676719a970857b to L2 0x16a83e3395bc921a2441db55dce24f0e0932636901a2e676fa68b9b2b9a644c1","module":"aztec:sandbox","timestamp":"2024-08-21T02:15:29.591Z"}
    {"level":"info","message":"Deployed Rollup at 0x4a679253410272dd5232b3ff7cf5dbb88f295319","module":"aztec:sandbox","timestamp":"2024-08-21T02:15:29.608Z"}
    {"level":"info","message":"Inbox available at 0x532b02bd614fd18aee45603d02866cfb77575cb3","module":"aztec:sandbox","timestamp":"2024-08-21T02:15:29.610Z"}
    {"level":"info","message":"Outbox available at 0xe801273f775eacc1d74d1d43f92ec4524cabbd35","module":"aztec:sandbox","timestamp":"2024-08-21T02:15:29.612Z"}
    {"level":"info","message":"Using NoopTelemetryClient","module":"aztec:telemetry-client","timestamp":"2024-08-21T02:15:29.644Z"}
    {"level":"info","message":"Creating ephemeral data store","module":"aztec:node:lmdb","timestamp":"2024-08-21T02:15:29.645Z"}
    {"level":"info","message":"Performing initial chain sync to rollup contract 0x4a679253410272dd5232b3ff7cf5dbb88f295319","module":"aztec:archiver","timestamp":"2024-08-21T02:15:29.661Z"}
    {"level":"info","message":"Synced to L1 block 27","module":"aztec:archiver","timestamp":"2024-08-21T02:15:29.672Z"}
    {"level":"info","message":"Started block downloader from block 1","module":"aztec:world_state","timestamp":"2024-08-21T02:15:29.743Z"}
    {"level":"info","message":"Using WASM ACVM simulation","module":"aztec:node","timestamp":"2024-08-21T02:15:29.744Z"}
    {"level":"info","message":"Sequencer started","module":"aztec:sequencer","timestamp":"2024-08-21T02:15:29.745Z"}
    {"level":"info","message":"Started Aztec Node against chain 0x7a69 with contracts - \nRollup: 0x4a679253410272dd5232b3ff7cf5dbb88f295319\nRegistry: 0x3aa5ebb10dc797cac828524e59a333d0a371443c\nInbox: 0x532b02bd614fd18aee45603d02866cfb77575cb3\nOutbox: 0xe801273f775eacc1d74d1d43f92ec4524cabbd35\nAvailability Oracle: 0xc6e7df5e7b4f2a278906862b61205850344d4e7d","module":"aztec:node","timestamp":"2024-08-21T02:15:29.746Z"}
    {"level":"info","message":"Creating ephemeral data store","module":"aztec:kv-store","timestamp":"2024-08-21T02:15:29.753Z"}
    {"level":"info","message":"Creating ephemeral data store","module":"aztec:kv-store","timestamp":"2024-08-21T02:15:29.760Z"}
    {"level":"info","message":"Added contract ContractClassRegisterer at 0x05d15342d76e46e5be07d3cda0d753158431cdc5e39d29ce4e8fe1f5c070564a","module":"aztec:pxe_service","timestamp":"2024-08-21T02:15:29.795Z"}
    {"level":"info","message":"Added contract ContractInstanceDeployer at 0x2ab1a2bd6d07d8d61ea56d85861446349e52c6b7c0612b702cb1e6db6ad0b089","module":"aztec:pxe_service","timestamp":"2024-08-21T02:15:29.802Z"}
    {"level":"info","message":"Added contract MultiCallEntrypoint at 0x0e028bd5acf2e33f647132af47ce05a37a022c3f7a51bb426271bd6764bc0aaf","module":"aztec:pxe_service","timestamp":"2024-08-21T02:15:29.811Z"}
    {"level":"info","message":"Added contract FeeJuice at 0x16a83e3395bc921a2441db55dce24f0e0932636901a2e676fa68b9b2b9a644c1","module":"aztec:pxe_service","timestamp":"2024-08-21T02:15:29.820Z"}
    {"level":"info","message":"Added contract NewKeyRegistry at 0x2ee3f8c67efa88f9e6fb44242f1e9dcc0f9a6752ded07af0d9fac3875a61d421","module":"aztec:pxe_service","timestamp":"2024-08-21T02:15:29.827Z"}
    {"level":"info","message":"Added contract AuthRegistry at 0x24877c50868f86712240eb535d90d1c97403d074805dd3758c3aecb02958f8d4","module":"aztec:pxe_service","timestamp":"2024-08-21T02:15:29.834Z"}
    {"level":"info","message":"Initial sync complete","module":"aztec:pxe_synchronizer","timestamp":"2024-08-21T02:15:29.836Z"}
    {"level":"info","message":"Started PXE connected to chain 31337 version 1","module":"aztec:pxe_service","timestamp":"2024-08-21T02:15:29.849Z"}
    {"level":"info","message":"Aztec Server listening on port 8080","module":"aztec:cli","timestamp":"2024-08-21T02:15:29.852Z"}
    {"level":"info","message":"Shutting down...","module":"aztec:cli","timestamp":"2024-08-21T02:15:49.618Z"}
    ```
    spalladino committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    368722b View commit details
    Browse the repository at this point in the history
  6. fix: add link to l1-artifacts (#7940)

    Fix wallet not working due to missing link to `l1-artifacts`
    alexghr authored and spalladino committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    7b8cc83 View commit details
    Browse the repository at this point in the history
  7. fix: stop WorldStrateSynchronizer in prover node (#8126)

    Stop world state instance used by prover-node when node is shutdown
    alexghr authored and spalladino committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    845eb65 View commit details
    Browse the repository at this point in the history
  8. feat: prover node metrics (#8086)

    Tracks block proving duration
    alexghr authored and spalladino committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    56235a5 View commit details
    Browse the repository at this point in the history
  9. fix: start trace exporter only if required (#8147)

    This PR splits the collector URL into `_METRICS_ENDPOINT` and
    `_TRACES_ENDPOINT` and makes exporting traces optional (if no processor
    is registered then the default processor is a no-op)
    
    See
    https://www.npmjs.com/package/@opentelemetry/exporter-trace-otlp-http#user-content-configuration-options-as-environment-variables
    alexghr authored and spalladino committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    5eaaac5 View commit details
    Browse the repository at this point in the history
  10. chore: Configure interval mining for anvil (#8211)

    For all setups that are not e2e tests, sets up anvil to use 12-second
    block times interval mining, as opposed to automine. This will make
    setup a bit slower unfortunately, but will reduce divergence with the
    work on block building.
    spalladino committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    11405ec View commit details
    Browse the repository at this point in the history
  11. chore: Less noisy AVM failures in proving (#8227)

    If AVM fails to prove and AVM_PROVING_STRICT is not set, do not log an
    error.
    spalladino committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    8c0306d View commit details
    Browse the repository at this point in the history
  12. feat: Add CLI command for gathering proving metrics (#8221)

    Run `yarn aztec prover-stats --l1-rpc-url
    https://provernet-mainnet-fork.aztec.network:8545/$APIKEY -c 677692
    --start-block 15918000 --batch-size 200 --node-url
    https://api.aztec.network/provernet/aztec-node-1/$APIKEY` to get prover
    stats from provernet.
    spalladino committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    2aa82f9 View commit details
    Browse the repository at this point in the history
  13. fix: Docker containers healthchecks (#8228)

    Fixes `start_period` for aztec node in provernet healtcheck, as since
    #8211 the startup takes much longer.
    
    Fixes the healthcheck defined in the image itself, so it works with the
    common `status` route and does not require the pxe to be exposed.
    spalladino committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    f71119e View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2024

  1. chore: Add check to just release images to devnet-deploys (#8242)

    Allows us to push fixes to users without nuking the deployed network.
    spalladino committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    ba23fea View commit details
    Browse the repository at this point in the history
  2. fix: Log public vm errors as warn in prover-agent (#8247)

    And not the other way around. I need to relearn how booleans work.
    spalladino committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    afb2610 View commit details
    Browse the repository at this point in the history
  3. feat: Add status check to prover agent (#8248)

    Adds a status check and an endpoint for querying current jobs to the
    prover agent:
    
    ```
    $ curl -XPOST -d'{"method": "prover_getCurrentJobs"}' http://localhost:8085/
    {"result":[{"id":"ab3f1629","type":"ROOT_PARITY"}]}
    ```
    spalladino committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    d88c6fe View commit details
    Browse the repository at this point in the history

Commits on Aug 29, 2024

  1. feat: Block cheat codes in anvil (#8277)

    Adds a second api to the mainnet-fork, located at
    `public-PUBLIC_API_KEY`, in which all cheat codes are blocked. We define
    a cheat code as a method in the evm, hardhat, or anvil namespaces. We
    use njs for parsing the JSON RPC request body and testing the method
    namespace.
    
    The PUBLIC_API_KEY is only set for provernet, for other networks, we
    load the same API_KEY secret as always.
    
    This PR also enables logging for anvil, including a logrotate config to
    ensure we don't fill up the disk just with logs.
    spalladino committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    21ca377 View commit details
    Browse the repository at this point in the history
  2. chore: Bump provernet agents to 8 and speed up bot (#8280)

    So we have capacity to prove larger blocks
    spalladino committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    5075a1a View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2024

  1. fix: Do not reuse anvil admin key

    In #8277 we added a PUBLIC_API_KEY that was used for accessing anvil
    with cheat codes disabled, and the API_KEY was left for "admin" access
    with cheat codes allowed. However, the API_KEY is also needed for
    connecting to the node, which is is needed for retrieving proofs, needed
    for prover nodes to operate.
    
    This PR changes how keys are handled. API_KEY is now mapped to the anvil
    public access, so it is not disclosed when sharing access to the node
    API. A new FORK_ADMIN_API_KEY is introduced for admin access, and only
    used for bootstrapping operations and for the bootstrapping sequencer.
    spalladino committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    52d6b02 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    80121a6 View commit details
    Browse the repository at this point in the history
  3. chore: Report prover metrics (#8155)

    Reports a new histogram metric with the time that passes between a block
    being submitted and its proof. Uses the timestamp from the L2 block
    header global variables as block submission time, and the L1 block time
    for the proof submission time.
    
    Depends on #8193 
    
    Fixes #7675
    spalladino committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    009b655 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    e127933 View commit details
    Browse the repository at this point in the history

Commits on Sep 2, 2024

  1. feat: Add bot config to toggle simulation (#8297)

    Allow turning off simulation and gas estimation on the bot via config.
    spalladino committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    36be03b View commit details
    Browse the repository at this point in the history
  2. fix: Bot config for skip public simulation (#8320)

    We were not actually using the config value 🤦
    spalladino committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    a1662aa View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    74051cc View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    365c079 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    89c16c3 View commit details
    Browse the repository at this point in the history
  6. feat: Track proving times in prover stats in CLI

    Adds an option to compute proving times in the prover-stats CLI command,
    and split stats based on a configurable cutoff. Also ensures that the
    same proven block is only counted once per prover.
    
    Adds lodash.chunk. Sorry about that.
    spalladino committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    7a1ed69 View commit details
    Browse the repository at this point in the history
  7. fix: actually start proof verifier (#8332)

    Please read [contributing guidelines](CONTRIBUTING.md) and remove this
    line.
    alexghr authored Sep 2, 2024
    Configuration menu
    Copy the full SHA
    6f180fc View commit details
    Browse the repository at this point in the history
  8. Fixes

    PhilWindle committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    fefd188 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    bf0da08 View commit details
    Browse the repository at this point in the history
  10. Devnet changes

    PhilWindle committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    675dad2 View commit details
    Browse the repository at this point in the history
  11. Fix

    PhilWindle committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    cbf29c1 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    cdc4a10 View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2024

  1. chore: Configurable mainnet fork resources (#8342)

    This PR allows us to specify mainnet fork resources on an environment
    basis
    PhilWindle authored Sep 3, 2024
    Configuration menu
    Copy the full SHA
    279f2bb View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    0c116b5 View commit details
    Browse the repository at this point in the history
  3. fix: Handle proofs for blocks submitted out of range (#8357)

    `prover-stats` was throwing an error when querying with a start-block
    that was not the start of the chain, since it'd look up for the
    submission time of the L2 block for a proof, but would not find it
    because it was out of range.
    spalladino authored Sep 3, 2024
    Configuration menu
    Copy the full SHA
    9934adb View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2024

  1. Configuration menu
    Copy the full SHA
    b403a38 View commit details
    Browse the repository at this point in the history
  2. Merge fixes

    PhilWindle committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    c83487b View commit details
    Browse the repository at this point in the history
  3. Update .github/workflows/devnet-deploys.yml

    Downgrade alphanet bots
    
    Co-authored-by: Santiago Palladino <santiago@aztecprotocol.com>
    PhilWindle and spalladino authored Sep 4, 2024
    Configuration menu
    Copy the full SHA
    7f4c3a0 View commit details
    Browse the repository at this point in the history
  4. Fix

    PhilWindle committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    d9d95d4 View commit details
    Browse the repository at this point in the history
  5. Merge branch 'pw/provernet-to-master' of github.com:AztecProtocol/azt…

    …ec-packages into pw/provernet-to-master
    PhilWindle committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    0d25be6 View commit details
    Browse the repository at this point in the history