Oracle V4 EPIC #495
Open
Oracle V4 EPIC #495
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add holesky WF
…-oracle into feat/log-requests-in-ejector-module
Co-authored-by: Sergey Khomutinin <31664571+skhomuti@users.noreply.github.com>
…stency Move lido linters to checks.yml
Fix/develop hotfix
…-module Feat: add more logs to code and log validators request in ejector module
Revert "Fix/develop hotfix"
* add label to FRAME_PREV_REPORT_REF_SLOT * add withdrawal exit epoch * add labels to CONTRACT_ON_PAUSE * linter fix * fix typo * fix negative block on forks * Fix typo in docs * Fix typo in test function name * Fix typo in test function name * feat: attestation and committees * fix: type * fix: return -> yield * fix: pylint in tests * fix: mypy issues * fix: test * chore: change typing for clients * refactor: strict way for streams `_get_without_fallbacks` * chore: remove non-mandatory pylint ignore * make dockerfile buildable * feat: add CSM performance oracle module * contracts create * temp changes * rename typings * fix get_latest_blockstamp * fix test consensus * tests fixes * run black * fix linters * add blockhash * fix tests * fix mypy * fix linter * feat: CSM contracts as a separate module * fix comments * pylint fixed * fix lru cache * test for cache * fix linter * fix linters * remove unused import * fix: off-by-one error on logs fetching * chore: op_id -> no_id * feat: lazy-loading of CSM module * fix: get ABIs for contracts * chore: update docs for env variables * docs: few more corrections to README * chore: fix timeout_decorator type stub * fix: remove lru_cache from CSFeeOracle * feat: collect data for report building * fix: processing * fix: review * fix: ignore * feat: CSM contracts as a separate module * fix: off-by-one error on logs fetching * chore: op_id -> no_id * feat: lazy-loading of CSM module * fix: get ABIs for contracts * chore: update docs for env variables * docs: few more corrections to README * chore: fix timeout_decorator type stub * fix: remove lru_cache from CSFeeOracle * fix: lint * unify get call function * back to HexBytes * feat: add dummy IPFS provider (#436) * feat: `get_module_validators_by_node_operators` (#438) * feat: build CSM oracle report (#437) * fix: calculated l_ref_slot and r_ref_slot (#441) * fix: calculated l_ref_slot and r_ref_slot * fix: calculated l_ref_slot and r_ref_slot * CSM oracle hot fixes (#442) * feat: handle thread exception in processing (#443) * feat: handle thread exception in processing * fix: remove unused things * fix: use `ValueError`, shutdown the executor * fix: linter * chore: add a comment * fix: rise `SystemExit` (#446) * fix: put a stone into a tree (#444) * feat: `epochs_to_process` (#445) * feat: `epochs_to_process` * feat: two sets * feat: advanced validation * fix: use `unprocessed_epochs` * fix: private state properties * revert: `_data` -> ``data` * feat: gw3.io provider and MultiIPFSProvider (#447) * Fix report (#448) * fix: checkpoint range assert (#449) * fix: use reference blockstamp as r_slot for stuck events search (#450) * feat: use attestations endpoint * feat: fetch stuck events for the given frame only (#452) * feat: configure workers count for CSM module * chore: remove DummyIPFSProvider from main * fix: tests import * fix: csm `module` property * fix: logging CID via repr * chore: remove stone when not needed * fix: submit_report_data accepts a tuple * chore: fix mypy * feat: implement dynamic versions * fix: black: two lines * replace wget to culr * fix: call get_contract_version with block hash * Close some TODOs (#456) * fix message * chore: move iterator to folder * chore: add timestamp to logs * feat: add tests to lido contracts * fix: test slot * fix pylint * black fix * update workflow * chore: add v2 iterator * fix tests * fix: review * chore: move iterator to folder * chore: add timestamp to logs * feat: add tests to lido contracts * fix: test slot * fix pylint * black fix * update workflow * fix: review * fix: pylint * feat: absolute performance leeway * fix integration tests * iterator update * fix tests * fix: iterator test * feat: Multi Third Phase * squash sr abi * chore: use TOTAL_BASIS_POINTS const * fix middleware test * fix test * feat: ao-multi-third-phase-partial * fix: review * fix tests * fix: cast to tuple issue * fix log * fix: first validator issue * fix pylint * fix all tests * feat: Multi Third Phase * feat: ao-multi-third-phase-partial * fix: review * fix tests * fix: cast to tuple issue * fix log * tweak ejector * feat: checkpoints refactoring + tests (#463) * feat: review fixes * fix: linters * fix: lru cache * fix: call from v2 old contract * fix: linters * fix: `CSFeeOracle` with no initial epoch * Update src/modules/csm/csm.py Co-authored-by: Don Perignom <10616301+madlabman@users.noreply.github.com> * fix: first report l_ref_slot (#466) * fix: error messages for `validate_for_report` (#467) * fix: error messages for `validate_for_report` * fix: use error message explicitly * fix: Add warning log * feat: Update python to 3.12 * fix: State network average calculation * fix: compute operators shares via assigned attestations * refactor: fix linters * refactor: calculate distribution * tests: fix missing assert * refactor: no cache for stuck_operators * fix tests and improve cache * chore: log is_deployed result Co-authored-by: Raman Siamionau <1590415904a@gmail.com> * chore: add CSM_MODULE_ADDRESS to .env.example * chore: add more catch statements for IPFS * chore: add missing variables to README * fix: check contracts via _load_contracts * refactor: CSFeeDistributor calls casts and logs * chore: debug -> info logs on calls in CSM contracts * refactor: add Contract suffix to CSM contracts * Fix: wf * refactor: InconsistentEvents * refactor: move TOTAL_BASIS_POINTS out of perf_leeway * chore: fix sequence assert * chore: update CSModule abi * fix linters * fix: use module address to get keys from keysapi * fix: remove `no_cache` method for committees * fix: merge `get` and `get stream` base methods * feat: check module address * refactor: extract get_events_in_range into utils * fix: rework `get_first_non_missed_slot` * chore: remove events inconsistency check from cs_module * refactor: use get_events_in_range across the codebase * fix: add `Final` for env vars * tests: fix test_calculate_distribution * fix after merge * fix: black * fix: tests * refactor: extract get_operators_with_stucks_in_range * refactor: block -> block_identifier * chore: alias for Shares * fix: get_stuck_operators_ids return * tests: rework events utils tests * fix test names * fix: calculate_distribution shares as a defaultdict * docs: CSM-only variables as CSM only in README * Update src/utils/slot.py * chore: fix logging for get_events_in_range * chore: StakingModuleAddress as ChecksumAddress * chore: update authors * chore: remove redundant cache * refactor: move State.load to __init__ * fix: consensus farFutureEpoch const * chore: add check for l_ref_slot * fix: `minimum checkpoint step is not reached` is not error (#472) * fix: review * fix: test * feat: modify cache * comment problematic integration test * feat: block roots from state, events (#474) * feat: block roots from state, events * fix: check events len * fix: LazyCSM -> CSM * fix: assert * feat: check providers for CSM frame * feat: add DepositContract * chore: fix linters --------- Co-authored-by: madlabman <10616301+madlabman@users.noreply.github.com> * fix: linters * fix: `get_attestation_committees` returns list now (#476) * fix: `get_attestation_committees` returns list now * fix: linter * fix: `_get_block_roots` * fix: test * fix: checkpoint slot and tests * fix: use condition from spec * chore: set default concurrency for CSM to "2" * fix: iterator conditions (#477) * fix: iterator conditions * fix: black * fix: use max_available_epoch_to_check * fix: _get_block_roots * fix: black * fix: _select_block_roots * fix: pylint * feat: `batched` * feat: get CSFeeOracle address via CSModule (#478) * feat: check IPFS provider for CSM (#481) * feat: check IPFS provider for CSM * feat: IPFS checker all providers * feat: IPFS checker mark skipped providers * fix: review (#480) * fix: review * fix: linter * fix: tests * fix: no faker for checker --------- Co-authored-by: F4ever <1590415904a@gmail.com> Co-authored-by: shuoer86 <129674997+shuoer86@users.noreply.github.com> Co-authored-by: vgorkavenko <vladimir.g@lido.fi> Co-authored-by: Vladimir Gorkavenko <32727352+vgorkavenko@users.noreply.github.com> Co-authored-by: F4ever <raman.s@lido.fi>
* refactor: timeit decorator * refactor: distinct type for Committees * chore: annotations for bits functions * refactor: simplify process_attestations * refactor: CheckpointProcessor executor * refactor: State class * refactor: current_frame_range to return epochs * refactor: raise NoModuleFound * fix: is_contract_reportable condition * chore: test_timeit formatting * refactor: error for invalid distribution numbers * refactor: InvalidState as ValueError * refactor: extract make_tree method from build_report * refactor: one more State refactor * fix: ThreadPoolExecutor cannot be used after shutdown * tests: fix no_commit fixture * fix: build_report frame range * chore: custom CSMError * chore: list -> tuple
* fix: get r_ref_slot before initial epoch * refactor: current_frame_range --------- Co-authored-by: madlabman <10616301+madlabman@users.noreply.github.com>
feat: separate CSM frame log publishing. additional tests
…report fix: negative rebase check with lido handleOracleReport simulation
chore: add explaining log to collect_data
Increase middleware coverage
Increase unit tests coverage
Improve unit tests coverage in metrics module
Do not calculate wr batches if no eth
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Feature:
More detailed logs to VEBO
More specified metrics
Upgrade dependencies
Oracle modules have dynamic version support
VEBO V4: Force exits and new eject order
AO V4: Multi Third Phase support
Fixes:
Zero division error
Unhealthy container
Transaction submission unhandled exception
list_of_dataclasses decorator can't proceed empty lists
Negative block number in event logs
Docker building fix
Fix typos
Dockerfile fix: no wget package
Oracle backward compitability
Workflows:
Add Holesky wf
Upgrade docker file
Move linters to separate file
Refactoring:
Feat: Abstract contract interaction
Tests:
Integration tests for contracts interaction