Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(rpc): reusable eth api EthCall #8683

Merged
merged 67 commits into from
Jun 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
cab888f
Make on_blocking_task method accessible in default trait method impls
emhane Jun 1, 2024
b902f35
Minimise thread interface eth api
emhane Jun 1, 2024
c35b40e
Update docs and symbol name
emhane Jun 1, 2024
8ccddd3
Add missing reth primitives to op feature
emhane Jun 1, 2024
257dc5d
Remove feature gate to fix broken test
emhane Jun 1, 2024
40194d6
Fix flaky test
emhane Jun 1, 2024
68e64b4
fixup! Fix flaky test
emhane Jun 1, 2024
30b63cc
fixup! Fix flaky test
emhane Jun 1, 2024
fcc64ea
Update lock file
emhane Jun 1, 2024
734ac99
Reinsert functions for spawning blocking threads into EthTransactions
emhane Jun 4, 2024
c000f43
Drive-by, update docs
emhane Jun 4, 2024
d1b91ad
Move spawn cpu heavy to trait
emhane Jun 4, 2024
eec99b8
Fix lint
emhane Jun 4, 2024
924024f
Revert impl for spawning blocking io
emhane Jun 4, 2024
ea6cbd3
Make impl of spawn blocking default trait methods
emhane Jun 4, 2024
ecb6771
Update docs
emhane Jun 5, 2024
fe494dc
Revert unrelated changes
emhane Jun 5, 2024
ba6f403
Merge branch 'main' into emhane/blocking-read
emhane Jun 5, 2024
55d2649
Merge branch 'matt/scaffold-ethapi' into emhane/blocking-read
emhane Jun 5, 2024
139f97f
Merge branch 'matt/scaffold-ethapi' into emhane/blocking-read
emhane Jun 5, 2024
8b8ed16
Fix lint
emhane Jun 5, 2024
f3ed6bd
Add super traits to spawn blocking
emhane Jun 5, 2024
16334cd
Move impl of methods using evm config to default trait methods
emhane Jun 5, 2024
20d2489
Move EthApi state access methods into trait
emhane Jun 5, 2024
4a40032
Move EthTransaction impls to default trait methods
emhane Jun 5, 2024
f2df915
Move state rpc methods into new trait default impl
emhane Jun 5, 2024
ca19811
fixup! Move state rpc methods into new trait default impl
emhane Jun 5, 2024
8ee373c
Move EthTransactions impl to default trait methods
emhane Jun 6, 2024
6dfac1a
Add trait for pending block
emhane Jun 6, 2024
90f67b6
Merge branch 'matt/scaffold-ethapi' into emhane/ethapi-txns
emhane Jun 6, 2024
0db764e
Merge branch 'matt/scaffold-ethapi' into emhane/ethapi-txns
emhane Jun 6, 2024
2340e87
Merge branch 'emhane/ethapi-txns' into emhane/ethapi-pending-block
emhane Jun 6, 2024
3c0b7ab
Fix lint
emhane Jun 6, 2024
6634723
Merge branch 'emhane/ethapi-txns' into emhane/ethapi-pending-block
emhane Jun 6, 2024
5e27f32
Move impl to get evm env at block to EthTransactions default trait me…
emhane Jun 6, 2024
e69b697
Move EthTransactions impl using evm_env_at into default trait methods
emhane Jun 6, 2024
ee101bc
Move block EthApi methods to EthBlocks default trait methods
emhane Jun 6, 2024
2ff0191
Move trace and call helper methods out of EthTransactions to use as d…
emhane Jun 7, 2024
cdd356b
Move EthApi fee methods to trait for reuse on any network and in eth_…
emhane Jun 7, 2024
bd029ff
Debug EthCall
emhane Jun 7, 2024
3211cfc
fixup! Debug EthCall
emhane Jun 8, 2024
42c494e
Keep hack wrappers in super scope
emhane Jun 8, 2024
c9d56e0
Merge matt/scaffold-ethapi
emhane Jun 8, 2024
9d12c70
Merge branch 'matt/scaffold-ethapi' into emhane/ethapi-call
emhane Jun 8, 2024
82de288
Debug EthFees trait
emhane Jun 8, 2024
3880ac0
Fix lint
emhane Jun 8, 2024
52ca8d9
Fix lint
emhane Jun 8, 2024
a414947
Elide lifetimes
emhane Jun 8, 2024
024a61b
Revert method name
emhane Jun 8, 2024
d613ccd
fixup! Revert method name
emhane Jun 8, 2024
c1d69d3
chore(deps): weekly `cargo update` (#8697)
github-actions[bot] Jun 9, 2024
2852893
chore: rename network-types to network-peers (#8699)
mattsse Jun 9, 2024
c7fd507
feat: remove Bandwidthmeter type (#8698)
kamuik16 Jun 9, 2024
2216b4b
chore: replace secp256k1 with fully qualified path (#8701)
mattsse Jun 9, 2024
da1536f
docs: improve custom-evm example docs (#8703)
mattsse Jun 9, 2024
ab8b458
feat: add storage consistency checks on CLI commands (#8667)
joshieDo Jun 10, 2024
40fc19f
chore: move workspace.package to top of Cargo.toml (#8691)
mattsse Jun 10, 2024
53f4eca
feat: add launch_with_fn (#8694)
mattsse Jun 10, 2024
ee7a829
feat: add ExExLauncher (#8693)
mattsse Jun 10, 2024
9a38b37
Merge branch 'main' into matt/scaffold-ethapi
emhane Jun 10, 2024
92c1750
Merge branch 'matt/scaffold-ethapi' into emhane/ethapi-call
emhane Jun 10, 2024
331fc3b
Revert unrelated changes
emhane Jun 10, 2024
6f4c5da
Return oppaque provider type
emhane Jun 10, 2024
43e64ee
Remove redundant trait bounds
emhane Jun 10, 2024
f69daca
Make extension trait for loading state
emhane Jun 10, 2024
7817e3c
Add extension traits
emhane Jun 11, 2024
7928b8d
Update module docs
emhane Jun 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 18 additions & 18 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

22 changes: 11 additions & 11 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,3 +1,12 @@
[workspace.package]
version = "0.2.0-beta.9"
edition = "2021"
rust-version = "1.76"
license = "MIT OR Apache-2.0"
homepage = "https://paradigmxyz.github.io/reth"
repository = "https://github.com/paradigmxyz/reth"
exclude = [".github/"]

[workspace]
members = [
"bin/reth/",
Expand Down Expand Up @@ -39,7 +48,7 @@ members = [
"crates/net/network-api/",
"crates/net/network/",
"crates/net/p2p/",
"crates/net/types/",
"crates/net/peers/",
"crates/node-core/",
"crates/node/api/",
"crates/node/builder/",
Expand Down Expand Up @@ -194,15 +203,6 @@ redundant_pub_crate = "allow"
significant_drop_in_scrutinee = "allow"
significant_drop_tightening = "allow"

[workspace.package]
version = "0.2.0-beta.9"
edition = "2021"
rust-version = "1.76"
license = "MIT OR Apache-2.0"
homepage = "https://paradigmxyz.github.io/reth"
repository = "https://github.com/paradigmxyz/reth"
exclude = [".github/"]

# Speed up tests.
[profile.dev.package]
proptest.opt-level = 3
Expand Down Expand Up @@ -282,7 +282,7 @@ reth-net-common = { path = "crates/net/common" }
reth-net-nat = { path = "crates/net/nat" }
reth-network = { path = "crates/net/network" }
reth-network-api = { path = "crates/net/network-api" }
reth-network-types = { path = "crates/net/types" }
reth-network-peers = { path = "crates/net/peers" }
reth-network-p2p = { path = "crates/net/p2p" }
reth-nippy-jar = { path = "crates/storage/nippy-jar" }
reth-node-api = { path = "crates/node/api" }
Expand Down
71 changes: 67 additions & 4 deletions bin/reth/src/commands/common.rs
Original file line number Diff line number Diff line change
@@ -1,20 +1,27 @@
//! Contains common `reth` arguments

use clap::Parser;
use reth_beacon_consensus::EthBeaconConsensus;
use reth_config::{config::EtlConfig, Config};
use reth_db::{init_db, open_db_read_only, DatabaseEnv};
use reth_db_common::init::init_genesis;
use reth_downloaders::{bodies::noop::NoopBodiesDownloader, headers::noop::NoopHeaderDownloader};
use reth_evm::noop::NoopBlockExecutorProvider;
use reth_node_core::{
args::{
utils::{chain_help, genesis_value_parser, SUPPORTED_CHAINS},
DatabaseArgs, DatadirArgs,
},
dirs::{ChainPath, DataDirPath},
};
use reth_primitives::ChainSpec;
use reth_provider::{providers::StaticFileProvider, ProviderFactory};
use reth_primitives::{stage::PipelineTarget, ChainSpec};
use reth_provider::{
providers::StaticFileProvider, HeaderSyncMode, ProviderFactory, StaticFileProviderFactory,
};
use reth_stages::{sets::DefaultStages, Pipeline};
use reth_static_file::StaticFileProducer;
use std::{path::PathBuf, sync::Arc};
use tracing::{debug, info};
use tracing::{debug, info, warn};

/// Struct to hold config and datadir paths
#[derive(Debug, Parser)]
Expand Down Expand Up @@ -77,14 +84,70 @@ impl EnvironmentArgs {
),
};

let provider_factory = ProviderFactory::new(db, self.chain.clone(), sfp);
let provider_factory = self.create_provider_factory(&config, db, sfp)?;
if access.is_read_write() {
debug!(target: "reth::cli", chain=%self.chain.chain, genesis=?self.chain.genesis_hash(), "Initializing genesis");
init_genesis(provider_factory.clone())?;
}

Ok(Environment { config, provider_factory, data_dir })
}

/// Returns a [`ProviderFactory`] after executing consistency checks.
///
/// If it's a read-write environment and an issue is found, it will attempt to heal (including a
/// pipeline unwind). Otherwise, it will print out an warning, advising the user to restart the
/// node to heal.
fn create_provider_factory(
&self,
config: &Config,
db: Arc<DatabaseEnv>,
static_file_provider: StaticFileProvider,
) -> eyre::Result<ProviderFactory<Arc<DatabaseEnv>>> {
let has_receipt_pruning = config.prune.as_ref().map_or(false, |a| a.has_receipts_pruning());
let factory = ProviderFactory::new(db, self.chain.clone(), static_file_provider);

info!(target: "reth::cli", "Verifying storage consistency.");

// Check for consistency between database and static files.
if let Some(unwind_target) = factory
.static_file_provider()
.check_consistency(&factory.provider()?, has_receipt_pruning)?
{
if factory.db_ref().is_read_only() {
warn!(target: "reth::cli", ?unwind_target, "Inconsistent storage. Restart node to heal.");
return Ok(factory)
}

let prune_modes = config.prune.clone().map(|prune| prune.segments).unwrap_or_default();

// Highly unlikely to happen, and given its destructive nature, it's better to panic
// instead.
assert_ne!(unwind_target, PipelineTarget::Unwind(0), "A static file <> database inconsistency was found that would trigger an unwind to block 0");

info!(target: "reth::cli", unwind_target = %unwind_target, "Executing an unwind after a failed storage consistency check.");

// Builds and executes an unwind-only pipeline
let mut pipeline = Pipeline::builder()
.add_stages(DefaultStages::new(
factory.clone(),
HeaderSyncMode::Continuous,
Arc::new(EthBeaconConsensus::new(self.chain.clone())),
NoopHeaderDownloader::default(),
NoopBodiesDownloader::default(),
NoopBlockExecutorProvider::default(),
config.stages.clone(),
prune_modes.clone(),
))
.build(factory.clone(), StaticFileProducer::new(factory.clone(), prune_modes));

// Move all applicable data from database to static files.
pipeline.move_to_static_files()?;
pipeline.unwind(unwind_target.unwind_target().expect("should exist"), None)?;
}

Ok(factory)
}
}

/// Environment built from [`EnvironmentArgs`].
Expand Down
2 changes: 1 addition & 1 deletion crates/consensus/auto-seal/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ reth-evm.workspace = true
reth-engine-primitives.workspace = true
reth-consensus.workspace = true
reth-rpc-types.workspace = true
reth-network-types.workspace = true
reth-network-peers.workspace = true
reth-tokio-util.workspace = true

# async
Expand Down
2 changes: 1 addition & 1 deletion crates/consensus/auto-seal/src/client.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ use reth_network_p2p::{
headers::client::{HeadersClient, HeadersFut, HeadersRequest},
priority::Priority,
};
use reth_network_types::{PeerId, WithPeerId};
use reth_network_peers::{PeerId, WithPeerId};
use reth_primitives::{BlockBody, BlockHashOrNumber, Header, HeadersDirection, B256};
use std::fmt::Debug;
use tracing::{trace, warn};
Expand Down
12 changes: 6 additions & 6 deletions crates/evm/src/provider.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,13 @@ use reth_storage_errors::provider::ProviderResult;
use revm::primitives::{BlockEnv, CfgEnv, CfgEnvWithHandlerCfg, SpecId};

/// A provider type that knows chain specific information required to configure a
/// [CfgEnvWithHandlerCfg].
/// [`CfgEnvWithHandlerCfg`].
///
/// This type is mainly used to provide required data to configure the EVM environment that is
/// usually stored on disk.
#[auto_impl::auto_impl(&, Arc)]
pub trait EvmEnvProvider: Send + Sync {
/// Fills the [CfgEnvWithHandlerCfg] and [BlockEnv] fields with values specific to the given
/// Fills the [`CfgEnvWithHandlerCfg`] and [BlockEnv] fields with values specific to the given
/// [BlockHashOrNumber].
fn fill_env_at<EvmConfig>(
&self,
Expand All @@ -24,7 +24,7 @@ pub trait EvmEnvProvider: Send + Sync {
where
EvmConfig: ConfigureEvmEnv;

/// Fills the default [CfgEnvWithHandlerCfg] and [BlockEnv] fields with values specific to the
/// Fills the default [`CfgEnvWithHandlerCfg`] and [BlockEnv] fields with values specific to the
/// given [Header].
fn env_with_header<EvmConfig>(
&self,
Expand All @@ -40,7 +40,7 @@ pub trait EvmEnvProvider: Send + Sync {
Ok((cfg, block_env))
}

/// Fills the [CfgEnvWithHandlerCfg] and [BlockEnv] fields with values specific to the given
/// Fills the [`CfgEnvWithHandlerCfg`] and [BlockEnv] fields with values specific to the given
/// [Header].
fn fill_env_with_header<EvmConfig>(
&self,
Expand All @@ -66,7 +66,7 @@ pub trait EvmEnvProvider: Send + Sync {
header: &Header,
) -> ProviderResult<()>;

/// Fills the [CfgEnvWithHandlerCfg] fields with values specific to the given
/// Fills the [`CfgEnvWithHandlerCfg`] fields with values specific to the given
/// [BlockHashOrNumber].
fn fill_cfg_env_at<EvmConfig>(
&self,
Expand All @@ -77,7 +77,7 @@ pub trait EvmEnvProvider: Send + Sync {
where
EvmConfig: ConfigureEvmEnv;

/// Fills the [CfgEnvWithHandlerCfg] fields with values specific to the given [Header].
/// Fills the [`CfgEnvWithHandlerCfg`] fields with values specific to the given [Header].
fn fill_cfg_env_with_header<EvmConfig>(
&self,
cfg: &mut CfgEnvWithHandlerCfg,
Expand Down
1 change: 0 additions & 1 deletion crates/net/common/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,4 @@ workspace = true
alloy-primitives.workspace = true

# async
pin-project.workspace = true
tokio = { workspace = true, features = ["full"] }
Loading
Loading