-
Notifications
You must be signed in to change notification settings - Fork 2.6k
Conversation
…bstrate-test-runner
…bstrate-test-runner
…bstrate into seun-substrate-test-runner
there's something funky going on with the diff |
fn spec_factory(id: String) -> Result<Box<dyn ChainSpec>, String> { | ||
Ok(match &*id { | ||
"dev" => Box::new(chain_spec::development_config()?), | ||
"" | "local" => Box::new(chain_spec::local_testnet_config()?), | ||
path => Box::new(chain_spec::ChainSpec::from_json_file( | ||
std::path::PathBuf::from(path), | ||
)?), | ||
}) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why are you moving that out again? It wasn't nice inside SubstrateCli?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It wasn't nice inside SubstrateCli?
unfortunately, having the spec_factory in the cli complicates the API for paritytech/substrate-test-runner, where users would also have to provide impl SubstrateCli
simply because we want the spec_factory. Splitting it out keeps our API lean and simple.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah it's for this? paritytech/substrate-test-runner#2
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok ok
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it related in any way with this? #6651
Are you trying to run nodes inside tests?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@seunlanlege I don't see the full picture yet so bare with me I might be completely wrong. If you can provide me documentation and more information on the project it would help me understand better the big picture.
Wild guess: I don't think you need sc-cli at all. sc-cli is for building clis, here you are building a library, you should be using sc-service and use the sc_service::Configuration object directly. It's sc-service that is the glue code and the center of everything, not sc-cli. You can't just pick the elements you want from sc-cli without applying to the full logic of sc-cli's usage.
Example: if you pass the arguments of the cli by code, I'm confident that this is wrong. sc-cli is really only meant for building a binary and the arguments are passed by a user.
impl<Runtime> InternalNode<Runtime> {
pub fn builder() -> InternalNodeBuilder<Runtime> {
InternalNodeBuilder::new()
}
pub fn new(logs: Logger, cli: &[String]) -> Self {
let cli = node_cli::Cli::from_iter(cli.iter());
...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(In other words: sc-cli must be a dependency of a bin crate, not a lib crate)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@cecton Please show them your code you have done for the polkadot test service. Which solves this problem AFAIK.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
https://github.com/paritytech/polkadot/blob/master/node/test-service/src/lib.rs
@seunlanlege This is a testing crate for polkadot. It uses sc-service directly. I just pass the chain-spec I want, tweak the sc_service::Configuration object the way I want, and that's how I can run a node. The nodes can communicate with each other using memory network so it doesn't open a port on the machine and I don't need to know the port number.
The PR #6651 is meant to improve testing nodes by providing an async runtime for the test and a timeout.
The PR #6555 is also meant to improve testing nodes by providing a "wait_for_blocks" function (to check that blocks are being created) and "send_transaction" to arbitrarily send a transaction to the node.
…bstrate-test-runner
01173c1
to
4b54972
Compare
@@ -336,9 +336,12 @@ impl TaskManager { | |||
} | |||
} | |||
|
|||
/// Set what the task manager should keep alive. | |||
pub(super) fn keep_alive<T: 'static + Send + Sync>(&mut self, to_keep_alive: T) { | |||
self.keep_alive = Box::new(to_keep_alive); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@expenses I was sure that you did make a tuple magic thingy here that would make this function callable multiple times 🤔 why was is it gone? I'm missing something...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You mean keep_alive((SOmehing, else, yolo))
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I totally mean S0mething, else, yolo.
I think she refactored her code before pushing and since keep_alive here is privat-ish she didn't foresee it could be misused.
Clearly something to fix, it's a pitfall.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah SOmething :P You should change your font to distinguish O and 0 :P
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The mix between the monospace font and sans font tricked my eyes
receiver.await? | ||
let res = receiver.await?; | ||
res |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What are these changes about?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
stray debug changes
pub fn io_handler(&self) -> Arc<jsonrpc_core::MetaIoHandler<sc_rpc::Metadata>> { | ||
self.0.clone() | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you absolutely need this function, or can you add the MetaIoHandler
methods directly to RpcHandlers
? If so, that would be cleaner and more abstract IMO.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok so the use case for this is to allow end users (and me 😇 ) use the jsonrpc-core-client
local transport, which takes a Deref<Target=MetaIoHandler> + 'static
. Hence the need to have the MetaIoHandler
behind an Arc
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, fair enough : )
/// Set what the task manager should keep alive. | ||
pub(super) fn keep_alive<T: 'static + Send + Sync>(&mut self, to_keep_alive: T) { | ||
self.keep_alive = Box::new(to_keep_alive); | ||
/// Set what the task manager should keep alive, can be called multiple times. | ||
pub fn keep_alive<T: 'static + Send + Sync>(&mut self, to_keep_alive: T) { | ||
// allows this fn to safely called multiple times. | ||
use std::mem; | ||
let old = mem::replace(&mut self.keep_alive, Box::new(())); | ||
self.keep_alive = Box::new((to_keep_alive, old)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you absolutely need to call this function more than once, then this is fine..
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's just to tempting to call it more than once imo. If there is no downside we should do it
Cargo.lock
Outdated
@@ -12,24 +12,24 @@ dependencies = [ | |||
|
|||
[[package]] | |||
name = "addr2line" | |||
version = "0.13.0" | |||
version = "0.12.2" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like you ran cargo update
here. Would it be possible to undo this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Running git checkout origin/master -- Cargo.lock
and cargo check
briefly would do nicely
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me!
Does this really need a polkadot pr? |
@seunlanlege Yes, because it breaks polkadot (see CI) 😉 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome
/// Set what the task manager should keep alive. | ||
pub(super) fn keep_alive<T: 'static + Send + Sync>(&mut self, to_keep_alive: T) { | ||
self.keep_alive = Box::new(to_keep_alive); | ||
/// Set what the task manager should keep alive, can be called multiple times. | ||
pub fn keep_alive<T: 'static + Send + Sync>(&mut self, to_keep_alive: T) { | ||
// allows this fn to safely called multiple times. | ||
use std::mem; | ||
let old = mem::replace(&mut self.keep_alive, Box::new(())); | ||
self.keep_alive = Box::new((to_keep_alive, old)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's just to tempting to call it more than once imo. If there is no downside we should do it
bot merge |
Checks failed; merge aborted. |
well that spell backfired |
You need to restart the polkadot test manually @seunlanlege |
Yeah I already did, but the test keeps failing |
You forgot to put the polkadot companion in the SUBSTRATE PR description |
As you can see at the end of the logs, now it's using the right branch! But the error is still there. Since you created your polkadot branch much later maybe you should check that your substrate branch is up-to-date with master. Then re-update your polkadot branch too. If the problem persists, it means the current substrate master doesnt build with the current polkadot master. You can select the commit of polkadot that does work instead:
(Note that this command will work only if the last merged commit is older than the one it will find.) This will make sure you are using the exact same version of substrate. |
…bstrate-test-runner
…bstrate into seun-substrate-test-runner
bot merge |
Trying merge. |
commit f8c83bd Author: Roman Borschel <romanb@users.noreply.github.com> Date: Tue Aug 18 07:59:32 2020 +0200 Add support for sourced metrics. (paritytech#6895) * Add support for sourced metrics. A sourced metric is a metric that obtains its values from an existing source, rather than the values being independently recorded. It thus allows collecting metrics from existing counters or gauges without having to duplicate them in a dedicated prometheus counter or gauge (and hence another atomic value). The first use-case is to feed the bandwidth counters from libp2p directly into prometheus. * Tabs, not spaces. * Tweak bandwidth counter registration. * Add debug assertion for variable labels and values. * Document monotonicity requirement for sourced counters. * CI * Update client/network/src/service.rs Co-authored-by: Max Inden <mail@max-inden.de> Co-authored-by: Max Inden <mail@max-inden.de> commit 8e1ed7d Author: Shawn Tabrizi <shawntabrizi@gmail.com> Date: Mon Aug 17 22:59:23 2020 +0200 WeightInfo for System, Timestamp, and Utility (paritytech#6868) * initial updates to system * fix compile * Update writer.rs * update weights * finish system weights * timestamp weights * utility weight * Fix overflow in weight calculations * add back weight notes * Update for whitelisted benchmarks * add trait bounds * Revert "add trait bounds" This reverts commit 12b08b7. * Update weights for unaccounted for read commit 399421a Author: Wei Tang <wei@that.world> Date: Mon Aug 17 21:07:30 2020 +0200 Derive Clone for AlwaysCanAuthor, NeverCanAuthor, CanAuthorWithNativeVersion (paritytech#6906) commit 287ecc2 Author: Wei Tang <wei@that.world> Date: Mon Aug 17 19:36:29 2020 +0200 pow: add access to pre-digest for algorithm verifiers (paritytech#6900) * pow: fetch pre-runtime digest to verifier * Add Other error type * Fix log target and change docs to refer to pre_runtime commit 488b7c7 Author: Wei Tang <wei@that.world> Date: Mon Aug 17 13:41:09 2020 +0200 babe, aura, pow: only call check_inherents if authoring version is compatible (paritytech#6862) * pow: check can_author_with before calling check_inherents * babe: check can_author_with before calling check_inherents * aura: check can_author_with before calling check_inherents * Fix node and node template compile * Add missing comma * Put each parameter on its own line * Add debug print * Fix line width too long * Fix pow line width issue commit fc743da Author: Pierre Krieger <pierre.krieger1708@gmail.com> Date: Mon Aug 17 11:19:16 2020 +0200 Add a DirectedGossip struct (paritytech#6803) * Add a DirectedGossip struct * Move protocol from prototype::new to biuld * More traits impls * Explain ordering * Apply suggestions from code review Co-authored-by: Toralf Wittner <tw@dtex.org> * Address concerns * Add basic test * Concerns * More concerns * Remove QueueSenderPrototype * Rename * Apply suggestions from code review Co-authored-by: Max Inden <mail@max-inden.de> Co-authored-by: Toralf Wittner <tw@dtex.org> Co-authored-by: parity-processbot <> Co-authored-by: Max Inden <mail@max-inden.de> commit 0079140 Author: Bastian Köcher <bkchr@users.noreply.github.com> Date: Sun Aug 16 00:05:36 2020 +0200 Don't take the origin in `can_set_code` (paritytech#6899) It makes no sense that `can_set_code` takes the origin for checking it. Everybody reusing this function is only interested in the other checks that are done by this function. The origin should be checked by every dispatchable individually. commit cd3b62b Author: Seun Lanlege <seunlanlege@gmail.com> Date: Sat Aug 15 10:08:31 2020 +0100 RpcHandlers Refactorings (paritytech#6846) * allow access to the underlying Pubsub instance from RpcHandlers * bump Cargo.lock * no more Arc<RpcHandlers> * bump Cargo.lock * Debug,. * Arc<RpcHandlers> * RpcHandler * RpcHandlers::io_handler * remove chain spec from cli * address pr comments * remove stray newline Co-authored-by: Ashley <ashley.ruglys@gmail.com> Co-authored-by: Tomasz Drwięga <tomasz@parity.io> Co-authored-by: Ashley <ashley.ruglys@gmail.com> commit eec7d71 Author: Max Inden <mail@max-inden.de> Date: Fri Aug 14 18:15:45 2020 +0200 client/authority-discovery: Revert query interval change (paritytech#6897) Revert the accidental query interval change from every one minute back to every 10 minutes. commit 13b0650 Author: Roman Borschel <romanb@users.noreply.github.com> Date: Fri Aug 14 10:41:47 2020 +0200 Update to libp2p-0.23. (paritytech#6870) * Update to libp2p-0.23. Thereby incorporate bandwidth measurement along the lines previously done by libp2p itself. * Tweak dependencies for wasm32 compilation. For wasm32 we need to enable unstable features to make `task::Builder::local` available. * Simplify dependencies. * Simplify. Leave the calculation of bytes sent/received per second to the outer layers of the code, subject to their own individual update intervals. * Cleanup * Re-add lost dev dependency. * Avoid division by zero. * Remove redundant metric. * Enable sending of noise legacy handshakes. * Add comment about monotonic gauge. * CI commit 0e703a5 Author: Alan Sapede <alan.sapede@gmail.com> Date: Fri Aug 14 04:15:59 2020 -0400 Adds debug logs to EVM frame (paritytech#6887) commit f16cbc1 Author: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Date: Thu Aug 13 23:30:22 2020 +0200 More renaming to move away from phragmen. (paritytech#6886) commit 8993a75 Author: André Silva <123550+andresilva@users.noreply.github.com> Date: Thu Aug 13 19:38:14 2020 +0100 network: don't log re-discovered addresses (paritytech#6881) * network: move LruHashSet to network crate utils * network: don't log re-discovered external addresses * Update client/network/src/utils.rs Co-authored-by: mattrutherford <44339188+mattrutherford@users.noreply.github.com> Co-authored-by: mattrutherford <44339188+mattrutherford@users.noreply.github.com> commit 4d3c948 Author: Alexander Popiak <alexander.popiak@parity.io> Date: Thu Aug 13 18:54:05 2020 +0200 add runtime migrations to release notes/changelog (paritytech#6875) commit d019a66 Author: Wei Tang <wei@that.world> Date: Thu Aug 13 14:53:42 2020 +0200 pallet-evm: avoid double fee payment (paritytech#6858) * pallet-evm: avoid double fee payment * Only skip fee payment for successful calls commit ed4f7a1 Author: Bastian Köcher <bkchr@users.noreply.github.com> Date: Wed Aug 12 21:35:10 2020 +0200 Make `HexDisplay` useable in `no_std` (paritytech#6883) Actually I use this quite often when debugging some WASM bugs and there is no harm in enabling it by default. Before I just always copied it everytime I needed it. commit 473a23f Author: Max Inden <mail@max-inden.de> Date: Wed Aug 12 16:16:40 2020 +0200 client/authority-discovery: Introduce AuthorityDiscoveryService (paritytech#6760) * client/authority-discovery: Rename AuthorityDiscovery to XXXWorker * client/authority-discovery: Introduce AuthorityDiscoveryService Add a basic `AuthorityDiscoveryService` implementation which enables callers to get the addresses for a given `AuthorityId` from the local cache. * client/authority-discovery: Split into worker and service mod Move `Service` and `Worker` to their own Rust modules resulting in the following file structure. ├── build.rs ├── Cargo.toml └── src ├── error.rs ├── lib.rs ├── service.rs ├── tests.rs ├── worker │ ├── addr_cache.rs │ ├── schema │ │ └── dht.proto │ └── tests.rs └── worker.rs * client/authority-discovery: Cache PeerId -> AuthorityId mapping * client/authority-discovery: Update priority group on interval Instead of updating the authority discovery peerset priority group each time a new DHT value is found, update it regularly on an interval. This removes the need for deterministic random selection. Instead of trying to return a random stable set of `Multiaddr`s, the `AddrCache` now returns a random set on each call. * client/authority-discovery: Implement Service::get_authority_id * client/authority-discovery: Use HashMap instead of BTreeMap * client/authority-discovery: Rework priority group interval * client/authority-discovery: Fix comment * bin/node/cli: Update authority discovery constructor * client/authority-discovery: Fuse from_service receiver * client/authority-discovery: Remove Rng import * client/authority-discovery: Ignore Multiaddr without PeerId * client/authority-discovery/service: Add note on returned None * client/authority-discovery/addr_cache: Replace double clone with deref commit c495f89 Author: Cecile Tonglet <cecile@parity.io> Date: Wed Aug 12 16:07:11 2020 +0200 Add async test helper to timeout and provide a task_executor automatically (paritytech#6651) * Initial commit Forked at: 60e3a69 Parent branch: origin/master * Add async test helper to timeout and provide a task_executor automatically * simplify error message to avoid difference between CI and locally * forgot env var * Use runtime env var instead of build env var * Rename variable to SUBSTRATE_TEST_TIMEOUT * CLEANUP Forked at: 60e3a69 Parent branch: origin/master * Apply suggestions from code review Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Re-export from test-utils * Default value to 120 * fix wrong crate in ci * Revert "Default value to 120" This reverts commit 8e45871. * Fix version * WIP Forked at: 60e3a69 Parent branch: origin/master * WIP Forked at: 60e3a69 Parent branch: origin/master * WIP Forked at: 60e3a69 Parent branch: origin/master * remove feature flag * fix missing dependency * CLEANUP Forked at: 60e3a69 Parent branch: origin/master * fix test * Removed autotests=false * Some doc... * Apply suggestions from code review Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * WIP Forked at: 60e3a69 Parent branch: origin/master * WIP Forked at: 60e3a69 Parent branch: origin/master * Update test-utils/src/lib.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> commit 0c3cdf1 Author: mattrutherford <44339188+mattrutherford@users.noreply.github.com> Date: Wed Aug 12 12:53:21 2020 +0100 Implement tracing::Event handling & parent_id for spans and events (paritytech#6672) * implement events handling, implement parent_id for spans & events * add events to sp_io::storage * update test * add tests * adjust limit * let tracing crate handle parent_ids * re-enable current-id tracking * add test for threads with CurrentSpan * fix log level * remove redundant check for non wasm traces * remove duplicate definition in test * Adding conditional events API * prefer explicit parent_id over current, enhance test * limit changes to client::tracing event implementation * remove From impl due to fallback required on parent_id * implement SPAN_LIMIT change event log output * change version of tracing-core * update dependancies * revert limit * remove duplicate dependency * Apply suggestions from code review Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> Co-authored-by: Matt Rutherford <mattrutherford@users.noreply.github.com> Co-authored-by: Benjamin Kampmann <ben@parity.io> Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> commit 5b809d2 Author: Wei Tang <wei@that.world> Date: Wed Aug 12 12:46:28 2020 +0200 pallet-evm: fix wrong logic in mutate_account_basic (paritytech#6786) * pallet-evm: fix wrong logic in mutate_account_basic * Add test for mutate account commit f6d66db Author: Pierre Krieger <pierre.krieger1708@gmail.com> Date: Wed Aug 12 11:58:01 2020 +0200 Add a warning if users pass --sentry or --sentry-nodes (paritytech#6779) * Add a warning if users pass --sentry or --sentry-nodes * Apply suggestions from code review Co-authored-by: Max Inden <mail@max-inden.de> * Fix text Co-authored-by: parity-processbot <> Co-authored-by: Max Inden <mail@max-inden.de> commit d7979d0 Author: Shaopeng Wang <spxwang@gmail.com> Date: Wed Aug 12 21:21:36 2020 +1200 Implement 'transactional' annotation for runtime functions. (paritytech#6763) * Implement 'transactional' annotation for runtime functions. * Allow function attributes for dispatchable calls in decl_module. * decl_module docs: add transactional function example. * decl_module docs: add function attributes notes. * Fix license header. commit d4efdf0 Author: Pierre Krieger <pierre.krieger1708@gmail.com> Date: Wed Aug 12 10:58:16 2020 +0200 Fuse the import queue receiver (paritytech#6876) * Fix the import queue receiver * Add logging commit a20fbd5 Author: André Silva <123550+andresilva@users.noreply.github.com> Date: Tue Aug 11 22:21:45 2020 +0100 docs: fix references to code of conduct document (paritytech#6879) commit e7cc595 Author: h4x3rotab <h4x3rotab@gmail.com> Date: Wed Aug 12 04:12:34 2020 +0800 Add Phala Network SS58 address type (paritytech#6758) commit fe3fc04 Author: André Silva <123550+andresilva@users.noreply.github.com> Date: Tue Aug 11 20:55:15 2020 +0100 docs: convert code of conduct to markdown (paritytech#6878) commit 72addfa Author: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Date: Tue Aug 11 17:07:17 2020 +0200 Fix wrong staking doc about transaction payment. (paritytech#6873) * Fx paritytech#4616 * Fix paritytech#4616 commit 4064378 Author: André Silva <123550+andresilva@users.noreply.github.com> Date: Tue Aug 11 16:05:59 2020 +0100 grandpa: change some logging from trace to debug (paritytech#6872) * grandpa: change some logging from trace to debug * grandpa: cleanup unused import commit 6f57582 Author: Nikolay Volf <nikvolf@gmail.com> Date: Tue Aug 11 18:05:31 2020 +0300 Move to upstream wasmtime, refactor globals snapshot (paritytech#6759) * refactor globals snapshot * ignore test * update pwasm-utils ref * line width * add doc comment for internal struct * add explanation for iteration * Demote rustdoc to a comment * use 0.14 Co-authored-by: Sergei Shulepov <sergei@parity.io> commit a362997 Author: Arkadiy Paronyan <arkady.paronyan@gmail.com> Date: Tue Aug 11 16:12:30 2020 +0200 Block packet size limit (paritytech#6398) * Block packet size limit * Update client/network/src/protocol.rs Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com> * Add block response limit Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
This provides access to the underlying
MetaIoHandler
fromRpcHandlers
polkadot companion: paritytech/polkadot#1568