Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix algorithmic complexity of on-demand scheduler with regards to number of cores. #3190

Merged
merged 35 commits into from
Mar 20, 2024
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
fd41678
max -> min
Jan 25, 2024
9ee7aa1
Reduce default queue size on-demand
Jan 25, 2024
0b3c03d
Introduce max max queue size.
Feb 1, 2024
edad3a4
Use max max queue size.
Feb 1, 2024
ee74614
Implementation complete
Feb 1, 2024
16afcfa
Revert min/max fix.
Feb 1, 2024
c750146
Fixes + EncodeableBinaryHeap.
Feb 2, 2024
b0edbcf
Remove EncodeableBinaryHeap
Feb 2, 2024
cac2752
Patch scale-info for now.
Feb 2, 2024
a9b13f7
Revert default value.
Feb 2, 2024
f9ea4e4
Fixes + tests.
Feb 2, 2024
457b225
Bring back copyright.
Feb 2, 2024
ce7c8e8
Fix benchmark.
Feb 2, 2024
1bab42e
binary heap got merged
Feb 2, 2024
c21540d
Calculate on demand traffic on idle blocks
antonva Feb 9, 2024
999b7bb
Add migration for on demand provider
antonva Mar 4, 2024
4df8590
Merge branch 'master' into rk-on-demand-perf-proper-fix
antonva Mar 4, 2024
ecbae81
Readd missing export
antonva Mar 7, 2024
079c78f
Add storage version to on demand pallet
antonva Mar 7, 2024
ee91d8b
Merge branch 'master' into rk-on-demand-perf-proper-fix
antonva Mar 7, 2024
5308f3a
Address comments, add new scale-info
antonva Mar 13, 2024
2253a14
Merge branch 'master' into rk-on-demand-perf-proper-fix
antonva Mar 14, 2024
1ae8b7b
Bump scale-info version again
antonva Mar 14, 2024
3e60c09
Add test to migration to preserve queue ordering
antonva Mar 14, 2024
2dd2512
Merge branch 'master' of https://github.com/paritytech/polkadot-sdk i…
Mar 18, 2024
600fd5e
".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime…
Mar 18, 2024
ee29cc2
Fix post_upgrade in migration
antonva Mar 18, 2024
b344bc6
Add prdoc
antonva Mar 18, 2024
76e4e76
Remove unused on-demand max size import
antonva Mar 18, 2024
ab51045
Remove unused mut from test
antonva Mar 18, 2024
9a705cb
Remove benchmark todo
antonva Mar 18, 2024
f2ad19f
Simplify PartialOrd for EnqueuedOrder
antonva Mar 18, 2024
70f468e
".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime…
Mar 18, 2024
d8f1543
Address nits
antonva Mar 20, 2024
be3a3c1
Type sums in post migration
antonva Mar 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -604,3 +604,6 @@ wasmi = { opt-level = 3 }
x25519-dalek = { opt-level = 3 }
yamux = { opt-level = 3 }
zeroize = { opt-level = 3 }

[patch.crates-io]
scale-info = { git = 'https://github.com/paritytech/scale-info.git', branch="master" }
2 changes: 1 addition & 1 deletion polkadot/primitives/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ pub use v6::{
ValidationCodeHash, ValidatorId, ValidatorIndex, ValidatorSignature, ValidityAttestation,
ValidityError, ASSIGNMENT_KEY_TYPE_ID, LEGACY_MIN_BACKING_VOTES, LOWEST_PUBLIC_ID,
MAX_CODE_SIZE, MAX_HEAD_DATA_SIZE, MAX_POV_SIZE, ON_DEMAND_DEFAULT_QUEUE_MAX_SIZE,
PARACHAINS_INHERENT_IDENTIFIER, PARACHAIN_KEY_TYPE_ID,
ON_DEMAND_MAX_QUEUE_MAX_SIZE, PARACHAINS_INHERENT_IDENTIFIER, PARACHAIN_KEY_TYPE_ID,
};

#[cfg(feature = "std")]
Expand Down
7 changes: 7 additions & 0 deletions polkadot/primitives/src/v6/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -395,6 +395,13 @@ pub const MAX_POV_SIZE: u32 = 5 * 1024 * 1024;
/// Can be adjusted in configuration.
pub const ON_DEMAND_DEFAULT_QUEUE_MAX_SIZE: u32 = 10_000;

/// Maximum for maximum queue size.
///
/// Setting `on_demand_queue_max_size` to a value higher than this is unsound. This is more a
/// theoretical limit, just below enough what the target type supports, so comparisons are possible
/// even with indices that are overflowing the underyling type.
pub const ON_DEMAND_MAX_QUEUE_MAX_SIZE: u32 = 1_000_000_000;

/// Backing votes threshold used from the host prior to runtime API version 6 and from the runtime
/// prior to v9 configuration migration.
pub const LEGACY_MIN_BACKING_VOTES: u32 = 2;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -70,11 +70,7 @@ mod benchmarks {
let para_id = ParaId::from(111u32);
init_parathread::<T>(para_id);
T::Currency::make_free_balance_be(&caller, BalanceOf::<T>::max_value());
let order = EnqueuedOrder::new(para_id);

for _ in 0..s {
Pallet::<T>::add_on_demand_order(order.clone(), QueuePushDirection::Back).unwrap();
}
Pallet::<T>::populate_queue(para_id, s);

#[extrinsic_call]
_(RawOrigin::Signed(caller.into()), BalanceOf::<T>::max_value(), para_id)
Expand All @@ -87,11 +83,8 @@ mod benchmarks {
let para_id = ParaId::from(111u32);
init_parathread::<T>(para_id);
T::Currency::make_free_balance_be(&caller, BalanceOf::<T>::max_value());
let order = EnqueuedOrder::new(para_id);

for _ in 0..s {
Pallet::<T>::add_on_demand_order(order.clone(), QueuePushDirection::Back).unwrap();
}
Pallet::<T>::populate_queue(para_id, s);

#[extrinsic_call]
_(RawOrigin::Signed(caller.into()), BalanceOf::<T>::max_value(), para_id)
Expand Down
Loading
Loading