Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Elastic scaling: runtime v2 descriptor support #5423

Open
wants to merge 117 commits into
base: master
Choose a base branch
from
Open

Elastic scaling: runtime v2 descriptor support #5423

wants to merge 117 commits into from
+1,541 −299

Conversation

sandreim
Copy link
Contributor

@sandreim sandreim commented Aug 20, 2024 •
edited
Loading

Closes #5045 and #5046

On top of #5362

TODO:

  • storage migration for allowed relay parents tracker
  • check session index
  • PRdoc
  • tests
  • ensure UMP queue cannot be abused with this change
  • Zombienet runtime upgrade test

@sandreim
WIP primitives
793142e 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
WIP
3a29fdf 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
Working version.
4a53577 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
Better version
8285ae7 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
Add missing primitives and fix things
2831c5e 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
Implement v2 receipts in polkadot-runtime-parachains
c767d60 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
add missing stuff
96999e3 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
Switch parachains runtime to use new primitives
c5f2dc3 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
use vstaging primitives
dbb0160 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
update rococo and westend
5efab68 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
client keeps using the old primitives
c2232e4 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
no unsafe pls
87b079f 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
move async backing primtiives to own file
00e8c13 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
fix
cd4d02f 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
fix test build
5509e33 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
fix test-runtime
f8b86d2 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
self review feedback
fe2fbfb 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
review feedback
975e13b 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
feedback
1c7ac55 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
feedback
653873b 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
clippy
dc98149 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
chores
0a6bce3 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
Filter v2 candidate descriptors
5e4dac2 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
fix
f12ca7a 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
fix prospective parachains tests
13734de 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
fix fix
effb1cc 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
fmt
3f75cba 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
fix comment
75a47bb 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
another one
12ed853 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
fix build
f2c0882 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim sandreim mentioned this pull request Sep 11, 2024
Open
1 task
alindima
alindima reviewed Sep 13, 2024
View reviewed changes
Copy link
Contributor

@alindima alindima left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good! Have you tested the runtime migration with an upgrade in zombienet?
I've always discovered bugs by doing that in the past 😆

polkadot/primitives/src/vstaging/mod.rs Show resolved Hide resolved
polkadot/primitives/src/vstaging/mod.rs Outdated Show resolved Hide resolved
polkadot/runtime/parachains/src/inclusion/mod.rs Outdated Show resolved Hide resolved
polkadot/runtime/parachains/src/paras_inherent/mod.rs Outdated Show resolved Hide resolved
polkadot/runtime/parachains/src/paras_inherent/mod.rs Outdated Show resolved Hide resolved
polkadot/runtime/parachains/src/shared.rs Outdated Show resolved Hide resolved
polkadot/runtime/parachains/src/shared/migration.rs Outdated Show resolved Hide resolved
polkadot/runtime/parachains/src/shared/migration.rs Outdated Show resolved Hide resolved
polkadot/runtime/parachains/src/shared/migration.rs Outdated Show resolved Hide resolved
polkadot/runtime/parachains/src/shared/migration.rs Show resolved Hide resolved
@sandreim
feedback
caff543 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
refactor
218f530 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
Copy link
Contributor Author

Looking good! Have you tested the runtime migration with an upgrade in zombienet? I've always discovered bugs by doing that in the past 😆

Not yet, but I don't really expect any issues, it is a very simple migration.

@sandreim
fix try-runtime
216937a 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
check ump signal count and test
c0aee8c 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
Copy link
Contributor Author

Runtime upgrade successful in Zombienet using a Westend local chain.

@alindima alindima mentioned this pull request Sep 18, 2024
Open
5 tasks
@sandreim
remove unused
1ef7952 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
fix prdoc
5790b8e 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
more tests cases
ba9d3ff 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
alindima
alindima approved these changes Sep 19, 2024
View reviewed changes
Copy link
Contributor

@alindima alindima left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

ordian
ordian reviewed Sep 19, 2024
View reviewed changes
Copy link
Member

@ordian ordian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good overall!
Left some questions for my understanding and potential corner-cases

polkadot/primitives/src/vstaging/mod.rs Show resolved Hide resolved
polkadot/runtime/parachains/src/inclusion/mod.rs Show resolved Hide resolved
polkadot/runtime/parachains/src/shared.rs Show resolved Hide resolved
polkadot/primitives/src/vstaging/mod.rs Outdated Show resolved Hide resolved
sandreim and others added 3 commits September 23, 2024 13:02
@sandreim
stricter UMP signal checks and tests
9c4e2ae 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
type alias
43bbb9d 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
Merge branch 'master' into sandreim/runtime_v2_descriptor_support
d7e57fd
ordian
ordian reviewed Sep 30, 2024
View reviewed changes
Copy link
Member

@ordian ordian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ready to approve once the comments are addressed

polkadot/runtime/parachains/src/inclusion/mod.rs Outdated Show resolved Hide resolved
polkadot/runtime/parachains/src/inclusion/mod.rs Outdated Show resolved Hide resolved
polkadot/runtime/parachains/src/inclusion/mod.rs Outdated
@@ -935,6 +946,27 @@ impl<T: Config> Pallet<T> {
para: ParaId,
upward_messages: &[UpwardMessage],
) -> Result<(), UmpAcceptanceCheckErr> {
// Filter any pending UMP signals and the separator.
let upward_messages = if let Some(separator_index) =
upward_messages.iter().rposition(|message| message.is_empty())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we still don't check here for multiple separators, as rposition will only find the last separator. I'd probably go for position instead

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed to position.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is position better here? IIUC we still don't check for multiple separators

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We check that the length of ump signals together with a separator is 2. It is possible that the signal is a separator because we don't attempt to deserialize it.
Ideally, we'd have a single piece of logic that does validation and ump signal extraction and share it with selected_core. But this looks good enough already.

polkadot/runtime/parachains/src/shared.rs Show resolved Hide resolved
@alindima alindima mentioned this pull request Oct 1, 2024
Open
@sandreim
missing feedback
7a1f382 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
Merge remote-tracking branch 'origin' into sandreim/runtime_v2_descri…
4757dab 
…ptor_support
@sandreim
Merge branch 'sandreim/runtime_v2_descriptor_support' of github.com:p…
a2a0795 
…aritytech/polkadot-sdk into sandreim/runtime_v2_descriptor_support
@sandreim
Copy link
Contributor Author

sandreim commented Oct 2, 2024

Ready to approve once the comments are addressed

Fixed, PTAL

ordian
ordian approved these changes Oct 2, 2024
View reviewed changes
Copy link
Member

@ordian ordian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@sandreim
🤦
01ce087 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
Merge branch 'master' of github.com:paritytech/polkadot-sdk into sand…
c0b36b1 
…reim/runtime_v2_descriptor_support
@sandreim sandreim added this pull request to the merge queue Oct 2, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Oct 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ordian ordian ordian approved these changes

@alindima alindima alindima approved these changes

Assignees
No one assigned
Labels
T8-polkadot This PR/Issue is related to/affects the Polkadot network.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Elastic scaling: relay chain support for new candidate receipts
4 participants
@sandreim @alindima @paritytech-cicd-pr @ordian