Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Elastic scaling: add e2e test #3929

Merged
merged 27 commits into from
Apr 5, 2024
Merged

Elastic scaling: add e2e test #3929

merged 27 commits into from
Apr 5, 2024

Conversation

sandreim
Copy link
Contributor

@sandreim sandreim commented Apr 1, 2024

On top of #3879

I've also moved the previous test where we ensure multiple cores per para doesn't break non elastic parachains.

@sandreim
refactor: move claim queue fetch utils to utils crate
e3b5192 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
cache all groups per para and remove para:group 1:1 assumption
01ceb3b 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
Merge branch 'master' of github.com:paritytech/polkadot-sdk into sand…
5ed7841 
…reim/elastic_statement_dist
@sandreim
typo
5f0872f 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
fix test build
034b61a 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
testing some local test run weirdness
9c91193 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
fix doctest
f87a624 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
remove panic
70213c5 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
add new cluster test
4c66001 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
add grid test
21ae63d 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
Merge branch 'master' of github.com:paritytech/polkadot-sdk into sand…
e923dac 
…reim/elastic_statement_dist

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
fmt
27ee145 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
explicit max_candidate_depth
be1a2c7 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
fix test comment
b9f1afc 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
Add elastic scaling zombienet test
6340e32 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
js script
8495e6f 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
move mvp test
04dc5fd 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
fix cmd
bd431cb 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim sandreim added R0-silent Changes should not be mentioned in any release notes T10-tests This PR/Issue is related to tests. labels Apr 1, 2024
@sandreim sandreim requested a review from a team as a code owner April 1, 2024 14:52
@sandreim
fix path
1713be1 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
alindima
alindima reviewed Apr 2, 2024
View reviewed changes
Copy link
Contributor

@alindima alindima left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you also make some changes to the adder-collator for this test to pass?

polkadot/zombienet_tests/elastic_scaling/0001-basic-3cores-6s-blocks.toml Show resolved Hide resolved
polkadot/zombienet_tests/elastic_scaling/0001-basic-3cores-6s-blocks.zndsl Show resolved Hide resolved
@sandreim
Copy link
Contributor Author

sandreim commented Apr 2, 2024

Did you also make some changes to the adder-collator for this test to pass?

No direct changes were needed as we just define the collator_fn which is used by collation-generation that has been already upgraded to support elastic scaling

@sandreim
rename mvp test
14fcb4b 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
proper js args
07b58a3 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
non elast scaling para assert
2b2d2ab 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
some tolerance
332fda9 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
properly call assign-core.js
86c22a4 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Base automatically changed from sandreim/elastic_statement_dist to master April 3, 2024 08:57
@sandreim
Merge branch 'master' of github.com:paritytech/polkadot-sdk into sand…
a32c2c7 
…reim/elastic_zombies

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
minor script change
7bde9d8 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
Copy link
Contributor Author

sandreim commented Apr 4, 2024

Seems like deployment error: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5751023

Error: Timeout(0) for: waitLog(): pod: temp-collator, container: temp-collator, log: waiting for copy files to finish

CC @pepoviola

@pepoviola
Copy link
Contributor

Seems like deployment error: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5751023

Error: Timeout(0) for: waitLog(): pod: temp-collator, container: temp-collator, log: waiting for copy files to finish

CC @pepoviola

Fixed in a4531235d2808a8a614d96df9bf62a31ec283da1. The config image was the problem.

Thx!

@sandreim
Copy link
Contributor Author

sandreim commented Apr 5, 2024

Seems like deployment error: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5751023
Error: Timeout(0) for: waitLog(): pod: temp-collator, container: temp-collator, log: waiting for copy files to finish
CC @pepoviola

Fixed in a4531235d2808a8a614d96df9bf62a31ec283da1. The config image was the problem.

Thx!

thank you @pepoviola I totally missed it.

eskimor
eskimor approved these changes Apr 5, 2024
View reviewed changes
Copy link
Member

@eskimor eskimor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I read this right, that we are doing elastic scaling with 3 collators here instead of just one connecting to multiple backing groups? I guess that works with the adder collator. Do we also support one collator collating on 3 cores?

@sandreim
Copy link
Contributor Author

sandreim commented Apr 5, 2024

Do I read this right, that we are doing elastic scaling with 3 collators here instead of just one connecting to multiple backing groups? I guess that works with the adder collator. Do we also support one collator collating on 3 cores?

Just single collator case, the test has 2 parachains each with one collator:

  • 2000 has 3 cores assigned
  • 2001 has 1 core assigned

@sandreim sandreim added this pull request to the merge queue Apr 5, 2024
Merged via the queue into master with commit 0dc0d40 Apr 5, 2024
133 of 135 checks passed
@sandreim sandreim deleted the sandreim/elastic_zombies branch April 5, 2024 10:09
Ank4n pushed a commit that referenced this pull request Apr 9, 2024
@sandreim @pepoviola @Ank4n
Elastic scaling: add e2e test (#3929)
bfd7ae2 
On top of #3879

I've also moved the previous test where we ensure multiple cores per
para doesn't break non elastic parachains.

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: Javier Viola <363911+pepoviola@users.noreply.github.com>
dharjeezy pushed a commit to dharjeezy/polkadot-sdk that referenced this pull request Apr 9, 2024
@sandreim @pepoviola @dharjeezy
Elastic scaling: add e2e test (paritytech#3929)
954ea07 
On top of paritytech#3879

I've also moved the previous test where we ensure multiple cores per
para doesn't break non elastic parachains.

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: Javier Viola <363911+pepoviola@users.noreply.github.com>
This was referenced Jun 5, 2024
Closed
Closed
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alindima alindima alindima left review comments

@pepoviola pepoviola pepoviola left review comments

@eskimor eskimor eskimor approved these changes

@tdimitrov tdimitrov tdimitrov approved these changes

Assignees
No one assigned
Labels
R0-silent Changes should not be mentioned in any release notes T10-tests This PR/Issue is related to tests.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@sandreim @pepoviola @tdimitrov @eskimor @alindima