Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove the need to wait for target block header in warp sync implementation #5431

Merged

Conversation

nazar-pc
Copy link
Contributor

I'm not sure if this is exactly what #3537 meant, but I think it should be fine to wait for relay chain before initializing parachain node fully, which removed the need for background task and extra hacks throughout the stack just to know where warp sync should start.

Previously there were both WarpSyncParams and WarpSyncConfig, but there was no longer any point in having two data structures, so I simplified it to just WarpSyncConfig.

Fixes #3537

@bkchr bkchr added the T0-node This PR/Issue is related to the topic “node”. label Aug 21, 2024
Copy link
Contributor

@lexnv lexnv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Nicely done here!

From the testing perspective, did we encounter any issues while trying to sync a full node to kusama for example? We've used that reliably in the past to filter out extra edge-cases.

The code logic looks good to me, I would also wait for Dmitry for a review 🙏

Copy link
Contributor

@dmitry-markin dmitry-markin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is exactly what #3537 meant. Thank you!

@nazar-pc
Copy link
Contributor Author

I see no reason why it wouldn't work, but I didn't run it on Kusama myself. Can someone give it a try, please?

@dmitry-markin dmitry-markin added this pull request to the merge queue Aug 23, 2024
Merged via the queue into paritytech:master with commit 6d819a6 Aug 23, 2024
226 of 228 checks passed
@nazar-pc nazar-pc deleted the refactor-warp-sync-initialization branch August 23, 2024 12:51
github-merge-queue bot pushed a commit that referenced this pull request Sep 10, 2024
This is a step towards
#5333

The commits with code moving (but no other changes) and actual changes
are separated for easier review.

Essentially this results in `SyncingStrategy` trait replacing struct
(which is renamed to `PolkadotSyncingStrategy`, open for better name
suggestions). Technically it is already possible to replace
`PolkadotSyncingStrategy<B, Client>` with `Box<dyn SyncingStrategy<B>`
in syncing engine, but I decided to postpone such change until we
actually have an ability to customize it. It should also be possible to
swap `PolkadotSyncingStrategy` with just `ChainSync` that only supports
regular full sync from genesis (it also implements `SyncingStrategy`
trait).

While extracted trait still has a lot of non-generic stuff in it like
exposed knowledge of warp sync and `StrategyKey` with hardcoded set of
options, I believe this is a step in the right direction and will
address those in upcoming PRs.

With #5431 that landed
earlier warp sync configuration is more straightforward, but there are
still numerous things interleaved that will take some time to abstract
away nicely and expose in network config for developers. For now this is
an internal change even though data structures are technically public
and require major version bump.
mordamax pushed a commit to paritytech-stg/polkadot-sdk that referenced this pull request Sep 10, 2024
This is a step towards
paritytech#5333

The commits with code moving (but no other changes) and actual changes
are separated for easier review.

Essentially this results in `SyncingStrategy` trait replacing struct
(which is renamed to `PolkadotSyncingStrategy`, open for better name
suggestions). Technically it is already possible to replace
`PolkadotSyncingStrategy<B, Client>` with `Box<dyn SyncingStrategy<B>`
in syncing engine, but I decided to postpone such change until we
actually have an ability to customize it. It should also be possible to
swap `PolkadotSyncingStrategy` with just `ChainSync` that only supports
regular full sync from genesis (it also implements `SyncingStrategy`
trait).

While extracted trait still has a lot of non-generic stuff in it like
exposed knowledge of warp sync and `StrategyKey` with hardcoded set of
options, I believe this is a step in the right direction and will
address those in upcoming PRs.

With paritytech#5431 that landed
earlier warp sync configuration is more straightforward, but there are
still numerous things interleaved that will take some time to abstract
away nicely and expose in network config for developers. For now this is
an internal change even though data structures are technically public
and require major version bump.
mordamax pushed a commit to paritytech-stg/polkadot-sdk that referenced this pull request Sep 10, 2024
This is a step towards
paritytech#5333

The commits with code moving (but no other changes) and actual changes
are separated for easier review.

Essentially this results in `SyncingStrategy` trait replacing struct
(which is renamed to `PolkadotSyncingStrategy`, open for better name
suggestions). Technically it is already possible to replace
`PolkadotSyncingStrategy<B, Client>` with `Box<dyn SyncingStrategy<B>`
in syncing engine, but I decided to postpone such change until we
actually have an ability to customize it. It should also be possible to
swap `PolkadotSyncingStrategy` with just `ChainSync` that only supports
regular full sync from genesis (it also implements `SyncingStrategy`
trait).

While extracted trait still has a lot of non-generic stuff in it like
exposed knowledge of warp sync and `StrategyKey` with hardcoded set of
options, I believe this is a step in the right direction and will
address those in upcoming PRs.

With paritytech#5431 that landed
earlier warp sync configuration is more straightforward, but there are
still numerous things interleaved that will take some time to abstract
away nicely and expose in network config for developers. For now this is
an internal change even though data structures are technically public
and require major version bump.
mordamax pushed a commit to paritytech-stg/polkadot-sdk that referenced this pull request Sep 10, 2024
This is a step towards
paritytech#5333

The commits with code moving (but no other changes) and actual changes
are separated for easier review.

Essentially this results in `SyncingStrategy` trait replacing struct
(which is renamed to `PolkadotSyncingStrategy`, open for better name
suggestions). Technically it is already possible to replace
`PolkadotSyncingStrategy<B, Client>` with `Box<dyn SyncingStrategy<B>`
in syncing engine, but I decided to postpone such change until we
actually have an ability to customize it. It should also be possible to
swap `PolkadotSyncingStrategy` with just `ChainSync` that only supports
regular full sync from genesis (it also implements `SyncingStrategy`
trait).

While extracted trait still has a lot of non-generic stuff in it like
exposed knowledge of warp sync and `StrategyKey` with hardcoded set of
options, I believe this is a step in the right direction and will
address those in upcoming PRs.

With paritytech#5431 that landed
earlier warp sync configuration is more straightforward, but there are
still numerous things interleaved that will take some time to abstract
away nicely and expose in network config for developers. For now this is
an internal change even though data structures are technically public
and require major version bump.
mordamax pushed a commit to paritytech-stg/polkadot-sdk that referenced this pull request Sep 10, 2024
This is a step towards
paritytech#5333

The commits with code moving (but no other changes) and actual changes
are separated for easier review.

Essentially this results in `SyncingStrategy` trait replacing struct
(which is renamed to `PolkadotSyncingStrategy`, open for better name
suggestions). Technically it is already possible to replace
`PolkadotSyncingStrategy<B, Client>` with `Box<dyn SyncingStrategy<B>`
in syncing engine, but I decided to postpone such change until we
actually have an ability to customize it. It should also be possible to
swap `PolkadotSyncingStrategy` with just `ChainSync` that only supports
regular full sync from genesis (it also implements `SyncingStrategy`
trait).

While extracted trait still has a lot of non-generic stuff in it like
exposed knowledge of warp sync and `StrategyKey` with hardcoded set of
options, I believe this is a step in the right direction and will
address those in upcoming PRs.

With paritytech#5431 that landed
earlier warp sync configuration is more straightforward, but there are
still numerous things interleaved that will take some time to abstract
away nicely and expose in network config for developers. For now this is
an internal change even though data structures are technically public
and require major version bump.
mordamax pushed a commit to paritytech-stg/polkadot-sdk that referenced this pull request Sep 10, 2024
This is a step towards
paritytech#5333

The commits with code moving (but no other changes) and actual changes
are separated for easier review.

Essentially this results in `SyncingStrategy` trait replacing struct
(which is renamed to `PolkadotSyncingStrategy`, open for better name
suggestions). Technically it is already possible to replace
`PolkadotSyncingStrategy<B, Client>` with `Box<dyn SyncingStrategy<B>`
in syncing engine, but I decided to postpone such change until we
actually have an ability to customize it. It should also be possible to
swap `PolkadotSyncingStrategy` with just `ChainSync` that only supports
regular full sync from genesis (it also implements `SyncingStrategy`
trait).

While extracted trait still has a lot of non-generic stuff in it like
exposed knowledge of warp sync and `StrategyKey` with hardcoded set of
options, I believe this is a step in the right direction and will
address those in upcoming PRs.

With paritytech#5431 that landed
earlier warp sync configuration is more straightforward, but there are
still numerous things interleaved that will take some time to abstract
away nicely and expose in network config for developers. For now this is
an internal change even though data structures are technically public
and require major version bump.
mordamax pushed a commit to paritytech-stg/polkadot-sdk that referenced this pull request Sep 10, 2024
This is a step towards
paritytech#5333

The commits with code moving (but no other changes) and actual changes
are separated for easier review.

Essentially this results in `SyncingStrategy` trait replacing struct
(which is renamed to `PolkadotSyncingStrategy`, open for better name
suggestions). Technically it is already possible to replace
`PolkadotSyncingStrategy<B, Client>` with `Box<dyn SyncingStrategy<B>`
in syncing engine, but I decided to postpone such change until we
actually have an ability to customize it. It should also be possible to
swap `PolkadotSyncingStrategy` with just `ChainSync` that only supports
regular full sync from genesis (it also implements `SyncingStrategy`
trait).

While extracted trait still has a lot of non-generic stuff in it like
exposed knowledge of warp sync and `StrategyKey` with hardcoded set of
options, I believe this is a step in the right direction and will
address those in upcoming PRs.

With paritytech#5431 that landed
earlier warp sync configuration is more straightforward, but there are
still numerous things interleaved that will take some time to abstract
away nicely and expose in network config for developers. For now this is
an internal change even though data structures are technically public
and require major version bump.
mordamax pushed a commit to paritytech-stg/polkadot-sdk that referenced this pull request Sep 11, 2024
This is a step towards
paritytech#5333

The commits with code moving (but no other changes) and actual changes
are separated for easier review.

Essentially this results in `SyncingStrategy` trait replacing struct
(which is renamed to `PolkadotSyncingStrategy`, open for better name
suggestions). Technically it is already possible to replace
`PolkadotSyncingStrategy<B, Client>` with `Box<dyn SyncingStrategy<B>`
in syncing engine, but I decided to postpone such change until we
actually have an ability to customize it. It should also be possible to
swap `PolkadotSyncingStrategy` with just `ChainSync` that only supports
regular full sync from genesis (it also implements `SyncingStrategy`
trait).

While extracted trait still has a lot of non-generic stuff in it like
exposed knowledge of warp sync and `StrategyKey` with hardcoded set of
options, I believe this is a step in the right direction and will
address those in upcoming PRs.

With paritytech#5431 that landed
earlier warp sync configuration is more straightforward, but there are
still numerous things interleaved that will take some time to abstract
away nicely and expose in network config for developers. For now this is
an internal change even though data structures are technically public
and require major version bump.
nazar-pc added a commit to autonomys/polkadot-sdk that referenced this pull request Sep 27, 2024
This is a step towards
paritytech#5333

The commits with code moving (but no other changes) and actual changes
are separated for easier review.

Essentially this results in `SyncingStrategy` trait replacing struct
(which is renamed to `PolkadotSyncingStrategy`, open for better name
suggestions). Technically it is already possible to replace
`PolkadotSyncingStrategy<B, Client>` with `Box<dyn SyncingStrategy<B>`
in syncing engine, but I decided to postpone such change until we
actually have an ability to customize it. It should also be possible to
swap `PolkadotSyncingStrategy` with just `ChainSync` that only supports
regular full sync from genesis (it also implements `SyncingStrategy`
trait).

While extracted trait still has a lot of non-generic stuff in it like
exposed knowledge of warp sync and `StrategyKey` with hardcoded set of
options, I believe this is a step in the right direction and will
address those in upcoming PRs.

With paritytech#5431 that landed
earlier warp sync configuration is more straightforward, but there are
still numerous things interleaved that will take some time to abstract
away nicely and expose in network config for developers. For now this is
an internal change even though data structures are technically public
and require major version bump.

(cherry picked from commit 1f1f20a)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T0-node This PR/Issue is related to the topic “node”.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Pass warp sync target block to parachain node's SyncingEngine during initialization
4 participants