Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[stateless_validation] Load memtrie async on catchup #10982

Closed
Tracked by #46
staffik opened this issue Apr 8, 2024 · 0 comments
Closed
Tracked by #46

[stateless_validation] Load memtrie async on catchup #10982

staffik opened this issue Apr 8, 2024 · 0 comments
Assignees
@staffik
Labels
A-stateless-validation Area: stateless validation T-core Team: issues relevant to the core team

Comments

@staffik
Copy link
Contributor

staffik commented Apr 8, 2024

Follow-up to #10564.
Memtrie loading on catch-up must happen asynchronously.
@staffik staffik added T-core Team: issues relevant to the core team A-stateless-validation Area: stateless validation labels Apr 8, 2024
@staffik staffik self-assigned this Apr 8, 2024
This was referenced Apr 8, 2024
Open
Merged
@walnut-the-cat walnut-the-cat changed the title [stateless_validation] Load memtrie async on catchup [[stateless_validation] Load memtrie async on catchup Apr 10, 2024
@walnut-the-cat walnut-the-cat changed the title [[stateless_validation] Load memtrie async on catchup [P2][stateless_validation] Load memtrie async on catchup Apr 10, 2024
@staffik staffik changed the title [P2][stateless_validation] Load memtrie async on catchup [stateless_validation] Load memtrie async on catchup Apr 12, 2024
github-merge-queue bot pushed a commit that referenced this issue Apr 15, 2024
@staffik
[stateless_validation] Load memtrie async on catchup (#10983)
579c53c 
**Context**
Issue: #10982
Follow up to: #10820.

Modifies StateSync state machine so that memtrie load happens
asynchronously on catchup.

**Summary**
* Split `chain.set_state_finalize()` into:
  * `create_flat_storage_for_shard()`
  * `schedule_load_memtrie()`
  * actual `set_state_finalize()`
* ^ we need it because creating flat storage and state finalize requires
`chain` which cannot be passed in a message to the separate thread.
* Code to trigger memtrie load in a separate thread, analogously to how
apply state parts is done.
* Modify shard sync stages:
  * `StateDownloadScheduling` --> `StateApplyScheduling`
* Just changed the name as it was confusing. What happens there is
scheduling applying of state parts.
  * `StateDownloadApplying` --> `StateApplyComplete`
* What it actually did before was initializing flat storage and
finalizing state update after state apply from previous stage.
* Now it only initializes flat storage and schedules memtrie loading.
  * `StateDownloadComplete` --> `StateApplyFinalizing`
    * Before it was just deciding what next stage to transit into.
* Now it also contains the finalizing state update logic that was
previously in the previous stage.

Integration tests are to be done as a part of:
#10844.
@staffik staffik closed this as completed Apr 15, 2024
@github-actions github-actions bot mentioned this issue May 1, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@staffik staffik

Labels
A-stateless-validation Area: stateless validation T-core Team: issues relevant to the core team
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@staffik