Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

indexer alt restorer: init commit and restore object infos #20507

Merged
merged 7 commits into from
Dec 23, 2024

Conversation

gegaowp
Copy link
Contributor

@gegaowp gegaowp commented Dec 4, 2024

Description

title, the main parts are:

  • read archival storage for checkpoint info of seq number and chain identifier
  • read snapshot files and restore obj_info table

Test plan

local run

cargo run --bin sui-indexer-alt-restorer -- --start-epoch 600 --endpoint "https://formal-snapshot.mainnet.sui.io" --snapshot-bucket "mysten-mainnet-formal" --archive-bucket "mysten-mainnet-archives" --snapshot-local-dir "/Users/gegao/Desktop/formal" --database-url "postgres://postgres:postgres@localhost/gegao"

it's worth doing some data integrity check eventually and not done yet.


Release notes

Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.

For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates.

  • Protocol:
  • Nodes (Validators and Full nodes):
  • Indexer:
  • JSON-RPC:
  • GraphQL:
  • CLI:
  • Rust SDK:
  • REST API:

Copy link

vercel bot commented Dec 4, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
sui-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Dec 23, 2024 6:48pm
3 Skipped Deployments
Name Status Preview Comments Updated (UTC)
multisig-toolkit ⬜️ Ignored (Inspect) Visit Preview Dec 23, 2024 6:48pm
sui-kiosk ⬜️ Ignored (Inspect) Visit Preview Dec 23, 2024 6:48pm
sui-typescript-docs ⬜️ Ignored (Inspect) Visit Preview Dec 23, 2024 6:48pm

@gegaowp gegaowp temporarily deployed to sui-typescript-aws-kms-test-env December 4, 2024 20:29 — with GitHub Actions Inactive
@gegaowp gegaowp changed the base branch from main to idx-restore-reuse December 4, 2024 20:30
@gegaowp gegaowp requested review from pchrysochoidis and mystieanwaya and removed request for a team December 4, 2024 20:30
@gegaowp gegaowp force-pushed the indexer-restore-init branch from fdbecad to a06e76e Compare December 4, 2024 20:32
@gegaowp gegaowp temporarily deployed to sui-typescript-aws-kms-test-env December 4, 2024 20:32 — with GitHub Actions Inactive
@gegaowp gegaowp requested review from lxfind and bmwill and removed request for a team, stefan-mysten, suiwombat, pchrysochoidis and mystieanwaya December 4, 2024 20:33
@gegaowp gegaowp force-pushed the indexer-restore-init branch from a06e76e to 4981347 Compare December 4, 2024 20:40
Copy link
Member

@amnn amnn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can still get rid of the kv_genesis logic, and there's a small simplification for the inner loop over live objects, but this is much easier to follow now, thanks @gegaowp ! Good to go 🚀

crates/sui-indexer-alt-restorer/src/lib.rs Outdated Show resolved Hide resolved
crates/sui-indexer-alt-restorer/src/lib.rs Outdated Show resolved Hide resolved
crates/sui-indexer-alt-restorer/src/snapshot.rs Outdated Show resolved Hide resolved
crates/sui-indexer-alt-restorer/src/snapshot.rs Outdated Show resolved Hide resolved
crates/sui-indexer-alt-schema/src/objects.rs Show resolved Hide resolved
@gegaowp gegaowp temporarily deployed to sui-typescript-aws-kms-test-env December 23, 2024 18:44 — with GitHub Actions Inactive
@gegaowp gegaowp merged commit e287607 into MystenLabs:main Dec 23, 2024
47 of 48 checks passed
@gegaowp gegaowp deleted the indexer-restore-init branch December 23, 2024 19:44
@gegaowp gegaowp changed the title indexer restorer: init commit and restore object infos indexer alt restorer: init commit and restore object infos Dec 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants