Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add directory paths to Snapshotter and SnapshotProvider #5283

Merged
merged 29 commits into from
Nov 14, 2023

Conversation

joshieDo
Copy link
Collaborator

@joshieDo joshieDo commented Nov 3, 2023

PR into #5249

As title suggests.

Additionally:

  • Add some strum::EnumString for some primitive types.
  • update_highest_snapshots_tracker will look into the directory and find the highest block number for each segment type.
  • default name of a snapshot no longer has its configuration. Only the segment type and inclusive range.

@joshieDo joshieDo added C-enhancement New feature or request A-static-files Related to static files labels Nov 3, 2023
bin/reth/src/db/snapshots/headers.rs Outdated Show resolved Hide resolved
crates/primitives/src/snapshot/segment.rs Outdated Show resolved Hide resolved
range.end(),
)
.into()
// ATTENTION: if changing the name format, be sure to reflect those changes in
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we add a test to check that the result of filename_with_configuration can be parsed by parse_filename?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added on #5373

crates/snapshot/src/snapshotter.rs Outdated Show resolved Hide resolved
crates/snapshot/src/snapshotter.rs Outdated Show resolved Hide resolved
crates/snapshot/src/snapshotter.rs Outdated Show resolved Hide resolved
crates/snapshot/src/snapshotter.rs Show resolved Hide resolved
crates/snapshot/src/snapshotter.rs Show resolved Hide resolved
Comment on lines 84 to +91
pub fn with_snapshots(
mut self,
snapshots_path: PathBuf,
highest_snapshot_tracker: watch::Receiver<Option<HighestSnapshots>>,
) -> Self {
self.snapshot_provider = Some(Arc::new(
SnapshotProvider::default().with_highest_tracker(Some(highest_snapshot_tracker)),
SnapshotProvider::new(snapshots_path)
.with_highest_tracker(Some(highest_snapshot_tracker)),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these arguments became too verbose, can we just pass a Arc<SnapshotProvider> as an argument?

joshieDo and others added 5 commits November 13, 2023 11:40
Co-authored-by: Alexey Shekhirin <a.shekhirin@gmail.com>
Co-authored-by: Alexey Shekhirin <a.shekhirin@gmail.com>
Co-authored-by: Alexey Shekhirin <a.shekhirin@gmail.com>
Co-authored-by: Alexey Shekhirin <a.shekhirin@gmail.com>
Co-authored-by: Alexey Shekhirin <a.shekhirin@gmail.com>
Copy link
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion for rename

bin/reth/src/db/snapshots/headers.rs Outdated Show resolved Hide resolved
bin/reth/src/db/snapshots/headers.rs Outdated Show resolved Hide resolved
bin/reth/src/db/snapshots/receipts.rs Outdated Show resolved Hide resolved
Copy link
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pedantic nit

crates/primitives/src/fs.rs Outdated Show resolved Hide resolved
Comment on lines +88 to +90
snapshots_path: PathBuf,
chain_spec: Arc<ChainSpec>,
block_interval: u64,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

I'd like to move snapshots_path + block_interval to a new SnapshotterConfig struct

joshieDo and others added 2 commits November 14, 2023 15:43
Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>
Base automatically changed from joshie/db-snap-provider to main November 14, 2023 18:48
@joshieDo joshieDo added this pull request to the merge queue Nov 14, 2023
Merged via the queue into main with commit 7b781eb Nov 14, 2023
26 checks passed
@joshieDo joshieDo deleted the joshie/db-snap-provider2 branch November 14, 2023 21:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-static-files Related to static files C-enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants