Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: prefetch accounts and access keys #7590

Merged
merged 30 commits into from
Sep 14, 2022
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
bedd1c8
fix: rocksdb flush all column families
jakmeier Aug 26, 2022
11a9c4b
squashed prefetcher poc
jakmeier Aug 30, 2022
8e0cd00
limit scheduler queue and share io request queue
jakmeier Aug 30, 2022
c7d31be
enforce memory limits
jakmeier Aug 31, 2022
fb99226
move prefetching storage in separate module
jakmeier Aug 31, 2022
2ba42f9
wip
jakmeier Sep 2, 2022
a5c1755
Merge remote-tracking branch 'upstream/master' into prefetch-accounts
jakmeier Sep 7, 2022
f7514e0
add tests
jakmeier Sep 8, 2022
14bb8a2
also prefetch delayed receipts
jakmeier Sep 8, 2022
f4b53d5
simplify API and clean up for review
jakmeier Sep 8, 2022
344bb45
move test-only function
jakmeier Sep 8, 2022
8aeee82
Update runtime/runtime/src/prefetch.rs
jakmeier Sep 10, 2022
bf6d12b
apply reviewer suggested refactoring
jakmeier Sep 10, 2022
d57e094
simplify indentation
jakmeier Sep 10, 2022
931d566
fix get_and_set_if_empty memory limit check order
jakmeier Sep 10, 2022
d714200
add cargo.lock changes
jakmeier Sep 10, 2022
07bd456
more improvements from reviewer suggestions
jakmeier Sep 13, 2022
de99c15
handle case of parallel chunk execution
jakmeier Sep 13, 2022
5107eda
finish impl for keeping prefetcher alive
jakmeier Sep 13, 2022
09db2f2
address marcin's review comments
jakmeier Sep 13, 2022
de4f395
Don't treat non-existing keys as failure
jakmeier Sep 13, 2022
b6e9bc2
fix tests
jakmeier Sep 13, 2022
3844b15
improve comments
jakmeier Sep 13, 2022
8cccc75
resolve capacity leakage issues
jakmeier Sep 14, 2022
a4731d3
disable prefetching for view calls
jakmeier Sep 14, 2022
9a8935a
add comment
jakmeier Sep 14, 2022
261fa56
refactor: size tracked hash map
jakmeier Sep 14, 2022
e82f409
Merge remote-tracking branch 'upstream/master' into HEAD
jakmeier Sep 14, 2022
c4dad69
fix merge test conflicts
jakmeier Sep 14, 2022
faf7895
fix state viewer merge conflicts
jakmeier Sep 14, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions core/store/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,9 @@ use crate::db::{
pub use crate::trie::iterator::TrieIterator;
pub use crate::trie::update::{TrieUpdate, TrieUpdateIterator, TrieUpdateValuePtr};
pub use crate::trie::{
estimator, split_state, ApplyStatePartResult, KeyForStateChanges, PartialStorage, ShardTries,
Trie, TrieAccess, TrieCache, TrieCachingStorage, TrieChanges, TrieConfig, TrieStorage,
WrappedTrieChanges,
estimator, split_state, ApplyStatePartResult, KeyForStateChanges, PartialStorage, PrefetchApi,
ShardTries, Trie, TrieAccess, TrieCache, TrieCachingStorage, TrieChanges, TrieConfig,
TrieStorage, WrappedTrieChanges,
};

mod columns;
Expand Down
2 changes: 2 additions & 0 deletions core/store/src/trie/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ pub use crate::trie::config::TrieConfig;
use crate::trie::insert_delete::NodesStorage;
use crate::trie::iterator::TrieIterator;
use crate::trie::nibble_slice::NibbleSlice;
pub use crate::trie::prefetching_trie_storage::PrefetchApi;
pub use crate::trie::shard_tries::{KeyForStateChanges, ShardTries, WrappedTrieChanges};
pub use crate::trie::trie_storage::{TrieCache, TrieCachingStorage, TrieStorage};
use crate::trie::trie_storage::{TrieMemoryPartialStorage, TrieRecordingStorage};
Expand All @@ -29,6 +30,7 @@ mod config;
mod insert_delete;
pub mod iterator;
mod nibble_slice;
mod prefetching_trie_storage;
mod shard_tries;
pub mod split_state;
mod state_parts;
Expand Down
Loading