Add a few counters for BlockSTM size #12581
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
igor-aptos
requested review from
gelash,
runtian-zhou,
sasha8,
danielxiangzl,
zekun000 and
georgemitenkov
as code owners
georgemitenkov
approved these changes
Mar 19, 2024
| @@ -244,6 +244,13 @@ impl DelayedFieldValue { | |||
| }, | |||
| }) | |||
| } | |||
|
|
|||
| pub fn get_approx_memory_size(&self) -> usize { | |||
There was a problem hiding this comment.
Why not spell fully, get_approximate_memory_size?
|
|
||
| pub fn get_approx_memory_size(&self) -> usize { | ||
| match &self { | ||
| DelayedFieldValue::Aggregator(_) | DelayedFieldValue::Snapshot(_) => 1 + 16, |
There was a problem hiding this comment.
Can you please add an explantation here? If I see this code for the first time, comment would be super helpful to explain what this even mean.
| pub static BLOCK_VIEW_BASE_VALUES_MEMORY_USAGE: Lazy<HistogramVec> = Lazy::new(|| { | ||
| register_avg_counter_vec( | ||
| "aptos_execution_block_view_base_values_memory_usage", | ||
| "Size (number of keys) ", |
There was a problem hiding this comment.
This line is wrong? Should be values?
| name, | ||
| desc, | ||
| labels, | ||
| // We need to have at least one bucket in histogram, otherwise default buckets are used |
There was a problem hiding this comment.
nit: . in the end of a comment
| next_idx_to_commit: AtomicTxnIndex, | ||
| next_idx_to_commit: CachePadded<AtomicTxnIndex>, | ||
|
|
||
| total_base_value_size: CachePadded<AtomicU64>, |
There was a problem hiding this comment.
You make this cache padded to ensure so that both counters are not colocated on the same cache line, and we avoid invalidations? But what about values and next_idx_to_commit. Is the effect actually observable here?
There was a problem hiding this comment.
Let's sync up with @gelash when he is back. for now this makes new counter not conflict, doesn't improve on existing conflicts
| .entry(id) | ||
| .or_insert(VersionedValue::new(Some(base_value))); | ||
| use dashmap::mapref::entry::Entry::*; | ||
| match self.values.entry(id) { |
There was a problem hiding this comment.
nit: if let Vacant(...) = ... { ... }? Or even or_insert_with actually
| @@ -1272,6 +1275,8 @@ where | |||
|
|
|||
| ret.resize_with(num_txns, E::Output::skip_output); | |||
|
|
|||
| counters::update_state_counters(unsync_map.stats(), true); | |||
There was a problem hiding this comment.
This value should be false, right? Because it is sequential?
igor-aptos
added
the
CICD:run-execution-performance-test
igor-aptos
force-pushed
the
igor/block_stm_mem_counters
branch
from
6203413 to
b9bf67b
Compare
igor-aptos
force-pushed
the
igor/block_stm_mem_counters
branch
from
b9bf67b to
c807df6
Compare
zekun000
approved these changes
Mar 19, 2024
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
✅ Forge suite
|
✅ Forge suite
|
igor-aptos
added a commit
that referenced
this pull request
Mar 19, 2024
igor-aptos
added a commit
that referenced
this pull request
Mar 21, 2024
#12581) (#12594) * Optimize BlockSTM memory usage for delayed fields (#12580) This reduces memory footprint of amortized usage of delayed field with a single version from 3000 bytes to 600 bytes per delayed field. Basically BTreeMap stores things in vectors of 5, and size of the value is large. This invalidates BTreeMap's cache optimization of storing them all together. * Limit number of DelayedFields (aggregators) per resource temporarily, until gas charges for aggregator loading are implemented, limit number of aggregators per resource * Add a few counters for BlockSTM size (#12581) * update TOO_MANY_DELAYED_FIELDS error code (#12603) ---------
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Type of Change
Which Components or Systems Does This Change Impact?
How Has This Been Tested?
run forge tests, and looked at the counters
Key Areas to Review
Checklist