Use a sparse array in DecryptNotesResponse
#5128
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
andiflabs
force-pushed
the
andrea/decrypt-notes-response-sparse-array
branch
from
d47ad02 to
9aa2688
Compare
andiflabs
force-pushed
the
andrea/worker-pool-transfer-and-shared-mem
branch
from
6f3dd00 to
148f41b
Compare
andiflabs
force-pushed
the
andrea/decrypt-notes-response-sparse-array
branch
from
9aa2688 to
d29c962
Compare
andiflabs
force-pushed
the
andrea/worker-pool-transfer-and-shared-mem
branch
from
148f41b to
8e8a2b7
Compare
andiflabs
force-pushed
the
andrea/decrypt-notes-response-sparse-array
branch
from
d29c962 to
b4fe7d0
Compare
andiflabs
force-pushed
the
andrea/worker-pool-transfer-and-shared-mem
branch
from
8e8a2b7 to
f280136
Compare
andiflabs
force-pushed
the
andrea/decrypt-notes-response-sparse-array
branch
from
b4fe7d0 to
720b052
Compare
andiflabs
force-pushed
the
andrea/worker-pool-transfer-and-shared-mem
branch
from
f280136 to
abc46be
Compare
hughy
approved these changes
Jul 11, 2024
andiflabs
force-pushed
the
andrea/worker-pool-transfer-and-shared-mem
branch
from
abc46be to
1dcff5a
Compare
andiflabs
force-pushed
the
andrea/decrypt-notes-response-sparse-array
branch
from
720b052 to
403f126
Compare
andiflabs
force-pushed
the
andrea/worker-pool-transfer-and-shared-mem
branch
from
1dcff5a to
930aa7f
Compare
andiflabs
force-pushed
the
andrea/decrypt-notes-response-sparse-array
branch
from
403f126 to
f8763fc
Compare
Base automatically changed from
andrea/worker-pool-transfer-and-shared-mem
to
staging
July 11, 2024 23:15
andiflabs
force-pushed
the
andrea/decrypt-notes-response-sparse-array
branch
from
f8763fc to
8c4ec51
Compare
Most of the note decryption will fail. Currently, if a `DecryptNotesResponse` contains 100 undecrypted notes, it will create a dense array containing 100 `null` values, causing unnecessary large memory allocations and large memory writes. It was in fact observed that the arrays from `DecryptNotesResponse` were one of the top contributors to memory usage during wallet scans. This commit changes the implementation so that now `DecryptNotesResponse` would create a sparse array containing no values at all. This means that, when no notes are decrypted (i.e. the most common case), all `DecryptNotesResponse` objects consume the same amount of memory, regardless of how many accounts are in the wallet, or how many encrypted notes were requested. This was shown to reduce the peak memory usage of the node process, as well as reduce the number of minor page faults.
andiflabs
force-pushed
the
andrea/decrypt-notes-response-sparse-array
branch
from
8c4ec51 to
dad63ba
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Most of the note decryption will fail. Currently, if a
DecryptNotesResponsecontains 100 undecrypted notes, it will create a dense array containing 100nullvalues, causing unnecessary large memory allocations and large memory writes. It was in fact observed that the arrays fromDecryptNotesResponsewere one of the top contributors to memory usage during wallet scans.This commit changes the implementation so that now
DecryptNotesResponsewould create a sparse array containing no values at all. This means that, when no notes are decrypted (i.e. the most common case), allDecryptNotesResponseobjects consume the same amount of memory, regardless of how many accounts are in the wallet, or how many encrypted notes were requested.This was shown to reduce the peak memory usage of the node process, as well as reduce the number of minor page faults.
Testing Plan
Documentation
N/A
Breaking Change
N/A