feat: make wallet db calls synchronous rather than async #3982
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hansieodendaal
changed the title
base_layer/wallet/src/output_manager_service/storage/database/mod.rs
Outdated
Show resolved
Hide resolved
Removed spawn blocking calls for db operations from the wallet in the output manager service. (This is the first PR in a couple of PRs required to implement this fully throughout the wallet code.)
hansieodendaal
force-pushed
the
ho_remove_spawn_bocking_calls_from_wallet_db
branch
from
01aeb59 to
5475a3a
Compare
stringhandler
approved these changes
Mar 31, 2022
stringhandler
pushed a commit
that referenced
this pull request
Aug 30, 2022
#4564) Description --- Removed spawn blocking calls for db operations from the wallet in the key manager service. (This is another PR in a couple of PRs required to implement this fully throughout the wallet code.) Motivation and Context --- As per #3982 and #4555 How Has This Been Tested? --- Unit tests Cucumber tests
stringhandler
pushed a commit
that referenced
this pull request
Aug 31, 2022
…4575) Description --- - Removed spawn blocking calls for db operations from the wallet in the contacts service. (This is another PR in a couple of PRs required to implement this fully throughout the wallet code.) - Reset the wallet's default db connection pool size back to 16 (from 5). Motivation and Context --- As per #3982 and #4555 How Has This Been Tested? --- Unit tests Cucumber tests System-level test
jorgeantonio21
pushed a commit
to jorgeantonio21/tari
that referenced
this pull request
Aug 31, 2022
…ari-project#4575) Description --- - Removed spawn blocking calls for db operations from the wallet in the contacts service. (This is another PR in a couple of PRs required to implement this fully throughout the wallet code.) - Reset the wallet's default db connection pool size back to 16 (from 5). Motivation and Context --- As per tari-project#3982 and tari-project#4555 How Has This Been Tested? --- Unit tests Cucumber tests System-level test
stringhandler
pushed a commit
that referenced
this pull request
Aug 31, 2022
Description --- Removed spawn blocking calls for db operations from the wallet in the wallet storage. (This is another PR in a couple of PRs required to implement this fully throughout the wallet code.) Motivation and Context --- As per #3982 and #4555 How Has This Been Tested? --- Unit tests Cucumber tests
|
A note after the fact. I was curious to see how removing spawn blocking calls (with the assumption that they would become non-blocking) would help with locks, so I looked at this PR a little more closely. Turns out that it's not what this PR does. Rather it's changing the DB requests from async to synchronous calls. So it's even more blocking than before :) (but likely would help with locks). I'm just updating the PR title to reflect this, to help identify the root purpose of the PR |
CjS77
changed the title
|
Personally, I would be more confused by the new title. I would have been happy with |
sdbondi
pushed a commit
that referenced
this pull request
Sep 2, 2022
…ing (transaction service) (#4597) Description --- Removed spawn blocking calls for db operations from the wallet in the transaction service. (This is the last PR in a couple of PRs required to implement this fully throughout the wallet code.) Motivation and Context --- As per #3982 and #4555 How Has This Been Tested? --- Unit tests Cucumber tests
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Removed spawn blocking calls for db operations from the wallet in the output manager service. (This is the first PR in a couple of PRs required to implement this fully throughout the wallet code.)
We are using SQLite in WAL mode, which provides concurrent read access and single thread write access for the number of pooled connections configured. Implementing spawn blocking calls on top of that for every db operation is counter productive and easily results in interlock situations within the wallet code, as is the case currently with sending transactions in batch mode.
Although batch mode transactions still does not work with only this PR, it showed a definite improvement monitoring the action with tokio console; all interlocks are now within the transaction service and not in the output manager service anymore.
Motivation and Context
When sending multiple transactions in batch mode (with for example the
make-it-raincommand), the wallet locks up and no transactions are being sent.How Has This Been Tested?
System level tests sending multiple transactions in batch mode to three receiving wallets.
Monitoring with tokio console.