feat(trading-proto-upgrade): wasm DB, kickstart, refund states, v2 RPCs #2015

artemii235 · 2023-11-27T10:16:56Z

#1895

What's done:

Implemented WASM storage for upgraded swaps.
Implemented old swaps data migration.
Swaps kickstart on MM2 reload.
Swaps file lock, preventing the same swap from starting from different processes.
my_swap_status, my_recent_swaps V2 RPCs.
V1 RPCs can also return upgraded swaps data.
Added BoolAsInt struct because boolean can be used as IndexedDb indexes.
Implemented refund states for both swap types.

Important notes:

Seednodes should be updated to support/rebroadcast new swap protocol messages.

Test plan:

Prerequisites for all test cases:

Use only test coins or very small amounts of assets with real market value!
Run two nodes (seed and light) with "use_trading_proto_v2": true in MM2.conf.
Select coins to trade, both must have "protocol": { "type": "UTXO" }.
Initiate trading using setprice and buy/sell calls.
Each case should be checked under both native and WASM targets.

RPC

Start swaps.
"my_swap_status" and "my_recent_swaps" of both RPC versions must return swap data.

Kickstart

Start swaps.
Stop MM node.
Restart node.
coins_needed_for_kick_start must return tickers used in swap.
Reactivate coins.
Swap must kickstart and complete.

Coin deactivation prevention

Start swaps.
Try to disable coin used in active swap.
The RPC must return an error containing UUID of the ongoing swap.

Old swaps data migration

Use the node that already has old swaps in DB.
Backup database.
Run updated MM2.
Open MM2.db (SQLite), check that all completed swaps in "my_swaps" table have "is_finished:1".
Restart MM. Ensure that old completed swaps are not kickstarted.

…ration-4 # Conflicts: # mm2src/mm2_main/src/lp_swap/swap_wasm_db.rs

…ration-4

artemii235 · 2023-12-01T04:13:24Z

@shamardy I've fixed your review notes 🙂

shamardy

Thank you for the fixes! Next review Iteration!

shamardy · 2023-12-01T09:29:04Z

mm2src/mm2_main/src/lp_swap/maker_swap_v2.rs

+        loop {
+            match state_machine
+                .maker_coin
+                .can_refund_htlc(state_machine.maker_payment_locktime())


I think we should add the case of immediate refund using the taker refund secret here. I am sure you have this on your rader and plan to implement it in next sprints but this is just a reminder :)
Also, recover_funds_of_swap for swaps v2 is probably needed.

Yeah, I have this on radar, will do in the next iteration 🙂

mm2src/mm2_main/src/lp_swap/maker_swap_v2.rs

shamardy · 2023-12-01T18:04:05Z

mm2src/mm2_main/src/lp_swap/swap_v2_common.rs

+pub(super) fn init_additional_context_impl(ctx: &MmArc, swap_info: ActiveSwapV2Info) {
+    subscribe_to_topic(ctx, swap_v2_topic(&swap_info.uuid));
+    let swap_ctx = SwapsContext::from_ctx(ctx).expect("SwapsContext::from_ctx should not fail");
+    swap_ctx.init_msg_v2_store(swap_info.uuid, bits256::default());


I am just wondering why was accept_only_from needed in legacy swap implementation?

This part is not finished yet: we should limit processing of the swap messages only to the node, with which we matched our order.

shamardy · 2023-12-01T18:07:50Z

mm2src/mm2_main/src/lp_swap/maker_swap_v2.rs

+            maker_coin_swap_contract: None,
+            taker_coin_swap_contract: None,


These are None here but retrieved in the mirror implementation for taker swap similar to p2p_keypair.

These are placeholders for the ETH/ERC-20 support in upgraded swaps.

…ration-4

shamardy

🔥

shamardy · 2023-12-05T18:06:22Z

@borngraced did you finish review for this PR?

borngraced · 2023-12-06T09:34:30Z

mm2src/mm2_main/src/lp_swap/saved_swap.rs

+            .reverse()
+            .open_cursor("migration")
+            .await?
+            // TODO refactor when "closure invoked recursively or after being dropped" is fixed


we started working on this already.
#2019

borngraced

great work 🔥

shamardy · 2023-12-06T19:47:11Z

@artemii235 I will merge this PR to dev once 2.0.0-beta is released.

borngraced · 2023-12-10T14:32:21Z

mm2src/mm2_main/src/lp_swap/saved_swap.rs

+    pub(super) async fn get_current_migration(
+        migration_table: &DbTable<'_, SwapsMigrationTable>,
+    ) -> MmResult<u32, SavedSwapError> {
+        let migrations = migration_table
+            .cursor_builder()
+            .bound("migration", 0, u32::MAX)
+            .reverse()
+            .open_cursor("migration")
+            .await?
+            // TODO refactor when "closure invoked recursively or after being dropped" is fixed
+            .collect()
+            .await?;
+
+        Ok(migrations.first().map(|(_, m)| m.migration).unwrap_or_default())


I have a fix for for this issue now..So I think this pr can be hold till #2028 is merged then you can query first directly instead of collecting all.

@artemii235 @shamardy

Perfect, thanks!

…ration-4 # Conflicts: # mm2src/mm2_main/src/lp_native_dex.rs

ca333

secure code reviewed

* dev: (22 commits) chore(config): remove vscode launchjson (KomodoPlatform#2040) feat(trading-proto-upgrade): wasm DB, kickstart, refund states, v2 RPCs (KomodoPlatform#2015) feat(UTXO): balance event streaming for Electrum clients (KomodoPlatform#2013) feat(tx): add new sign_raw_transaction rpc for UTXO and EVM coins (KomodoPlatform#1930) fix(p2p): handle encode_and_sign errors (KomodoPlatform#2038) chore(release): add changelog entries for v2.0.0-beta (KomodoPlatform#2037) chore(network): write network information to stdout (KomodoPlatform#2034) fix(price_endpoints): add cached url (KomodoPlatform#2032) deps(network): sync with upstream yamux (KomodoPlatform#2030) fix(config): accept a string as rpcport value (KomodoPlatform#2026) feat(nft): move db lock, add tx fee and confirmations (KomodoPlatform#1989) chore(network): update seednodes for netid 8762 (KomodoPlatform#2024) chore(network): add todo on peer storage behaviour (KomodoPlatform#2025) chore(network): exclude `168.119.236.249` from the seednode list (KomodoPlatform#2021) feat(network): deprecate 7777 network (KomodoPlatform#2020) chore(release): bump mm2 version to 2.0.0-beta (KomodoPlatform#2018) feat(UTXO swaps): kmd burn plan impl (KomodoPlatform#2006) chore(docs): fix the link to simple market maker in README.md (KomodoPlatform#2011) refactor(cli): cli dependency updates and warn on bad config perm (KomodoPlatform#1956) chore(containers and docs): update docs and container images (KomodoPlatform#2003) ... # Conflicts: # mm2src/mm2_main/tests/mm2_tests/mm2_tests_inner.rs # mm2src/mm2_test_helpers/src/for_tests.rs

* dev: (24 commits) chore(release): bump mm2 version to 2.1.0-beta (KomodoPlatform#2044) feat(trezor): add segwit support for withdraw with trezor (KomodoPlatform#1984) chore(config): remove vscode launchjson (KomodoPlatform#2040) feat(trading-proto-upgrade): wasm DB, kickstart, refund states, v2 RPCs (KomodoPlatform#2015) feat(UTXO): balance event streaming for Electrum clients (KomodoPlatform#2013) feat(tx): add new sign_raw_transaction rpc for UTXO and EVM coins (KomodoPlatform#1930) fix(p2p): handle encode_and_sign errors (KomodoPlatform#2038) chore(release): add changelog entries for v2.0.0-beta (KomodoPlatform#2037) chore(network): write network information to stdout (KomodoPlatform#2034) fix(price_endpoints): add cached url (KomodoPlatform#2032) deps(network): sync with upstream yamux (KomodoPlatform#2030) fix(config): accept a string as rpcport value (KomodoPlatform#2026) feat(nft): move db lock, add tx fee and confirmations (KomodoPlatform#1989) chore(network): update seednodes for netid 8762 (KomodoPlatform#2024) chore(network): add todo on peer storage behaviour (KomodoPlatform#2025) chore(network): exclude `168.119.236.249` from the seednode list (KomodoPlatform#2021) feat(network): deprecate 7777 network (KomodoPlatform#2020) chore(release): bump mm2 version to 2.0.0-beta (KomodoPlatform#2018) feat(UTXO swaps): kmd burn plan impl (KomodoPlatform#2006) chore(docs): fix the link to simple market maker in README.md (KomodoPlatform#2011) ...

* evm-hd-wallet: (27 commits) Fix todo comments Fix HDAddressOps::Address trait bounds fix(indexeddb): fix IDB cursor.continue_() call after drop (KomodoPlatform#2028) security bump for `h2` (KomodoPlatform#2062) fix(makerbot): allow more than one prices url in makerbot (KomodoPlatform#2027) fix(wasm worker env): refactor direct usage of `window` (KomodoPlatform#1953) feat(nft): nft abi in withdraw_nft RPC, clear_nft_db RPC (KomodoPlatform#2039) refactor(utxo): refactor utxo output script creation (KomodoPlatform#1960) feat(ETH): balance event streaming for ETH (KomodoPlatform#2041) chore(release): bump mm2 version to 2.1.0-beta (KomodoPlatform#2044) feat(trezor): add segwit support for withdraw with trezor (KomodoPlatform#1984) chore(config): remove vscode launchjson (KomodoPlatform#2040) feat(trading-proto-upgrade): wasm DB, kickstart, refund states, v2 RPCs (KomodoPlatform#2015) feat(UTXO): balance event streaming for Electrum clients (KomodoPlatform#2013) feat(tx): add new sign_raw_transaction rpc for UTXO and EVM coins (KomodoPlatform#1930) fix(p2p): handle encode_and_sign errors (KomodoPlatform#2038) chore(release): add changelog entries for v2.0.0-beta (KomodoPlatform#2037) chore(network): write network information to stdout (KomodoPlatform#2034) fix(price_endpoints): add cached url (KomodoPlatform#2032) deps(network): sync with upstream yamux (KomodoPlatform#2030) ...

artemii235 and others added 30 commits October 30, 2023 13:35

WIP. Adding fields to MySwapsFiltersTable.

354e918

WIP. SwapDb modifications.

a9fb7b3

WASM WIP.

37c26f5

Merge remote-tracking branch 'origin/dev' into swap-proto-upgrade-ite…

2d22180

…ration-4 # Conflicts: # mm2src/mm2_main/src/lp_swap/swap_wasm_db.rs

WIP WASM DB.

4e4db07

Merge remote-tracking branch 'origin/dev' into swap-proto-upgrade-ite…

f3d3be3

…ration-4

WASM WIP.

7c92429

WIP WASM DB.

b803441

WASM WIP. Refactor.

1363d94

WIP. WASM trade_v2_test_rick_and_morty is green.

0b46e02

WIP. Error handling.

69e2a0e

WIP. Refactoring.

4d0bf50

WIP refactoring. WASM trade fails intentionally.

b53bab7

WIP. Trade v2 test is green on both native and WASM.

57b7a4b

WIP. Implementing storing on state machine's start.

19d8408

WIP. WASM compilation fails.

afb9a55

WIP. Implementing kickstart. Added Dockerized test.

225e06c

WIP. Implementing kickstart.

84f8bb3

WIP. Implementing kickstart.

96e847e

WIP. Implementing kickstart.

a37aa81

WIP. Kickstart PoC works on native.

1bcbcfd

WIP. Implementing kickstart. Error handling.

eeb45b4

WIP. Added BoolAsInt for WASM.

e269c45

Merge remote-tracking branch 'origin/dev' into swap-proto-upgrade-ite…

a52c55a

…ration-4

WIP. Error handling.

a2c959a

WIP. Adding support of SwapLock to new protocol.

d1dd632

WIP. Implemented SwapLock for upgraded swaps.

318283a

WIP. Remove code duplication for upgraded swaps kickstart handling.

50ff69f

WIP. Implemented swaps data migration (is_finished) on native.

a6be79c

WIP. Adding SwapsMigrationTable.

63ad3d1

Fix review notes.

7628efc

shamardy reviewed Dec 1, 2023

View reviewed changes

artemii235 added 3 commits December 4, 2023 09:56

Review fix.

c984d43

Use HD account 1 for both nodes in trade_v2_test_rick_and_morty.

765c63b

Merge remote-tracking branch 'origin/dev' into swap-proto-upgrade-ite…

bae1f48

…ration-4

shamardy previously approved these changes Dec 5, 2023

View reviewed changes

borngraced reviewed Dec 6, 2023

View reviewed changes

borngraced previously approved these changes Dec 6, 2023

View reviewed changes

shamardy added 2.1.0-beta QA sec-review no-deps labels Dec 6, 2023

borngraced reviewed Dec 10, 2023

View reviewed changes

ca333 requested review from ca333, DeckerSU and Alrighttt December 12, 2023 06:50

Merge remote-tracking branch 'origin/dev' into swap-proto-upgrade-ite…

8313bc4

…ration-4 # Conflicts: # mm2src/mm2_main/src/lp_native_dex.rs

artemii235 dismissed stale reviews from borngraced and shamardy via 8313bc4 December 13, 2023 07:00

ca333 approved these changes Dec 13, 2023

View reviewed changes

shamardy merged commit e61c41e into dev Dec 21, 2023
24 of 31 checks passed

shamardy deleted the swap-proto-upgrade-iteration-4 branch December 21, 2023 11:58

harsenyan3 mentioned this pull request Jul 16, 2024

Add v2 "my_swap_status" and "my_recent_swaps" RPC info on docs. KomodoPlatform/komodo-docs-mdx#280

Closed

shamardy mentioned this pull request Aug 1, 2024

chore(release): v2.1.0-beta #2043

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(trading-proto-upgrade): wasm DB, kickstart, refund states, v2 RPCs #2015

feat(trading-proto-upgrade): wasm DB, kickstart, refund states, v2 RPCs #2015

artemii235 commented Nov 27, 2023 •

edited

Loading

artemii235 commented Dec 1, 2023

shamardy left a comment

shamardy Dec 1, 2023

artemii235 Dec 4, 2023

shamardy Dec 1, 2023

artemii235 Dec 4, 2023

shamardy Dec 1, 2023

artemii235 Dec 4, 2023

shamardy left a comment

shamardy commented Dec 5, 2023

borngraced Dec 6, 2023

borngraced left a comment

shamardy commented Dec 6, 2023

borngraced Dec 10, 2023

artemii235 Dec 11, 2023

ca333 left a comment

		maker_coin_swap_contract: None,
		taker_coin_swap_contract: None,

feat(trading-proto-upgrade): wasm DB, kickstart, refund states, v2 RPCs #2015

feat(trading-proto-upgrade): wasm DB, kickstart, refund states, v2 RPCs #2015

Conversation

artemii235 commented Nov 27, 2023 • edited Loading

artemii235 commented Dec 1, 2023

shamardy left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shamardy left a comment

Choose a reason for hiding this comment

shamardy commented Dec 5, 2023

Choose a reason for hiding this comment

borngraced left a comment

Choose a reason for hiding this comment

shamardy commented Dec 6, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ca333 left a comment

Choose a reason for hiding this comment

artemii235 commented Nov 27, 2023 •

edited

Loading