SNO-304: Parachain -> Ethereum: bundle messages by account (paritytec…

…h#637) * WIP: attempt at new data structures * Specify path to config trait * Check AccountId's traits for EnqueuedMessage * Reorder MessageBundle type params * Add trait bounds to AccountId * Don't store AccountId in MessageBundle messages The AccountId is already in the MessageBundle, so no need to add it to every message. * Ensure that AccountId implements MaxEncodedLen * Finish rename to Nonces * Whitespace * Remove use of principal * Fix struct access in average_payload_size * WIP Collect MessageBundles partitioned by AccountId * Convert message queue to map of message bundles * Fix account id used for nonces in test * Fix warnings * Convert message bundles to Vec * Remove last uses of principal in basic channel * Found a better way to handle this Just use BoundedVec * Insert spaces to help with Markdown folding * Fix BTreeMap import * Revert "Remove last uses of principal in basic channel" This reverts commit 2f5d6e07a56bf74baeaf6f0421b68beb77b17644. * Fix some benchmarking types * Remove last uses of principal in basic channel Leave the set_principal call in place as a no-op to avoid removing the config for Call. We'll re-add it when we add the leaf node proof anyway. * First stab at Merklizing all message bundles I mean, if it compiles... 🤷 * Make keccak256 public to try resolve import issues * Use configured Hash type * rustfmt * Fix tests in merkle_proof * Fix nonce update * Remove not_authorized test * Add TODO * Ignore large tests * rustfmt * Store all message bundles in the event * Clean up commit() a bit * Extract function make_message_bundles * Eth-encode message bundles before merklizing them * Update comments * Remove unused function * Persist ABI-encoded message bundles * Use SCALE encoding for off-chain message bundles * Fix debug import * Switch to Token::FixedBytes * s/map/for/ * Remove loop implementation * Remove generic output type * rustfmt * Add StoredLeaves struct to communicate intent * WIP Custom RPC and runtime API * fixes (paritytech#647) * Update snow{blink,bridge} with fixes * Move merkle_proof to a separate crate * Add RPC call stub * Add runtime API implementation * Remove basic channel's RPC runtime API * WIP SCALE-encode Merkle proof from RPC call * Move StoredLeaves struct into primitives crate * Disable default features in basic channel crates * Add import for function * Use u64 instead of usize for proofs * Fix missed issues from removing the runtime API * Fix merkle-proof tests * Update basic outbound channel's tests for commit() * rustfmt * Clean up RPC call * Fix typo & remove unused import & comment * Replace encode with as_ref for ethabi encoding * Remove AsRef implementation for MessageBundle * Fix up RPC crate * Remove unused dependencies in basic channel * Remove redundant package names * Use import alias * Reorder rpc pallet * rustfmt * Mention outbound in Merkle proof RPC * Remove unused error * Remove set_principal from basic outbound channel * Update docstring for commit() * Ignore unused result * Shorten type * Remove unused dependency * Add prefix to incentivized channel in runtimes * Remove unused import * Remove primitives crate * Add V2 of basic inbound channel * Fix typos * Re-order message types * Fix use of nonces * Generate commitment from message bundle & proof * Fix contract name * Generate basic inbound channel v2 go bindings * Fix RPC name * Fix rpc handler (paritytech#659) * Detect leaf index out of bounds in RPC * Add tests for RPC handler * Overwrite BasicInboundChannel instead of using V2 * Link MerkleProof library for testing * Typos * Add account id filter to parachain relayer * Add MerkleProof library to channel deployment * Add MerkleProof RPC to parachain relayer * Extract account id decoding to getter * Simplify search for matching bundle * Forward Merkle proof to inbound channel contract * Fix mapstructure tag * Move decoded account id to listener struct * Tweak error messages * Extract finding bundle * Extract fetchBundleProof * Use existing parachain connection * Fix account id decoding * Fetch basic channel nonce by accountID * Reuse accountID stored on startup * Fix listener decoding & writer bounds check * Add account id to message bundle log * Remove TODO This is only relevant if we expect to have around 2**63 different accounts with messages in the same commitment. * Add RPC name to error messages * Use the Alice account id as default for ACCOUNT_ID * Mention build-essential for setting up lodestar * Fix log file names * Log leafProof and hashSides in relayer * Fix hash side value Should be hashed on the left when the side is true, which is when the index is even. * Use same names and param order as contract * Update contract fixture data * Revert "Fix hash side value" This reverts commit ec0b2e9aa7ee2a8605f68152392983e344f0057f. * Add reminder to use a port of a relay chain node * Fix basic inbound channel contract tests * Mention port forwarding in port reminder * Replace pointer with byte array * Improve error logs * Return proof value instead of mutating pointer * Add second message bundle to basic channel test * WIP Add second account to basic channel e2e test * Start accountID with lowercase * Update new dependencies to 0.9.25 * Fix RPC handler tests * Update parachain metadata * Allow multiple accountIDs in parachain relayer * Fix scanning logic * Update basic nonces log * Fix initial len of nonces slice * Fix jq for multiple account ids * Add bob key for E2E test * Check correct colletion's length to stop scanning * Fix param description * Whitespace * Ignore local asdf versions * Replace base 2 log with simpler bit calculation * Remove resolved TODOs * Typo * Add comment explaining ACCOUNT_IDS * s/accountIDs/accounts/g * Add issue key to TODO * Typo * Add geth version to README * Fix templating * Update parachain lockfile * Fix recipient address * Fix dotapp tests after rename Co-authored-by: Vincent Geddes <vincent.geddes@hey.com> Co-authored-by: Vincent Geddes <vincent@snowfork.com>
Snowfork · Aug 25, 2022 · 83b09ef · 83b09ef
1 parent 69fdd4b
commit 83b09ef
Show file tree

Hide file tree

Showing 59 changed files with 1,967 additions and 522 deletions.
diff --git a/.gitignore b/.gitignore
@@ -8,4 +8,7 @@ workspace.code-workspace
 node_modules/
 
 #Intellij project file
-.idea
+.idea
+
+# asdf plugin versions
+.tool-versions
diff --git a/ethereum/README.md b/ethereum/README.md
@@ -34,7 +34,7 @@ npx hardhat test
 
 ### Updating fixture data for unit tests
 
-We use logging artifacts from the relayer in E2E stack as a source of compliant fixture data.
+We use logging artifacts from the relayer in the E2E stack as a source of compliant fixture data.
 
 BEEFY commitment & proofs extracted from `../test/beefy-relay.log`.
 * test/fixtures/beefy-relay-basic.json
@@ -54,17 +54,20 @@ yarn test --grep 'should transfer DOT from Substrate to Ethereum \(incentivized
 ```
 
 Steps for updating the fixture data for the basic channel:
-1. Grep for `Sent transaction BasicInboundChannel.submit` in `parachain-relay.json`
+1. Grep for `Sent transaction BasicInboundChannel.submit` in `parachain-relay.log`
 2. Copy that log line into `./test/fixtures/parachain-relay-basic.json`
-3. In that file, take the `beefyBlock` field (a block hash) and use polkadot.js explorer to find the corresponding block number.
+3. In that file, take the `beefyBlock` field (a block hash) and use polkadot.js explorer to find the corresponding block number. Make sure
+   you're pointing the polkadot.js explorer at one of the relay chain nodes when you do, eg. port 9944 instead of 11144. See
+   [launch-config.json](../test/config/launch-config.json) for the ports used by relaychain & parachain nodes. Also update port forwarding
+   as necessary.
 4. Run the following (substituting `$BLOCKNUMBER`) and paste the output into `./test/fixtures/beefy-relay-basic.json`:
 ```bash
 jq -s --argjson blocknumber $BLOCKNUMBER '.[] | select( .message | contains("Sent SubmitFinal transaction")) | select( .params.commitment.blockNumber | contains($blocknumber))' beefy-relay.log
 ```
 5: NOTE: if the produced `./test/fixtures/beefy-relay-basic.json` doesn't contain a `params.leaf` field, repeat all the steps again.
 
-Steps for updating the fixture data for the basic channel:
-1. Grep for `Sent transaction IncentivizedInboundChannel.submit` in `parachain-relay.json`
+Steps for updating the fixture data for the incentivized channel:
+1. Grep for `Sent transaction IncentivizedInboundChannel.submit` in `parachain-relay.log`
 2. Copy that log line into `./test/fixtures/parachain-relay-incentivized.json`
 3. In that file, take the `beefyBlock` field (a hash) and use polkadot.js explorer to find the corresponding block number.
 4. Run the following (substituting `$BLOCKNUMBER`) and paste the output into `./test/fixtures/beefy-relay-incentivized.json`:
@@ -113,7 +116,7 @@ yarn hardhat renounce \
   --network localhost
 ```
 
-### View the address of deployed contract's
+### View the address of deployed contracts
 
 ```sh
 yarn hardhat  contractAddressList \

diff --git a/ethereum/contracts/BasicInboundChannel.sol b/ethereum/contracts/BasicInboundChannel.sol
@@ -2,19 +2,21 @@
 pragma solidity ^0.8.9;
 
 import "./ParachainClient.sol";
+import "./utils/MerkleProof.sol";
 
 contract BasicInboundChannel {
     uint256 public constant MAX_GAS_PER_MESSAGE = 100000;
     uint256 public constant GAS_BUFFER = 60000;
 
     uint8 public immutable sourceChannelID;
 
-    uint64 public nonce;
+    mapping(bytes32 => uint64) public nonces;
 
     ParachainClient public parachainClient;
 
     struct MessageBundle {
         uint8 sourceChannelID;
+        bytes32 account;
         uint64 nonce;
         Message[] messages;
     }
@@ -28,22 +30,27 @@ contract BasicInboundChannel {
     event MessageDispatched(uint64 id, bool result);
 
     constructor(uint8 _sourceChannelID, ParachainClient _parachainClient) {
-        nonce = 0;
         sourceChannelID = _sourceChannelID;
         parachainClient = _parachainClient;
     }
 
-    function submit(MessageBundle calldata bundle,  bytes calldata proof) external {
-        bytes32 commitment = keccak256(abi.encode(bundle));
+    function submit(
+        MessageBundle calldata bundle,
+        bytes32[] calldata leafProof,
+        bool[] calldata hashSides,
+        bytes calldata proof
+    ) external {
+        bytes32 leafHash = keccak256(abi.encode(bundle));
+        bytes32 commitment = MerkleProof.computeRootFromProofAndSide(leafHash, leafProof, hashSides);
 
         require(parachainClient.verifyCommitment(commitment, proof), "Invalid proof");
         require(bundle.sourceChannelID == sourceChannelID, "Invalid source channel");
-        require(bundle.nonce == nonce + 1, "Invalid nonce");
+        require(bundle.nonce == nonces[bundle.account] + 1, "Invalid nonce");
         require(
             gasleft() >= (bundle.messages.length * MAX_GAS_PER_MESSAGE) + GAS_BUFFER,
             "insufficient gas for delivery of all messages"
         );
-        nonce++;
+        nonces[bundle.account]++;
         dispatch(bundle);
     }
 

diff --git a/ethereum/contracts/utils/MerkleProof.sol b/ethereum/contracts/utils/MerkleProof.sol
@@ -34,8 +34,8 @@ library MerkleProof {
      *
      * @param leaf the leaf we want to prove
      * @param proof an array of nodes to be hashed in order that they should be hashed
-     * @param side an array of booleans signalling whether the corresponding node should be hashed on the left side or
-     * the right side of the current hash
+     * @param side an array of booleans signalling whether the corresponding proof hash should be hashed on the left side (true) or
+     * the right side (false) of the current hash
      */
     function computeRootFromProofAndSide(
         bytes32 leaf,

diff --git a/ethereum/deploy/03-channels.ts b/ethereum/deploy/03-channels.ts
@@ -19,12 +19,14 @@ module.exports = async ({
 
   let parachainClient = await deployments.get("ParachainClient")
   let scaleCodecLibrary = await deployments.get("ScaleCodec")
+  let merkleProof = await deployments.get("MerkleProof")
 
   await deployments.deploy("BasicInboundChannel", {
     from: deployer,
     args: [basicChannelSourceID, parachainClient.address],
     libraries: {
         ScaleCodec: scaleCodecLibrary.address,
+        MerkleProof: merkleProof.address,
     },
     log: true,
     autoMine: true,

diff --git a/ethereum/test/fixtures/beefy-relay-basic.json b/ethereum/test/fixtures/beefy-relay-basic.json
@@ -1,55 +1,51 @@
 {
-    "@timestamp": "2022-05-27T17:37:23.948296332+02:00",
-    "commitmentHash": "0xb2fd4ae9d90dc9157f1db4990531a4e7d72ea099ea11ce445b547fb72ea257eb",
-    "level": "info",
-    "message": "Sent SubmitFinal transaction",
-    "params": {
-        "commitment": {
-            "blockNumber": 191,
-            "payload": {
-                "mmrRootHash": "0x7c3891fa41dc873d0811cf92d9dc10df2dde7841644f923b1b105ef04fc2739a",
-                "prefix": "0x046d6880",
-                "suffix": "0x"
-            },
-            "validatorSetID": 19
-        },
-        "id": 23,
-        "leaf": {
-            "nextAuthoritySetID": 20,
-            "nextAuthoritySetLen": 3,
-            "nextAuthoritySetRoot": "0x42b63941ec636f52303b3c33f53349830d8a466e9456d25d22b28f4bb0ad0365",
-            "parachainHeadsRoot": "0xa75e3caee1bca43af9ab34fd112b60fe4888c87838418feb7fb59a60834c62c8",
-            "parentHash": "0xbe4fee01a03f38292ecbd140e6c0c0fa048c5a2cf4f21013154e44909d5e9aba",
-            "parentNumber": 190,
-            "version": 0
-        },
-        "leafProof": {
-            "items": [
-                "0x5fff8ec62e78aced1d381c13696fadb949750dcd2efe23731e6d935e750aed98",
-                "0x9dfad8852244ee9e2d2c76a5bf93e6a843a72049a2785af4158db17662755b83",
-                "0x8f8256410940445ae752acc7568640887c0e19c0ed267bc0eb7d5e74938e2166",
-                "0x7036cc132cfe5131a3ea728195fb0ea4a78f6811f1a2a3df3dc4d2ec1bdcf418",
-                "0xaccd2ce7aac449951790250ce67c39bb66d5078db8e442de637de9ad8d1c8a7d",
-                "0x6ba4f2edba4e9c866d6b3d29e38ba0b7aee2f2d4ecaab3f07e4aaa311110f412"
-            ],
-            "order": 0
-        },
-        "proof": {
-            "addrs": [
-                "0x5630a480727cd7799073b36472d9b1a6031f840b"
-            ],
-            "indices": [
-                2
-            ],
-            "merkleProofs": [
-                [
-                    "0x697ea2a8fe5b03468548a7a413424a6292ab44a82a6f5cc594c3fa7dda7ce402"
-                ]
-            ],
-            "signatures": [
-                "0x9d8696815f324152060cf80c58c1b519e8e137f0b3c232eda4497c28d2322911465563c7a3f75a83c48523975f0a48ba75dd0701a316a321dade3b61473cfd4f1b"
-            ]
-        }
+  "@timestamp": "2022-08-15T14:33:32.464197692Z",
+  "commitmentHash": "0x243baf0066d021d42716081dad0b30499dad95a300daa269ed8f6f6334d95975",
+  "level": "info",
+  "message": "Sent SubmitFinal transaction",
+  "params": {
+    "commitment": {
+      "blockNumber": 371,
+      "payload": {
+        "mmrRootHash": "0x482fcbd18294c4b4f339f825537530cfcc678eeea469caa807438d35ace62f04",
+        "prefix": "0x046d6880",
+        "suffix": "0x"
+      },
+      "validatorSetID": 37
     },
-    "txHash": "0xd19d28de5a83cf3f673d6299b1df1861b35403693b8e4ca6b2b436f97c76cba7"
+    "id": 37,
+    "leaf": {
+      "nextAuthoritySetID": 38,
+      "nextAuthoritySetLen": 3,
+      "nextAuthoritySetRoot": "0x42b63941ec636f52303b3c33f53349830d8a466e9456d25d22b28f4bb0ad0365",
+      "parachainHeadsRoot": "0xc992465982e7733f5f91c60f6c7c5d4433298c10b348487081f2356d80a0133f",
+      "parentHash": "0x2a74fc1410a321daefc1ae17adc69048db56f4d37660e7af042289480de59897",
+      "parentNumber": 370,
+      "version": 0
+    },
+    "leafProof": {
+      "items": [
+        "0xe8ae8d4c8027764aa0fdae351c30c6085f7822ad6295ae1bd445ee8bef564901",
+        "0xe4d591609cb75673ef8992d1ae6c518ad95d8f924f75249ce43153d01380c79f",
+        "0xb2852e70b508acbda330c6f842d51f4eab82d34b991fe6679d37f2eeedae6ccd",
+        "0x6a83a49e6424b0de032f730064213f4783f2c9f59dab4480f88673a042102ab2",
+        "0xee4688d1831443e4c7f2d47265fd529dd50e41a4c49c5f31a04bf45320f59614"
+      ],
+      "order": 0
+    },
+    "proof": {
+      "addrs": ["0x25451a4de12dccc2d166922fa938e900fcc4ed24"],
+      "indices": [1],
+      "merkleProofs": [
+        [
+          "0xaeb47a269393297f4b0a3c9c9cfd00c7a4195255274cf39d83dabc2fcc9ff3d7",
+          "0x3eb799651607280e854bd2e42c1df1c8e4a6167772dfb3c64a813e40f6e87136"
+        ]
+      ],
+      "signatures": [
+        "0xcfb8535b624c6c1e779aa9e5d28eb0b10ebf4459e890b55c2d3533644bfc8f3e3729f1ea4402838df59696079e1a393ef1136908753ae0932ca3f7661b7e61091b"
+      ]
+    }
+  },
+  "txHash": "0xbe72a9b6640b76ad5db4d47a138def511fc40c9b67ffae0bf303ebdb44e72bed"
 }