-
Notifications
You must be signed in to change notification settings - Fork 699
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
XCM Fee Payment Runtime API #3607
XCM Fee Payment Runtime API #3607
Conversation
refactor: rpc signatures feat(rococo): full impl `XcmPaymentApi` fix cargo fix use fix use
Hi @franciscoaguirre, it is the PR we spoke of yesterday :) |
@mrshiposha Awesome! I'd like to add this comment about how we should also have a way of estimating delivery fees. /// Given a message, returns the execution `Weight` and the delivery fees needed to be paid.
/// The execution weight can be converted to fees by using `query_acceptable_payment_assets` and `query_weight_to_asset_fee`.
fn query_xcm_fees(message: VersionedXcm<Call>) -> Result<(Weight, Assets), Error>; Alternatively, we could have it return just We could also keep this call for execution weight and have a separate API call for delivery fees. What do you think? Also, I'm adding @xlc and @acatangiu to this conversation. |
A reply to #3607 (comment). We haven't considered the delivery fee thing yet, but yeah, it should be covered somehow. We opened the PR in its current state to have a basis for further discussion, including the solution for the delivery fee |
This is great!
I think the currently proposed (do ☝️ in this PR) (👇 follow-up/different PR(s)) For the full fees story (like proposed in #690 (comment)), and providing users/builders/dapps with a nice UX, we will probably add specialized |
I think this is a good first thing to add. But I think we could add an API that given a message, it'll give you the delivery fees needed to execute that message. So it dry-runs it to find all messages that need to be sent and to which destinations, and returns the resulting total fees. |
Clarifying my previous comment. I think we should start simple. There are many complicated things we could do like the thing I described about dry-running, but those would be slow to review and merge. This API looks perfect for what we need right now, which is exposing runtime functions we don't expose currently. After we merge this we can build on top of it. Basically what Adrian said 😄 |
I added the new function exposing delivery fees in UniqueNetwork#3 |
Add function to query delivery fees
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
…ytech#3872) # Context Estimating fees for XCM execution and sending has been an area with bad UX. The addition of the [XcmPaymentApi](paritytech#3607) exposed the necessary components to be able to estimate XCM fees correctly, however, that was not the full story. The `XcmPaymentApi` works for estimating fees only if you know the specific XCM you want to execute or send. This is necessary but most UIs want to estimate the fees for extrinsics, they don't necessarily know the XCM program that's executed by them. # Main addition A new runtime API is introduced, the `XcmDryRunApi`, that given an extrinsic, or an XCM program, returns its effects: - Execution result - Local XCM (in the case of an extrinsic) - Forwarded XCMs - List of events This API can be used on its own for dry-running purposes, for double-checking or testing, but it mainly shines when used in conjunction with the `XcmPaymentApi`. UIs can use these two APIs to estimate transfers. # How it works New tests are added to exemplify how to incorporate both APIs. There's a mock test just to make sure everything works under `xcm-fee-payment-runtime-api`. There's a real-world test using Westend and AssetHubWestend under `cumulus/parachains/integration-tests/emulated/tests/assets/asset-hub-westend/src/tests/xcm_fee_estimation.rs`. Added both a test for a simple teleport between chains and a reserve transfer asset between two parachains going through a reserve. The steps to follow: - Use `XcmDryRunApi::dry_run_extrinsic` to get local XCM program and forwarded messages - For each forwarded message - Use `XcmPaymentApi::query_delivery_fee` LOCALLY to get the delivery fees - Use `XcmPaymentApi::query_xcm_weight` ON THE DESTINATION to get the remote execution weight - (optional) Use `XcmPaymentApi::query_acceptable_payment_assets` ON THE DESTINATION to know on which assets the execution fees can be paid - Use `XcmPaymentApi::query_weight_to_asset_fee` ON THE DESTINATION to convert weight to the actual remote execution fees - Use `XcmDryRunApi::dry_run_xcm` ON THE DESTINATION to know if a new message will be forwarded, if so, continue # Dear reviewer The changes in this PR are grouped as follows, and in order of importance: - Addition of new runtime API - Definition, mock and simple tests: polkadot/xcm/xcm-fee-payment-runtime-api/* - Implemented on Westend, Asset Hub Westend and Penpal, will implement on every runtime in a following PR - Addition of a new config item to the XCM executor for recording xcms about to be executed - Definition: polkadot/xcm/xcm-executor/* - Implementation: polkadot/xcm/pallet-xcm/* - had to update all runtime xcm_config.rs files with `type XcmRecorder = XcmPallet;` - Addition of a new trait for inspecting the messages in queues - Definition: polkadot/xcm/xcm-builder/src/routing.rs - Implemented it on all routers: - ChildParachainRouter: polkadot/runtime/common/src/xcm_sender.rs - ParentAsUmp: cumulus/primitives/utility/src/lib.rs (piggybacked on implementation in cumulus/pallets/parachain-system/src/lib.rs) - XcmpQueue: cumulus/pallets/xcmp-queue/src/lib.rs - Bridge: bridges/modules/xcm-bridge-hub-router/src/lib.rs - More complicated and useful tests: - cumulus/parachains/integration-tests/emulated/tests/assets/asset-hub-westend/src/tests/xcm_fee_estimation.rs ## Next steps With this PR, Westend, AssetHubWestend, Rococo and AssetHubRococo have the new API. UIs can test on these runtimes to create better experiences around cross-chain operations. Next: - Add XcmDryRunApi to all system parachains - Integrate xcm fee estimation in all emulated tests - Get this on the fellowship runtimes --------- Co-authored-by: Adrian Catangiu <adrian@parity.io>
…aritytech#4634) Depends on paritytech#4621. Implemented the [`XcmPaymentApi`](paritytech#3607) and [`DryRunApi`](paritytech#3872) on all system parachains. More scenarios can be tested on both rococo and westend if all system parachains implement this APIs. The objective is for all XCM-enabled runtimes to implement them. After demonstrating fee estimation in a UI on the testnets, come the fellowship runtimes. Step towards paritytech#690.
…i` nits (paritytech#4355) This PR: - changes `xcm::v4` to `xcm::prelude` imports for coretime stuff - changes `query_acceptable_payment_assets` / `query_weight_to_asset_fee` implementations to be more resilient to the XCM version change - adds `xcm_fee_payment_runtime_api::XcmPaymentApi` to the AssetHubRococo/Westend exposing a native token as acceptable payment asset Continuation of: paritytech#3607 Closes: paritytech#4297 ## Possible follow-ups - [ ] add all sufficient assets (`Assets`, `ForeignAssets`) as acceptable payment assets ?
…ytech#3872) Estimating fees for XCM execution and sending has been an area with bad UX. The addition of the [XcmPaymentApi](paritytech#3607) exposed the necessary components to be able to estimate XCM fees correctly, however, that was not the full story. The `XcmPaymentApi` works for estimating fees only if you know the specific XCM you want to execute or send. This is necessary but most UIs want to estimate the fees for extrinsics, they don't necessarily know the XCM program that's executed by them. A new runtime API is introduced, the `XcmDryRunApi`, that given an extrinsic, or an XCM program, returns its effects: - Execution result - Local XCM (in the case of an extrinsic) - Forwarded XCMs - List of events This API can be used on its own for dry-running purposes, for double-checking or testing, but it mainly shines when used in conjunction with the `XcmPaymentApi`. UIs can use these two APIs to estimate transfers. New tests are added to exemplify how to incorporate both APIs. There's a mock test just to make sure everything works under `xcm-fee-payment-runtime-api`. There's a real-world test using Westend and AssetHubWestend under `cumulus/parachains/integration-tests/emulated/tests/assets/asset-hub-westend/src/tests/xcm_fee_estimation.rs`. Added both a test for a simple teleport between chains and a reserve transfer asset between two parachains going through a reserve. The steps to follow: - Use `XcmDryRunApi::dry_run_extrinsic` to get local XCM program and forwarded messages - For each forwarded message - Use `XcmPaymentApi::query_delivery_fee` LOCALLY to get the delivery fees - Use `XcmPaymentApi::query_xcm_weight` ON THE DESTINATION to get the remote execution weight - (optional) Use `XcmPaymentApi::query_acceptable_payment_assets` ON THE DESTINATION to know on which assets the execution fees can be paid - Use `XcmPaymentApi::query_weight_to_asset_fee` ON THE DESTINATION to convert weight to the actual remote execution fees - Use `XcmDryRunApi::dry_run_xcm` ON THE DESTINATION to know if a new message will be forwarded, if so, continue The changes in this PR are grouped as follows, and in order of importance: - Addition of new runtime API - Definition, mock and simple tests: polkadot/xcm/xcm-fee-payment-runtime-api/* - Implemented on Westend, Asset Hub Westend and Penpal, will implement on every runtime in a following PR - Addition of a new config item to the XCM executor for recording xcms about to be executed - Definition: polkadot/xcm/xcm-executor/* - Implementation: polkadot/xcm/pallet-xcm/* - had to update all runtime xcm_config.rs files with `type XcmRecorder = XcmPallet;` - Addition of a new trait for inspecting the messages in queues - Definition: polkadot/xcm/xcm-builder/src/routing.rs - Implemented it on all routers: - ChildParachainRouter: polkadot/runtime/common/src/xcm_sender.rs - ParentAsUmp: cumulus/primitives/utility/src/lib.rs (piggybacked on implementation in cumulus/pallets/parachain-system/src/lib.rs) - XcmpQueue: cumulus/pallets/xcmp-queue/src/lib.rs - Bridge: bridges/modules/xcm-bridge-hub-router/src/lib.rs - More complicated and useful tests: - cumulus/parachains/integration-tests/emulated/tests/assets/asset-hub-westend/src/tests/xcm_fee_estimation.rs With this PR, Westend, AssetHubWestend, Rococo and AssetHubRococo have the new API. UIs can test on these runtimes to create better experiences around cross-chain operations. Next: - Add XcmDryRunApi to all system parachains - Integrate xcm fee estimation in all emulated tests - Get this on the fellowship runtimes --------- Co-authored-by: Adrian Catangiu <adrian@parity.io>
…i` nits (paritytech#4355) This PR: - changes `xcm::v4` to `xcm::prelude` imports for coretime stuff - changes `query_acceptable_payment_assets` / `query_weight_to_asset_fee` implementations to be more resilient to the XCM version change - adds `xcm_fee_payment_runtime_api::XcmPaymentApi` to the AssetHubRococo/Westend exposing a native token as acceptable payment asset Continuation of: paritytech#3607 Closes: paritytech#4297 ## Possible follow-ups - [ ] add all sufficient assets (`Assets`, `ForeignAssets`) as acceptable payment assets ?
…ytech#3872) Estimating fees for XCM execution and sending has been an area with bad UX. The addition of the [XcmPaymentApi](paritytech#3607) exposed the necessary components to be able to estimate XCM fees correctly, however, that was not the full story. The `XcmPaymentApi` works for estimating fees only if you know the specific XCM you want to execute or send. This is necessary but most UIs want to estimate the fees for extrinsics, they don't necessarily know the XCM program that's executed by them. A new runtime API is introduced, the `XcmDryRunApi`, that given an extrinsic, or an XCM program, returns its effects: - Execution result - Local XCM (in the case of an extrinsic) - Forwarded XCMs - List of events This API can be used on its own for dry-running purposes, for double-checking or testing, but it mainly shines when used in conjunction with the `XcmPaymentApi`. UIs can use these two APIs to estimate transfers. New tests are added to exemplify how to incorporate both APIs. There's a mock test just to make sure everything works under `xcm-fee-payment-runtime-api`. There's a real-world test using Westend and AssetHubWestend under `cumulus/parachains/integration-tests/emulated/tests/assets/asset-hub-westend/src/tests/xcm_fee_estimation.rs`. Added both a test for a simple teleport between chains and a reserve transfer asset between two parachains going through a reserve. The steps to follow: - Use `XcmDryRunApi::dry_run_extrinsic` to get local XCM program and forwarded messages - For each forwarded message - Use `XcmPaymentApi::query_delivery_fee` LOCALLY to get the delivery fees - Use `XcmPaymentApi::query_xcm_weight` ON THE DESTINATION to get the remote execution weight - (optional) Use `XcmPaymentApi::query_acceptable_payment_assets` ON THE DESTINATION to know on which assets the execution fees can be paid - Use `XcmPaymentApi::query_weight_to_asset_fee` ON THE DESTINATION to convert weight to the actual remote execution fees - Use `XcmDryRunApi::dry_run_xcm` ON THE DESTINATION to know if a new message will be forwarded, if so, continue The changes in this PR are grouped as follows, and in order of importance: - Addition of new runtime API - Definition, mock and simple tests: polkadot/xcm/xcm-fee-payment-runtime-api/* - Implemented on Westend, Asset Hub Westend and Penpal, will implement on every runtime in a following PR - Addition of a new config item to the XCM executor for recording xcms about to be executed - Definition: polkadot/xcm/xcm-executor/* - Implementation: polkadot/xcm/pallet-xcm/* - had to update all runtime xcm_config.rs files with `type XcmRecorder = XcmPallet;` - Addition of a new trait for inspecting the messages in queues - Definition: polkadot/xcm/xcm-builder/src/routing.rs - Implemented it on all routers: - ChildParachainRouter: polkadot/runtime/common/src/xcm_sender.rs - ParentAsUmp: cumulus/primitives/utility/src/lib.rs (piggybacked on implementation in cumulus/pallets/parachain-system/src/lib.rs) - XcmpQueue: cumulus/pallets/xcmp-queue/src/lib.rs - Bridge: bridges/modules/xcm-bridge-hub-router/src/lib.rs - More complicated and useful tests: - cumulus/parachains/integration-tests/emulated/tests/assets/asset-hub-westend/src/tests/xcm_fee_estimation.rs With this PR, Westend, AssetHubWestend, Rococo and AssetHubRococo have the new API. UIs can test on these runtimes to create better experiences around cross-chain operations. Next: - Add XcmDryRunApi to all system parachains - Integrate xcm fee estimation in all emulated tests - Get this on the fellowship runtimes --------- Co-authored-by: Adrian Catangiu <adrian@parity.io>
Adds the paritytech/polkadot-sdk#3607 and paritytech/polkadot-sdk#3872 to all runtimes. These APIs work together to allow dry running and estimating execution and delivery fees for XCMs. This PR doesn't allow querying the price in assets different than the relay token for asset hubs. Can be done in a following PR. Also tracking #388 and #389 for future improvemens. Old PR was #359, this one targets main already updated to Polkadot SDK 1.13. ## Dear reviewer Although there are a lot of changes, the main one is the addition of the two aforementioned APIs to all relays and system parachains. The rest of the files are tests, which all have the same format.
This pull request has been mentioned on Polkadot Forum. There might be relevant details there: https://forum.polkadot.network/t/polkadot-technical-summit-xcm/9384/1 |
…i` nits (paritytech#4355) This PR: - changes `xcm::v4` to `xcm::prelude` imports for coretime stuff - changes `query_acceptable_payment_assets` / `query_weight_to_asset_fee` implementations to be more resilient to the XCM version change - adds `xcm_fee_payment_runtime_api::XcmPaymentApi` to the AssetHubRococo/Westend exposing a native token as acceptable payment asset Continuation of: paritytech#3607 Closes: paritytech#4297 ## Possible follow-ups - [ ] add all sufficient assets (`Assets`, `ForeignAssets`) as acceptable payment assets ?
…ytech#3872) Estimating fees for XCM execution and sending has been an area with bad UX. The addition of the [XcmPaymentApi](paritytech#3607) exposed the necessary components to be able to estimate XCM fees correctly, however, that was not the full story. The `XcmPaymentApi` works for estimating fees only if you know the specific XCM you want to execute or send. This is necessary but most UIs want to estimate the fees for extrinsics, they don't necessarily know the XCM program that's executed by them. A new runtime API is introduced, the `XcmDryRunApi`, that given an extrinsic, or an XCM program, returns its effects: - Execution result - Local XCM (in the case of an extrinsic) - Forwarded XCMs - List of events This API can be used on its own for dry-running purposes, for double-checking or testing, but it mainly shines when used in conjunction with the `XcmPaymentApi`. UIs can use these two APIs to estimate transfers. New tests are added to exemplify how to incorporate both APIs. There's a mock test just to make sure everything works under `xcm-fee-payment-runtime-api`. There's a real-world test using Westend and AssetHubWestend under `cumulus/parachains/integration-tests/emulated/tests/assets/asset-hub-westend/src/tests/xcm_fee_estimation.rs`. Added both a test for a simple teleport between chains and a reserve transfer asset between two parachains going through a reserve. The steps to follow: - Use `XcmDryRunApi::dry_run_extrinsic` to get local XCM program and forwarded messages - For each forwarded message - Use `XcmPaymentApi::query_delivery_fee` LOCALLY to get the delivery fees - Use `XcmPaymentApi::query_xcm_weight` ON THE DESTINATION to get the remote execution weight - (optional) Use `XcmPaymentApi::query_acceptable_payment_assets` ON THE DESTINATION to know on which assets the execution fees can be paid - Use `XcmPaymentApi::query_weight_to_asset_fee` ON THE DESTINATION to convert weight to the actual remote execution fees - Use `XcmDryRunApi::dry_run_xcm` ON THE DESTINATION to know if a new message will be forwarded, if so, continue The changes in this PR are grouped as follows, and in order of importance: - Addition of new runtime API - Definition, mock and simple tests: polkadot/xcm/xcm-fee-payment-runtime-api/* - Implemented on Westend, Asset Hub Westend and Penpal, will implement on every runtime in a following PR - Addition of a new config item to the XCM executor for recording xcms about to be executed - Definition: polkadot/xcm/xcm-executor/* - Implementation: polkadot/xcm/pallet-xcm/* - had to update all runtime xcm_config.rs files with `type XcmRecorder = XcmPallet;` - Addition of a new trait for inspecting the messages in queues - Definition: polkadot/xcm/xcm-builder/src/routing.rs - Implemented it on all routers: - ChildParachainRouter: polkadot/runtime/common/src/xcm_sender.rs - ParentAsUmp: cumulus/primitives/utility/src/lib.rs (piggybacked on implementation in cumulus/pallets/parachain-system/src/lib.rs) - XcmpQueue: cumulus/pallets/xcmp-queue/src/lib.rs - Bridge: bridges/modules/xcm-bridge-hub-router/src/lib.rs - More complicated and useful tests: - cumulus/parachains/integration-tests/emulated/tests/assets/asset-hub-westend/src/tests/xcm_fee_estimation.rs With this PR, Westend, AssetHubWestend, Rococo and AssetHubRococo have the new API. UIs can test on these runtimes to create better experiences around cross-chain operations. Next: - Add XcmDryRunApi to all system parachains - Integrate xcm fee estimation in all emulated tests - Get this on the fellowship runtimes --------- Co-authored-by: Adrian Catangiu <adrian@parity.io>
…i` nits (paritytech#4355) This PR: - changes `xcm::v4` to `xcm::prelude` imports for coretime stuff - changes `query_acceptable_payment_assets` / `query_weight_to_asset_fee` implementations to be more resilient to the XCM version change - adds `xcm_fee_payment_runtime_api::XcmPaymentApi` to the AssetHubRococo/Westend exposing a native token as acceptable payment asset Continuation of: paritytech#3607 Closes: paritytech#4297 ## Possible follow-ups - [ ] add all sufficient assets (`Assets`, `ForeignAssets`) as acceptable payment assets ?
…ytech#3872) # Context Estimating fees for XCM execution and sending has been an area with bad UX. The addition of the [XcmPaymentApi](paritytech#3607) exposed the necessary components to be able to estimate XCM fees correctly, however, that was not the full story. The `XcmPaymentApi` works for estimating fees only if you know the specific XCM you want to execute or send. This is necessary but most UIs want to estimate the fees for extrinsics, they don't necessarily know the XCM program that's executed by them. # Main addition A new runtime API is introduced, the `XcmDryRunApi`, that given an extrinsic, or an XCM program, returns its effects: - Execution result - Local XCM (in the case of an extrinsic) - Forwarded XCMs - List of events This API can be used on its own for dry-running purposes, for double-checking or testing, but it mainly shines when used in conjunction with the `XcmPaymentApi`. UIs can use these two APIs to estimate transfers. # How it works New tests are added to exemplify how to incorporate both APIs. There's a mock test just to make sure everything works under `xcm-fee-payment-runtime-api`. There's a real-world test using Westend and AssetHubWestend under `cumulus/parachains/integration-tests/emulated/tests/assets/asset-hub-westend/src/tests/xcm_fee_estimation.rs`. Added both a test for a simple teleport between chains and a reserve transfer asset between two parachains going through a reserve. The steps to follow: - Use `XcmDryRunApi::dry_run_extrinsic` to get local XCM program and forwarded messages - For each forwarded message - Use `XcmPaymentApi::query_delivery_fee` LOCALLY to get the delivery fees - Use `XcmPaymentApi::query_xcm_weight` ON THE DESTINATION to get the remote execution weight - (optional) Use `XcmPaymentApi::query_acceptable_payment_assets` ON THE DESTINATION to know on which assets the execution fees can be paid - Use `XcmPaymentApi::query_weight_to_asset_fee` ON THE DESTINATION to convert weight to the actual remote execution fees - Use `XcmDryRunApi::dry_run_xcm` ON THE DESTINATION to know if a new message will be forwarded, if so, continue # Dear reviewer The changes in this PR are grouped as follows, and in order of importance: - Addition of new runtime API - Definition, mock and simple tests: polkadot/xcm/xcm-fee-payment-runtime-api/* - Implemented on Westend, Asset Hub Westend and Penpal, will implement on every runtime in a following PR - Addition of a new config item to the XCM executor for recording xcms about to be executed - Definition: polkadot/xcm/xcm-executor/* - Implementation: polkadot/xcm/pallet-xcm/* - had to update all runtime xcm_config.rs files with `type XcmRecorder = XcmPallet;` - Addition of a new trait for inspecting the messages in queues - Definition: polkadot/xcm/xcm-builder/src/routing.rs - Implemented it on all routers: - ChildParachainRouter: polkadot/runtime/common/src/xcm_sender.rs - ParentAsUmp: cumulus/primitives/utility/src/lib.rs (piggybacked on implementation in cumulus/pallets/parachain-system/src/lib.rs) - XcmpQueue: cumulus/pallets/xcmp-queue/src/lib.rs - Bridge: bridges/modules/xcm-bridge-hub-router/src/lib.rs - More complicated and useful tests: - cumulus/parachains/integration-tests/emulated/tests/assets/asset-hub-westend/src/tests/xcm_fee_estimation.rs ## Next steps With this PR, Westend, AssetHubWestend, Rococo and AssetHubRococo have the new API. UIs can test on these runtimes to create better experiences around cross-chain operations. Next: - Add XcmDryRunApi to all system parachains - Integrate xcm fee estimation in all emulated tests - Get this on the fellowship runtimes --------- Co-authored-by: Adrian Catangiu <adrian@parity.io>
…aritytech#4634) Depends on paritytech#4621. Implemented the [`XcmPaymentApi`](paritytech#3607) and [`DryRunApi`](paritytech#3872) on all system parachains. More scenarios can be tested on both rococo and westend if all system parachains implement this APIs. The objective is for all XCM-enabled runtimes to implement them. After demonstrating fee estimation in a UI on the testnets, come the fellowship runtimes. Step towards paritytech#690.
The PR provides API for obtaining:
AssetId
s for message execution payment,AssetId
.It is meant to address an issue where one has to guess how much fee to pay for execution. Also, at the moment, a client has to guess which assets are acceptable for fee execution payment.
See the related issue #690.
With this API, a client is supposed to query the list of the supported asset IDs (in the XCM version format the client understands), weigh the XCM program the client wants to execute and convert the weight into one of the acceptable assets. Note that the client is supposed to know what program will be executed on what chains. However, having a small companion JS library for the pallet-xcm and xtokens should be enough to determine what XCM programs will be executed and where (since these pallets compose a known small set of programs).
An example of a client side code.