Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework Subxt API to support offline and dynamic transactions #593

Merged
merged 76 commits into from
Aug 8, 2022
Merged
Show file tree
Hide file tree
Changes from 68 commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
298384d
WIP API changes
jsdw Jun 30, 2022
f1f65d9
debug impls
jsdw Jul 1, 2022
bee829b
Get main crate compiling with first round of changes
jsdw Jul 1, 2022
5335b3a
Some tidy up
jsdw Jul 1, 2022
f72f78d
Add WithExtrinsicParams, and have SubstrateConfig + PolkadotConfig, n…
jsdw Jul 1, 2022
be46e35
move transaction into extrinsic folder
jsdw Jul 1, 2022
90f4589
Add runtime updates back to OnlineClient
jsdw Jul 1, 2022
c2c9e9b
rework to be 'client first' to fit better with storage + events
jsdw Jul 4, 2022
02f4eae
add support for events to Client
jsdw Jul 4, 2022
7a609b8
tidy dupe trait bound
jsdw Jul 4, 2022
31b3eef
Wire storage into client, but need to remove static reliance
jsdw Jul 5, 2022
0e67e8a
various tidy up and start stripping codegen to remove bits we dont ne…
jsdw Jul 5, 2022
243b452
First pass updating calls and constants codegen
jsdw Jul 5, 2022
955e4ae
WIP storage client updates
jsdw Jul 7, 2022
bb76ffc
Merge branch 'master' into jsdw-wip-api-changes
jsdw Jul 7, 2022
7ab22aa
First pass migrated runtime storage over to new format
jsdw Jul 7, 2022
dc1c422
pass over codegen to generate StorageAddresses and throw other stuff out
jsdw Jul 7, 2022
e9070b0
don't need a Call trait any more
jsdw Jul 8, 2022
9b0a27c
shuffle things around a bit
jsdw Jul 8, 2022
7a939b7
Various proc_macro fixes to get 'cargo check' working
jsdw Jul 8, 2022
2d0cb0a
organise what's exposed from subxt
jsdw Jul 8, 2022
700c77e
Get first example working; balance_transfer_with_params
jsdw Jul 8, 2022
5e68da2
get balance_transfer example compiling
jsdw Jul 8, 2022
9d55c43
get concurrent_storage_requests.rs example compiling
jsdw Jul 8, 2022
6e1c5e2
get fetch_all_accounts example compiling
jsdw Jul 8, 2022
fff5b55
get a bunch more of the examples compiling
jsdw Jul 8, 2022
e452c75
almost get final example working; type mismatch to look into
jsdw Jul 8, 2022
00d51d1
wee tweaks
jsdw Jul 8, 2022
2fed937
move StorageAddress to separate file
jsdw Jul 11, 2022
6657dbf
pass Defaultable/Iterable info to StorageAddress in codegen
jsdw Jul 11, 2022
6621dcf
fix storage validation ne, and partial run through example code
jsdw Jul 12, 2022
e1af0f7
Remove static iteration and strip a generic param from everything
jsdw Jul 12, 2022
778fb15
fix doc tests in subxt crate
jsdw Jul 13, 2022
180ec97
update test utils and start fixing frame tests
jsdw Jul 13, 2022
374b68d
fix frame staking tests
jsdw Jul 13, 2022
89c2cff
fix the rest of the test compile issues, Borrow on storage values
jsdw Jul 14, 2022
71ca97a
cargo fmt
jsdw Jul 14, 2022
055b3dc
remove extra logging during tests
jsdw Jul 14, 2022
cef039a
Appease clippy and no more need for into_iter on events
jsdw Jul 14, 2022
eb7ed42
cargo fmt
jsdw Jul 14, 2022
9d8a2db
fix dryRun tests by waiting for blocks
jsdw Jul 15, 2022
bedf3d2
wait for blocks instead of sleeping or other test hacks
jsdw Jul 15, 2022
815e100
cargo fmt
jsdw Jul 15, 2022
a00b011
Fix doc links
jsdw Jul 15, 2022
470ab2d
Traitify StorageAddress
jsdw Jul 15, 2022
cf1bf9d
remove out-of-date doc comments
jsdw Jul 15, 2022
c319b9d
optimise decoding storage a little
jsdw Jul 18, 2022
dce6b61
cleanup tx stuff, trait for TxPayload, remove Err type param and deco…
jsdw Jul 18, 2022
3a1a766
clippy fixes
jsdw Jul 18, 2022
6d03b3f
fix doc links
jsdw Jul 18, 2022
00c4d7a
fix doc example
jsdw Jul 18, 2022
be89c65
constant address trait for consistency
jsdw Jul 18, 2022
fb461e6
fix a typo and remove EncodeWithMetadata stuff
jsdw Jul 18, 2022
01847fe
Put EventDetails behind a proper interface and allow decoding into to…
jsdw Jul 18, 2022
4b818f0
fix docs
jsdw Jul 18, 2022
1b7d023
tweak StorageAddress docs
jsdw Jul 18, 2022
ff5f20a
re-export StorageAddress at root for consistency
jsdw Jul 18, 2022
116f7f4
fix clippy things
jsdw Jul 18, 2022
ce0148d
Add support for dynamic values
jsdw Jul 19, 2022
a2d400b
fix double encoding of storage map key after refactor
jsdw Jul 20, 2022
644bd6e
clippy fix
jsdw Jul 20, 2022
1cfef3b
Fixes and add a dynamic usage example (needs new scale_value release)
jsdw Jul 20, 2022
12e483c
bump scale_value version
jsdw Jul 20, 2022
3e62bce
cargo fmt
jsdw Jul 20, 2022
09520b6
Tweak event bits
jsdw Jul 21, 2022
98af08a
cargo fmt
jsdw Jul 21, 2022
623911d
Add a test and bump scale-value to 0.4.0 to support this
jsdw Jul 22, 2022
b3287c8
remove unnecessary vec from dynamic example
jsdw Jul 22, 2022
b6b8f9d
Various typo/grammar fixes
jsdw Jul 27, 2022
ddcaf75
Address PR nits
jsdw Jul 27, 2022
e8ed053
Undo accidental rename in changelog
jsdw Aug 5, 2022
692f1da
Small PR nits/tidyups
jsdw Aug 5, 2022
427c308
fix tests; codegen change against latest substrate
jsdw Aug 5, 2022
4596327
tweak storage address util names
jsdw Aug 5, 2022
a649841
move error decoding to DecodeError and expose
jsdw Aug 8, 2022
c0c0c2c
impl some basic traits on the extrinsic param builder
jsdw Aug 8, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ parameters (such as mortality and tip payment) to be customized. See `examples/b
small usage example.

If you're targeting a node which involves custom additional and extra transaction data, you'll need to implement the
trait `subxt::extrinsic::ExtrinsicParams`, which determines the parameters that can be provided to `sign_and_submit`, as
trait `subxt::tx::ExtrinsicParams`, which determines the parameters that can be provided to `sign_and_submit`, as
well as how to encode these into the "additional" and "extra" data needed for a transaction. Have a look at
`subxt/src/extrinsic/params.rs` for the trait definition and Substrate/Polkadot implementations. The aim with this change
is to make it easier to customise this for your own chains, and provide a simple way to provide values at runtime.
Expand Down
Binary file modified artifacts/polkadot_metadata.scale
Binary file not shown.
123 changes: 43 additions & 80 deletions codegen/src/api/calls.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,31 +22,8 @@ use quote::{
};
use scale_info::form::PortableForm;

/// Generate calls from the provided pallet's metadata.
///
/// The function creates a new module named `calls` under the pallet's module.
/// ```ignore
/// pub mod PalletName {
/// pub mod calls {
/// ...
/// }
/// }
/// ```
///
/// The function generates the calls as rust structs that implement the `subxt::Call` trait
/// to uniquely identify the call's identity when creating the extrinsic.
///
/// ```ignore
/// pub struct CallName {
/// pub call_param: type,
/// }
/// impl ::subxt::Call for CallName {
/// ...
/// }
/// ```
///
/// Calls are extracted from the API and wrapped into the generated `TransactionApi` of
/// each module.
/// Generate calls from the provided pallet's metadata. Each call returns a `StaticTxPayload`
/// that can be passed to the subxt client to submit/sign/encode.
///
/// # Arguments
///
Expand Down Expand Up @@ -76,35 +53,42 @@ pub fn generate_calls(
let (call_structs, call_fns): (Vec<_>, Vec<_>) = struct_defs
.iter_mut()
.map(|(variant_name, struct_def)| {
let (call_fn_args, call_args): (Vec<_>, Vec<_>) =
match struct_def.fields {
CompositeDefFields::Named(ref named_fields) => {
named_fields
.iter()
.map(|(name, field)| {
let fn_arg_type = &field.type_path;
let call_arg = if field.is_boxed() {
quote! { #name: ::std::boxed::Box::new(#name) }
} else {
quote! { #name }
};
(quote!( #name: #fn_arg_type ), call_arg)
})
.unzip()
}
CompositeDefFields::NoFields => Default::default(),
CompositeDefFields::Unnamed(_) =>
abort_call_site!(
"Call variant for type {} must have all named fields",
call.ty.id()
)
};
let (call_fn_args, call_args): (Vec<_>, Vec<_>) = match struct_def.fields {
CompositeDefFields::Named(ref named_fields) => {
named_fields
.iter()
.map(|(name, field)| {
let fn_arg_type = &field.type_path;
let call_arg = if field.is_boxed() {
quote! { #name: ::std::boxed::Box::new(#name) }
} else {
quote! { #name }
};
(quote!( #name: #fn_arg_type ), call_arg)
})
.unzip()
}
CompositeDefFields::NoFields => Default::default(),
CompositeDefFields::Unnamed(_) => {
abort_call_site!(
"Call variant for type {} must have all named fields",
call.ty.id()
)
}
};

let pallet_name = &pallet.name;
let call_name = &variant_name;
let struct_name = &struct_def.name;
let call_hash = subxt_metadata::get_call_hash(metadata, pallet_name, call_name)
.unwrap_or_else(|_| abort_call_site!("Metadata information for the call {}_{} could not be found", pallet_name, call_name));
let call_hash =
subxt_metadata::get_call_hash(metadata, pallet_name, call_name)
.unwrap_or_else(|_| {
abort_call_site!(
"Metadata information for the call {}_{} could not be found",
pallet_name,
call_name
)
});

let fn_name = format_ident!("{}", variant_name.to_snake_case());
// Propagate the documentation just to `TransactionApi` methods, while
Expand All @@ -113,29 +97,19 @@ pub fn generate_calls(
// The call structure's documentation was stripped above.
let call_struct = quote! {
#struct_def

impl ::subxt::Call for #struct_name {
const PALLET: &'static str = #pallet_name;
const FUNCTION: &'static str = #call_name;
}
};
let client_fn = quote! {
#docs
pub fn #fn_name(
&self,
#( #call_fn_args, )*
) -> Result<::subxt::SubmittableExtrinsic<'a, T, X, #struct_name, DispatchError, root_mod::Event>, ::subxt::BasicError> {
let runtime_call_hash = {
let locked_metadata = self.client.metadata();
let metadata = locked_metadata.read();
metadata.call_hash::<#struct_name>()?
};
if runtime_call_hash == [#(#call_hash,)*] {
let call = #struct_name { #( #call_args, )* };
Ok(::subxt::SubmittableExtrinsic::new(self.client, call))
} else {
Err(::subxt::MetadataError::IncompatibleMetadata.into())
}
) -> ::subxt::tx::StaticTxPayload<#struct_name> {
::subxt::tx::StaticTxPayload::new(
#pallet_name,
#call_name,
#struct_name { #( #call_args, )* },
[#(#call_hash,)*]
)
}
};
(call_struct, client_fn)
Expand All @@ -155,20 +129,9 @@ pub fn generate_calls(

#( #call_structs )*

pub struct TransactionApi<'a, T: ::subxt::Config, X> {
client: &'a ::subxt::Client<T>,
marker: ::core::marker::PhantomData<X>,
}

impl<'a, T, X> TransactionApi<'a, T, X>
where
T: ::subxt::Config,
X: ::subxt::extrinsic::ExtrinsicParams<T>,
{
pub fn new(client: &'a ::subxt::Client<T>) -> Self {
Self { client, marker: ::core::marker::PhantomData }
}
pub struct TransactionApi;

impl TransactionApi {
#( #call_fns )*
}
}
Expand Down
27 changes: 8 additions & 19 deletions codegen/src/api/constants.rs
Original file line number Diff line number Diff line change
Expand Up @@ -63,17 +63,12 @@ pub fn generate_constants(

quote! {
#( #[doc = #docs ] )*
pub fn #fn_name(&self) -> ::core::result::Result<#return_ty, ::subxt::BasicError> {
let locked_metadata = self.client.metadata();
let metadata = locked_metadata.read();
if metadata.constant_hash(#pallet_name, #constant_name)? == [#(#constant_hash,)*] {
let pallet = metadata.pallet(#pallet_name)?;
let constant = pallet.constant(#constant_name)?;
let value = ::subxt::codec::Decode::decode(&mut &constant.value[..])?;
Ok(value)
} else {
Err(::subxt::MetadataError::IncompatibleMetadata.into())
}
pub fn #fn_name(&self) -> ::subxt::constants::StaticConstantAddress<::subxt::metadata::DecodeStaticType<#return_ty>> {
::subxt::constants::StaticConstantAddress::new(
#pallet_name,
#constant_name,
[#(#constant_hash,)*]
)
}
}
});
Expand All @@ -82,15 +77,9 @@ pub fn generate_constants(
pub mod constants {
use super::#types_mod_ident;

pub struct ConstantsApi<'a, T: ::subxt::Config> {
client: &'a ::subxt::Client<T>,
}

impl<'a, T: ::subxt::Config> ConstantsApi<'a, T> {
pub fn new(client: &'a ::subxt::Client<T>) -> Self {
Self { client }
}
pub struct ConstantsApi;

impl ConstantsApi {
#(#constant_fns)*
}
}
Expand Down
Loading