Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework Subxt API to support offline and dynamic transactions #593

Merged
merged 76 commits into from
Aug 8, 2022

Conversation

jsdw
Copy link
Collaborator

@jsdw jsdw commented Jul 4, 2022

  • Separate client logic from transaction building logic.
  • Support an online and offline client; any functionality which doesn't require an online client can then be used with an offline one instead, for purely offline workflows.
  • Once separated, it adds basic support for creating dynamic transactions and such where appropriate (this might land in this PR in an early stage to be built on further).

Tips for Reviewers

The best place to start is probably by looking at the examples and tests to see what using the new APIs looks like. Inparticular, a new dynamic_queries.rs example mirrors some of the other examples but shows how you'd use purely runtime Values and no codegen to do things.

Next, the client folder is the API entry point, and from there you can look at the individual TxClient, StorageClient, ConstantsClient and EventClient to see what methods are available for each and what data is accepted.

These clients generally accept Address/Payload traits, for which we have an implementation used by the static codegen and another used for submitting dynamic Values. This means that the actual API is tied to neither approach.

From here, the constants, events. storage and tx folders contain the respective logic for each area of the API. storage/storage_address.rs, ts/tx_payload.rs, constants/constant_address.rs have the traits and static/dynamic implementations for submitting things to the API.

A dynamic.rs file/module exists as a shortcut to all things dynamic (dynamic::tx(), dynamic::storage()) etc. These are just aliases to the relevant dynamic function in each location, so that it's easy to find everything in one place.

This closes #581, closes #406, closes #592 and closes #582

jsdw added 30 commits June 30, 2022 18:35
.metadata()
.constant_hash(address.pallet_name(), address.constant_name())?;
if actual_hash != expected_hash {
return Err(MetadataError::IncompatibleMetadata.into())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When this fails it would be super useful if the actual pallet constant name is embedded in the error.

This is hard to debug otherwise.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeahh Andrew raised this too; it's a separate issue somewhere so I didn't want to think about it in this PR!

@@ -273,9 +294,10 @@ impl PalletMetadata {
/// Metadata for specific events.
#[derive(Clone, Debug)]
pub struct EventMetadata {
pallet: String,
pallet: Arc<str>,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you explain why this is changed to an Arc?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah sure; Basically because the pallet name is duplicated across every event in a pallet ,so instead of clone()int a String to store with every event I just stuck it behind an arc so that it's an increment instead.

Probably worth a comment in the code :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(comment added)

@jsdw
Copy link
Collaborator Author

jsdw commented Aug 5, 2022

Ah, the checks ran foul of some recent substrate build change that we picked up on a couple of nights ago; guess I'll have to fix it now!


/// Return the root of a given [`StorageAddress`]: hash the pallet name and entry name
/// and append those bytes to the output.
pub fn storage_address_root_bytes<Address: StorageAddress>(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the naming of this function is a bit confusing.

I would prefer some like write_storage_address_root or something like that.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch; these names were rubbish! I changed them both :)

Copy link
Member

@niklasad1 niklasad1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a really high quality PR, nicely done

&self,
call: &Call,
signer: &(dyn Signer<T> + Send + Sync),
other_params: <T::ExtrinsicParams as ExtrinsicParams<T::Index, T::Hash>>::OtherParams,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we borrow other_params too? Consistent with other params and doesn't require Clone for copying instances.

Copy link
Collaborator Author

@jsdw jsdw Aug 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had a go at this, and where I landed was:

  • If we can accept a reference to OtherParams when building ExtrinsicParams that'd be nice, and probably reflect how it's used the best (and then passing references in would be ideal). But it turns out to be a pain to do actually, so the complexity didn't feel worth while.
  • If we can't, then we need to do a .clone() call internally, and I'd rather push that to the user (and then impls can be copy potentially and it's no biggie, or they can't and better the user has the chance to pass ownership rather than force a clone.

So for now I'd suggest leaving it as it is, but we might be able to revisit and improve!

subxt/src/client/offline_client.rs Show resolved Hide resolved
/// Events decoding error.
#[error("Events decoding error: {0}")]
EventsDecoding(#[from] DecodeError),
Runtime(DispatchError),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes much nicer 👌

tracing::warn!("Can't decode error: sp_runtime::DispatchError::Module details do not match known information");
return DispatchError::Other(bytes.to_vec())
}
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

.unwrap();

api.rpc()
.dry_run(signed_extrinsic.encoded(), None)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome!!!!!!!

Copy link

@shekohex shekohex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great Job!! Really looking forward to use this release!

// Attempt to decode a DispatchError, returning either the ModuleError it decodes to,
// along with additional details on the error, or returning the raw bytes to work with
// downstream if we couldn't decode it.
fn decode_dispatch_error(metadata: &Metadata, bytes: &[u8]) -> DispatchError {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may need access to this, see: https://github.com/paritytech/cargo-contract/blob/master/src/cmd/extrinsics/mod.rs#L246.

Essentially I want to decode the DispatchError returned from an RPC call rather than this extrinsic.

Could we hang it off Metadata? Or DispatchError?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ooh yup, sounds very reasonable to me! I stuck it onto DispatchError

Copy link
Contributor

@ascjones ascjones left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bravo! Really like this new API 👍

@jsdw jsdw merged commit e48f0e3 into master Aug 8, 2022
@jsdw jsdw deleted the jsdw-wip-api-changes branch August 8, 2022 10:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
6 participants