Extrinsic v5 definition and specification #124

georgepisaltu · 2024-10-18T19:33:39Z

No description provided.

Signed-off-by: georgepisaltu <george.pisaltu@parity.io>

carlosala

Nice starting point! Generally I'd try to keep the RFC as far away from the implementation as possible.
Great work George! 👏🏻

carlosala · 2024-10-21T14:04:06Z

text/0124-extrinsic-version-5.md

+The introduction of `General` transactions allows the authorization of any and all origins through
+extensions. This means that, with the appropriate extension, `General` transactions are capable of
+replicating the same behavior present day v4 `Signed` transactions. Specifically for Polkadot
+chains, an example implementation for such an extension is
+[`VerifySignature`](https://github.com/paritytech/polkadot-sdk/tree/master/substrate/frame/verify-signature),
+introduced in the Transaction Extension
+[PR3685](https://github.com/paritytech/polkadot-sdk/pull/3685). Other extensions can be inserted
+into the extension pipeline to authorize different custom origins. Therefore, a `Signed` extrinsic
+variant is redundant to a `General` one strictly in terms of functionality available to users and
+would eventually need to be deprecated and removed.


Besides some wording suggestions, I don't think we should introduce links to polkadot-sdk repo. The idea of authorization through transaction extensions is already clear without the example IMO.

Suggested change

The introduction of `General` transactions allows the authorization of any and all origins through

extensions. This means that, with the appropriate extension, `General` transactions are capable of

replicating the same behavior present day v4 `Signed` transactions. Specifically for Polkadot

chains, an example implementation for such an extension is

[`VerifySignature`](https://github.com/paritytech/polkadot-sdk/tree/master/substrate/frame/verify-signature),

introduced in the Transaction Extension

[PR3685](https://github.com/paritytech/polkadot-sdk/pull/3685). Other extensions can be inserted

into the extension pipeline to authorize different custom origins. Therefore, a `Signed` extrinsic

variant is redundant to a `General` one strictly in terms of functionality available to users and

would eventually need to be deprecated and removed.

The introduction of `General` transactions allows the authorization of any origin through transaction extensions. This means that, with the appropriate extension, `General` transactions can replicate the same behaviour in present-day v4 `Signed` transactions. Therefore, a `Signed` extrinsic variant is redundant to a `General` one strictly in terms of user functionality and could eventually be deprecated and removed.

This is in the explanation section and I think it adds valuable context to the discussion. I appreciate that you're familiar with the idea of authorization through transaction extensions, but I think it's still a novel topic and the example is useful. However, if others disagree, I will remove it.

text/0124-extrinsic-version-5.md

carlosala · 2024-10-21T14:43:37Z

text/0124-extrinsic-version-5.md

+As stated before, [PR3685](https://github.com/paritytech/polkadot-sdk/pull/3685) comes with a
+Transaction Extension which replicates the current `Signed` transactions in v5 extrinsics, namely
+[`VerifySignature`](https://github.com/paritytech/polkadot-sdk/tree/master/substrate/frame/verify-signature).
+This extension leverages the new inherited implication functionality introduced in
+`TransactionExtension` and creates a payload to be signed using the data of all extensions after
+itself in the extension pipeline. In order to run a transaction with a signed origin, a user must
+create the transaction with an instance of the extension which provides a signature. Alternatively,
+if users want to use some other origin, they should create the transaction with this particular
+extension disabled. More on this behavior in the extension documentation. This extension can be
+configured to accept a `MultiSignature`, which makes it compatible with all signature types
+currently used in Polkadot.


Same as before, this should not explain HOW polkadot-sdk will use the interface introduced in this RFC, but rather the shape of it and its goals. This paragraph adds very little value on that regard IMO.

The goal of it is to explain how to use the new v5 format to achieve provide the same functionality as signed transactions. I then use the example in polkadot-sdk to show this. The previous phrasing was a bit off and the intent wasn't clear, hopefully now it's better.

I don't think it's wrong to have links to polkadot-sdk or reference stuff from there. polkadot-sdk is used right now in Polkadot and Kusama runtimes, and quite extensively I might add. Even if we might not want to use it to define future functionality, it's inevitable that describing current functionality will point to polkadot-sdk. The upstream/consumer dependency is already there, I think making the RFC overly general by avoiding specific examples doesn't help in any way, it just makes it harder to read and understand. This does not in any way mean that the example pulled from polkadot-sdk is the only accepted implementation of specified functionality, it just helps me not unfurl a lot of code in this file.

That said, if others consider the examples are not useful, I will remove them.

carlosala · 2024-10-21T17:15:08Z

text/0124-extrinsic-version-5.md

+To generate the payload to be signed:
+
+1. The extension version byte, call, extension and extension implicit should be encoded;
+2. The result of the encoding should then be hashed using the `BLAKE2_256` hasher;
+3. The result of the hash should then be signed with the signature type specified in the extension definition.
+
+```rust
+// Step 1: encode the bytes
+let encoded = (extension_version_byte, call, transaction_extension, transaction_extension_implicit).encode();
+// Step 2: hash them
+let payload = blake2_256(&encoded[..]);
+// Step 3: sign the payload
+let signature = keyring.sign(&payload[..]);
+```


This will most likely depend on which origin and kind of signature is used. For instance, ZK circuits might have different levels of proofs and might not be signing the whole payload. This should go to another RFC, which defines every particular VerifySignature-like extension.

Yes, this is just an example on how to achieve Signed functionality in v5. I agree that the "chosen" way of providing this functionality in the actual runtimes should be a different RFC, dependent on this one.

However, there should be only one VerifySignature-like extension used to replicate this signed behavior. There is no need to have multiple extensions for this purpose. For custom origins, users can create infinite variations of an authorization extension and that doesn't need an RFC.

Yep, but anyways this should go on a different RFC. I agree that VerifySignature-extension requires its own RFC, yes, but this has nothing to do with extrnsics v5 RFC.

Again, VerifySignature is just an example, and also very clearly labeled as an example in the latest revision of the RFC. Because of the explanation above and the fact that it's not obvious from the start what the extension pipeline should look like for v5 extrinsics, I think the example is useful.

I think the example is useful, and since it's also explicitly labeled as an example I see no reason to remove it

carlosala · 2024-10-21T17:29:20Z

text/0124-extrinsic-version-5.md

+The metadata will have to accommodate two distinct extrinsic format versions at a given point in
+time in order to provide the new functionality in a non-breaking way for users and tooling.


This is not a drawback IMO. Metadata v15 should show v4 and metadata v16 and ahead have a vector of extrinsic versions.

It is something extra to support in the metadata for both the runtime and users, is this not a drawback?

it is both a drawback and an improvement - the new metadata support is an improvement, but having to do this enhancement to the metadata is a drawback to this RFC 😛

maybe add another line explicitly calling out that adding this metadata enhancement is ultimately a good thing that should be useful for potential future scenarios too

Added the explanation.

carlosala · 2024-10-21T17:32:57Z

text/0124-extrinsic-version-5.md

+This change makes the authorization through signatures configurable by runtime devs in version 5
+extrinsics, as opposed to version 4 where the signing payload algorithm and signatures were
+hardcoded. This moves the responsibility of ensuring proper authentication through
+`TransactionExtension` to the runtime devs, but a sensible default which closely resembles the
+present day behavior will be provided in `VerifySignature`.


This is not true. Signature schemes and addresses were configurable by runtime devs through rust generics. For example, Moonbeam uses only ECDSA signatures with EVM-like addresses.
Besides that, I wouldn't mention VerifySignature since it should be RFC-ed anyways.

Your statement about configurable signature schemes is true. However, I still consider my statement to be true because:

The signing payload generation algorithm was hardcoded; this is not the case anymore as any extension can take the inherited implication and add or subtract any data to it and mutate it in any way (such as hashing it - or not) before actually creating a signature.

There are now multiple ways of ending up with a Signed origin variant, with arbitrary logic in any TransactionExtension being able to authorize that origin; before, a user HAD to provide a transaction signed by a specific account.

All of this static logic is now moved to extensions. The extensions receive the inherited implication, the generation of which is still hardcoded and handled in this RFC, but is not in any way mandatory to be used in any signing scheme.

I'd agree though that the phrasing isn't clear, but I'm not sure how to improve it.

I think the phrasing is fine as it is. The point is to highlight the increase in configurability, which it does.

text/0124-extrinsic-version-5.md

Signed-off-by: georgepisaltu <george.pisaltu@parity.io>

acatangiu

Looks good! 👍

text/0124-extrinsic-version-5.md

acatangiu · 2024-11-01T14:43:47Z

text/0124-extrinsic-version-5.md

+A quick visualization of the encoding:
+
+- `Bare` extrinsics: `(extrinsic_encoded_len, 0b0000_0101, call)`
+- `General` transactions: `(extrinsic_encoded_len, , 0b0100_0101, extension_version_byte, extension, call)`


I thought the component is called extensions?

Suggested change

- `General` transactions: `(extrinsic_encoded_len, , 0b0100_0101, extension_version_byte, extension, call)`

- `General` transactions: `(extrinsic_encoded_len, , 0b0100_0101, extensions_version_byte, extensions, call)`

The extension itself is usually a tuple of multiple extensions, generally referred to as the extension pipeline. Technically it's only one extension, the TxExtension commonly defined in runtimes, but that is always a tuple of extensions like CheckNonce, CheckWeight, ChargeTransactionPayment etc., so it would be only one extension version, as it is the version of the tuple, but there are multiple extensions in the pipeline.

acatangiu · 2024-11-01T14:48:27Z

text/0124-extrinsic-version-5.md

+with a provided signature. Alternatively, if users want to use some other origin, they should create
+the transaction with this particular extension disabled.


dumb Q: would it not be possible to use a set of extensions that ultimately behave differently than classic "Signed Origins Extension", but still have one extension responsible for authorizing Signed origins within this set?

Respectively, I'm not sure this "alternative" sentence is correct.

Suggested change

with a provided signature. Alternatively, if users want to use some other origin, they should create

the transaction with this particular extension disabled.

with a provided signature.

To your question, yes it is entirely possible and, in fact, what I expect to be implemented in most runtimes. The "alternative" wasn't helpful so I removed it.

text/0124-extrinsic-version-5.md

acatangiu · 2024-11-01T14:56:28Z

text/0124-extrinsic-version-5.md

+The metadata will have to accommodate two distinct extrinsic format versions at a given point in
+time in order to provide the new functionality in a non-breaking way for users and tooling.


it is both a drawback and an improvement - the new metadata support is an improvement, but having to do this enhancement to the metadata is a drawback to this RFC 😛

maybe add another line explicitly calling out that adding this metadata enhancement is ultimately a good thing that should be useful for potential future scenarios too

acatangiu · 2024-11-01T14:59:32Z

text/0124-extrinsic-version-5.md

+This change makes the authorization through signatures configurable by runtime devs in version 5
+extrinsics, as opposed to version 4 where the signing payload algorithm and signatures were
+hardcoded. This moves the responsibility of ensuring proper authentication through
+`TransactionExtension` to the runtime devs, but a sensible default which closely resembles the
+present day behavior will be provided in `VerifySignature`.


I think the phrasing is fine as it is. The point is to highlight the increase in configurability, which it does.

text/0124-extrinsic-version-5.md

Signed-off-by: georgepisaltu <george.pisaltu@parity.io>

georgepisaltu added 4 commits October 18, 2024 19:25

initial extrinsic v5 RFC

380bdbb

Signed-off-by: georgepisaltu <george.pisaltu@parity.io>

links and formatting

01bc758

Signed-off-by: georgepisaltu <george.pisaltu@parity.io>

Fix title

f2f4f19

Signed-off-by: georgepisaltu <george.pisaltu@parity.io>

Rename RFC

dd9e353

Signed-off-by: georgepisaltu <george.pisaltu@parity.io>

gui1117 approved these changes Oct 20, 2024

View reviewed changes

georgepisaltu mentioned this pull request Oct 21, 2024

FRAME: Reintroduce TransactionExtension as a replacement for SignedExtension paritytech/polkadot-sdk#3685

Merged

carlosala suggested changes Oct 21, 2024

View reviewed changes

anaelleltd added the Proposed Is awaiting 3 formal reviews. label Oct 28, 2024

Rephrasing and other fixes

e13f615

Signed-off-by: georgepisaltu <george.pisaltu@parity.io>

acatangiu approved these changes Nov 1, 2024

View reviewed changes

Rephrasing and formatting

0ac87df

Signed-off-by: georgepisaltu <george.pisaltu@parity.io>

jasl mentioned this pull request Nov 8, 2024

Support Substrate extrinsic v5 subsquid/squid-sdk#348

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Extrinsic v5 definition and specification #124

Extrinsic v5 definition and specification #124

georgepisaltu commented Oct 18, 2024

carlosala left a comment

carlosala Oct 21, 2024

georgepisaltu Oct 28, 2024

carlosala Oct 21, 2024

georgepisaltu Oct 28, 2024

carlosala Oct 21, 2024

georgepisaltu Oct 28, 2024

carlosala Nov 6, 2024

georgepisaltu Nov 6, 2024

acatangiu Nov 6, 2024

carlosala Oct 21, 2024

georgepisaltu Oct 28, 2024

acatangiu Nov 1, 2024

georgepisaltu Nov 6, 2024

carlosala Oct 21, 2024

georgepisaltu Oct 28, 2024

acatangiu Nov 1, 2024

acatangiu left a comment

acatangiu Nov 1, 2024

georgepisaltu Nov 6, 2024

acatangiu Nov 1, 2024

georgepisaltu Nov 6, 2024

acatangiu Nov 1, 2024

acatangiu Nov 1, 2024

		The metadata will have to accommodate two distinct extrinsic format versions at a given point in
		time in order to provide the new functionality in a non-breaking way for users and tooling.

	- `General` transactions: `(extrinsic_encoded_len, , 0b0100_0101, extension_version_byte, extension, call)`
	- `General` transactions: `(extrinsic_encoded_len, , 0b0100_0101, extensions_version_byte, extensions, call)`

		with a provided signature. Alternatively, if users want to use some other origin, they should create
		the transaction with this particular extension disabled.

	with a provided signature. Alternatively, if users want to use some other origin, they should create
	the transaction with this particular extension disabled.
	with a provided signature.

Extrinsic v5 definition and specification #124

Are you sure you want to change the base?

Extrinsic v5 definition and specification #124

Conversation

georgepisaltu commented Oct 18, 2024

carlosala left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

acatangiu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment