Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Custom RPC for Merkle Mountain Range pallet #8137

Merged
23 commits merged into from
Mar 10, 2021
Merged

Custom RPC for Merkle Mountain Range pallet #8137

23 commits merged into from
Mar 10, 2021

Conversation

tomusdrw
Copy link
Contributor

@tomusdrw tomusdrw commented Feb 16, 2021

Follow up on #7891.

Initially I thought that it would be possible to use state_call RPC to use MMR RuntimeApi and be able to generate proofs.

However, RPC-originated calls DO NOT have access to Offchain DB, hence proof generation is not really possible. I think it's unreasonable to blanket-enable Offchain DB access for all runtime calls over RPC, hence this PR introduces a custom RPC for MMR pallet, which performs RuntimeApi call but with OffchainDB access enabled (OffchainCall with default execution extensions enabled).

Unfortunately I had to extract OffchainDB access from OffchainWorker context and add yet another execution extension for this to work (I'm sorry @bkchr :))

Companion PR: paritytech/polkadot#2463

@tomusdrw tomusdrw added A0-please_review Pull request needs code review. B3-apinoteworthy C1-low PR touches the given topic and has a low impact on builders. labels Feb 16, 2021
@tomusdrw tomusdrw requested a review from bkchr March 3, 2021 10:49
@tomusdrw
Copy link
Contributor Author

tomusdrw commented Mar 3, 2021

@bkchr could you take a look at the externalities extension? Is there any other way how you'd see this being done?

@kianenigma kianenigma removed their request for review March 3, 2021 11:03
Copy link
Contributor

@adoerr adoerr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did focus on the Mmr part, which LGTM

let leaf: mmr::Leaf = leaf
.into_opaque_leaf()
.try_decode()
.ok_or(mmr::Error::Verify)?;
Mmr::verify_leaf(leaf, proof)
}

fn verify_proof_stateless(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps verify_ephemeral_proof in order to emphasize, that it is not based on persistent data?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought that stateless is enough emphasis :) "Ephemeral proof" sounds like a special kind of a proof to me, while here we just don't query the state, hence stateless.


impl EncodableOpaqueLeaf {
/// Convert a concrete leaf into encodable opaque version.
pub fn from_leaf<T: FullLeaf>(leaf: &T) -> Self {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this throw some kind of error, if T doesn't meet the encoding requirements?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, but there isn't really any requirements other than FullLeaf implementation. And the encoding is infallible.

So EncodableOpqaueLeaf and OpaqueLeaf serve a bit different purposes.
Both are meant for leaves that we don't necessarily know the concrete type of (i.e. we know it's a leaf, but we don't have the struct/type to encode it anywhere in the code).

The first one is a wrapper type to send such "unknown leaves" to the RuntimeApi or over RPC.
The second one is for MMR implementation - i.e. it implements FullLeaf, but can't implement Encode/Decode because of conflicting trait implementations.

from_leaf is just a convenience method to convert a known leaf type into its encodable and opaque form - so there is no extra requirements here.

Copy link
Contributor

@HCastano HCastano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The MMR code looks fine, but I don't know anything about the Offchain DB - so I'll leave that for someone else to review

frame/merkle-mountain-range/rpc/src/lib.rs Outdated Show resolved Hide resolved
frame/merkle-mountain-range/rpc/src/lib.rs Outdated Show resolved Hide resolved
/// Generate MMR proof for a leaf under given index.
fn generate_proof(leaf_index: u64) -> Result<(Leaf, Proof<Hash>), Error>;
#[skip_initialize_block]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious, why's this okay?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not only okay, it's actually required :) By default the RuntimeApi will run block initialisation, which triggers on_initialize hooks of every modules.
MMR module adds new leaf in on_initialize, so the proofs are generated for "the next block" instead of for the current one.

@tomusdrw tomusdrw requested a review from HCastano March 8, 2021 12:58
@bkchr
Copy link
Member

bkchr commented Mar 8, 2021

Sorry for the delay, I will review this pr tomorrow.

Copy link
Contributor

@kianenigma kianenigma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The runtime portion lgtm, probably you want to wait for one last review from Basti as well.

Copy link
Member

@bkchr bkchr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nitpicks, otherwise looks good.

frame/merkle-mountain-range/rpc/src/lib.rs Outdated Show resolved Hide resolved
@@ -84,6 +96,7 @@ impl ExtensionsFactory for () {
pub struct ExecutionExtensions<Block: traits::Block> {
strategies: ExecutionStrategies,
keystore: Option<SyncCryptoStorePtr>,
offchain_db: Option<Box<dyn DbExternalitiesFactory>>,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why isn't this just a generic ExtensionFactory?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generic parameter would propagate up to the Client type, and the actual implementation of the DbExternalitiesFactory is in sc_offchain, so I think it would be quite inconvenient to use.

client/offchain/src/api.rs Outdated Show resolved Hide resolved
@@ -119,19 +108,73 @@ impl<Storage: OffchainStorage> OffchainExt for Api<Storage> {
old_value: Option<&[u8]>,
new_value: &[u8],
) -> bool {
log::debug!(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For these kind of logging, I can really recommend tracing::debug!/* in the future. They provide some nice "structured logging"

General comment, a target would be nice.

frame/merkle-mountain-range/primitives/src/lib.rs Outdated Show resolved Hide resolved
frame/merkle-mountain-range/rpc/src/lib.rs Outdated Show resolved Hide resolved
frame/merkle-mountain-range/rpc/src/lib.rs Show resolved Hide resolved
primitives/runtime/src/offchain/storage.rs Outdated Show resolved Hide resolved
primitives/runtime/src/offchain/storage.rs Outdated Show resolved Hide resolved
@tomusdrw
Copy link
Contributor Author

bot merge

@ghost
Copy link

ghost commented Mar 10, 2021

Checks failed; merge aborted.

@tomusdrw
Copy link
Contributor Author

bot merge force

@ghost
Copy link

ghost commented Mar 10, 2021

Trying merge.

@ghost ghost merged commit 3adefdc into master Mar 10, 2021
@ghost ghost deleted the td-mmr-custom branch March 10, 2021 15:28
This pull request was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A0-please_review Pull request needs code review. C1-low PR touches the given topic and has a low impact on builders.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants