Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for keyring in runtimes #2044

Merged
merged 69 commits into from
Mar 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
814b597
wip: sp-core::core
michalkucharczyk Oct 25, 2023
69f2168
wip: sp-keyring
michalkucharczyk Oct 25, 2023
f6a5ec0
wip: test-runtime
michalkucharczyk Oct 25, 2023
a6ee754
wip: keyring 2
michalkucharczyk Oct 26, 2023
a79bfb6
wip: core 2
michalkucharczyk Oct 26, 2023
d6f1530
wip: application-crypto 2
michalkucharczyk Oct 26, 2023
0cbfefb
fixes
michalkucharczyk Oct 27, 2023
bbc43c2
Merge remote-tracking branch 'origin/master' into mku-keyring-in-runt…
michalkucharczyk Oct 30, 2023
9650d42
Merge remote-tracking branch 'origin/master' into mku-keyring-in-runt…
michalkucharczyk Oct 30, 2023
cf11ccb
substrate-bip39 bumped to 0.4.5
michalkucharczyk Oct 30, 2023
7e941ec
use Mnemonic::parse_in_normalized in no-std
michalkucharczyk Oct 30, 2023
d3b803f
Cargo.lock updated
michalkucharczyk Oct 30, 2023
3f1e289
Merge remote-tracking branch 'origin/master' into mku-keyring-in-runt…
michalkucharczyk Oct 30, 2023
a5bc236
keyring: features fix
michalkucharczyk Oct 30, 2023
8dc7d71
".git/.scripts/commands/fmt/fmt.sh"
Oct 30, 2023
1c0b294
bandersnatch fixes
michalkucharczyk Oct 31, 2023
b26e654
compile feature removed
michalkucharczyk Oct 31, 2023
67de13d
keyring: fixes
michalkucharczyk Oct 31, 2023
0763902
Cargo.lock
michalkucharczyk Oct 31, 2023
c077713
keyring: sp_std cleanup
michalkucharczyk Oct 31, 2023
3ba69e6
debug added
michalkucharczyk Oct 31, 2023
6fc3c65
unicode-normalization also supported in no-std
michalkucharczyk Nov 6, 2023
d64694f
use fork of rust-bip39
michalkucharczyk Nov 6, 2023
7b763d3
Cargo.lock updated
michalkucharczyk Nov 6, 2023
18db2bf
more fixes
michalkucharczyk Nov 6, 2023
5ad0403
Merge remote-tracking branch 'origin/master' into mku-keyring-in-runt…
michalkucharczyk Nov 20, 2023
d77e3cc
address_uri support in no-std, no-full-crypto
michalkucharczyk Nov 20, 2023
1b57b65
verify available in non-full-crypto
michalkucharczyk Nov 20, 2023
06d53e4
application-crypto: app_crypto_pair_common added
michalkucharczyk Nov 20, 2023
d0bf8bb
cargo.lock
michalkucharczyk Nov 20, 2023
5783730
Update substrate/primitives/core/Cargo.toml
michalkucharczyk Nov 20, 2023
7d55ce2
Update substrate/frame/support/procedural/Cargo.toml
michalkucharczyk Nov 20, 2023
617593e
Update substrate/test-utils/runtime/Cargo.toml
michalkucharczyk Nov 20, 2023
9b03603
proc-macro-crate version reverted
michalkucharczyk Nov 20, 2023
89a06e2
zepter fix
michalkucharczyk Nov 20, 2023
bec4955
CI features checks added
michalkucharczyk Nov 20, 2023
2c2f6d1
Merge remote-tracking branch 'origin/master' into mku-keyring-in-runt…
michalkucharczyk Dec 11, 2023
7519e75
Cargo.lock updated
michalkucharczyk Dec 11, 2023
4aa562b
make taplo happy
michalkucharczyk Dec 11, 2023
cbc7f5c
Merge branch 'master' into mku-keyring-in-runtime2
michalkucharczyk Dec 11, 2023
a1f1f30
Merge remote-tracking branch 'origin/master' into mku-keyring-in-runt…
michalkucharczyk Dec 20, 2023
4b58e50
Cargo.lock updated
michalkucharczyk Dec 20, 2023
c35e461
Merge remote-tracking branch 'origin/master' into mku-keyring-in-runt…
michalkucharczyk Jan 2, 2024
093b105
Cargo.lock updated
michalkucharczyk Jan 2, 2024
4b276b7
Merge branch 'master' into mku-keyring-in-runtime2
michalkucharczyk Jan 10, 2024
7029890
Update substrate/primitives/application-crypto/check-features-variant…
michalkucharczyk Jan 16, 2024
3ecd95a
Merge remote-tracking branch 'origin/master' into mku-keyring-in-runt…
michalkucharczyk Feb 1, 2024
3879d35
Cargo.lock updated
michalkucharczyk Feb 1, 2024
d1661db
fix
michalkucharczyk Feb 1, 2024
667eda6
clippy
michalkucharczyk Feb 1, 2024
2eaa0d0
Update substrate/primitives/core/src/address_uri.rs
michalkucharczyk Feb 2, 2024
28ec96c
review comments applied
michalkucharczyk Feb 2, 2024
8508715
clippy
michalkucharczyk Feb 2, 2024
6cb449a
bls + paired_crypto reworked
michalkucharczyk Feb 2, 2024
86c7bc9
".git/.scripts/commands/fmt/fmt.sh"
Feb 2, 2024
e26685f
Merge branch 'master' into mku-keyring-in-runtime2
michalkucharczyk Feb 2, 2024
cdf6f6f
Merge branch 'master' into mku-keyring-in-runtime2
michalkucharczyk Feb 10, 2024
97cb49e
Merge branch 'master' into mku-keyring-in-runtime2
michalkucharczyk Feb 22, 2024
8d93400
Merge branch 'master' into mku-keyring-in-runtime2
michalkucharczyk Feb 28, 2024
b6dbd3a
Merge remote-tracking branch 'origin/master' into mku-keyring-in-runt…
michalkucharczyk Mar 9, 2024
e53f516
format
michalkucharczyk Mar 9, 2024
bf8455d
Cargo.lock
michalkucharczyk Mar 9, 2024
861ea0b
fixes
michalkucharczyk Mar 9, 2024
037f70c
ecdsa: more verify variants enabled w/o full_crypto
michalkucharczyk Mar 9, 2024
c55ec33
ecdsa: full_crypto guard removed where not needed
michalkucharczyk Mar 9, 2024
58512bd
Merge branch 'master' into mku-keyring-in-runtime2
michalkucharczyk Mar 10, 2024
4ec9e42
Use parity-bip39 instead of git forks
michalkucharczyk Mar 12, 2024
b763059
Merge remote-tracking branch 'origin/master' into mku-keyring-in-runt…
michalkucharczyk Mar 12, 2024
92fcab2
fix
michalkucharczyk Mar 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .gitlab/pipeline/check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -259,3 +259,19 @@ find-fail-ci-phrase:
echo "No $ASSERT_REGEX was found, exiting with 0";
exit 0;
fi

check-core-crypto-features:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to permanently keep the check-features-variants.sh scripts?
I'm not sure about this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not?
I assume we want to check if all the variants compile fine. Script allows to run quickly check locally w/o looking into yml to check what variants are required.

stage: check
extends:
- .docker-env
- .common-refs
script:
- pushd substrate/primitives/core
- ./check-features-variants.sh
- popd
- pushd substrate/primitives/application-crypto
- ./check-features-variants.sh
- popd
- pushd substrate/primitives/keyring
- ./check-features-variants.sh
- popd
43 changes: 38 additions & 5 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion substrate/client/cli/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ rpassword = "7.0.0"
serde = { workspace = true, default-features = true }
serde_json = { workspace = true, default-features = true }
thiserror = { workspace = true }
bip39 = "2.0.0"
# personal fork here as workaround for: https://github.com/rust-bitcoin/rust-bip39/pull/64
bip39 = { package = "parity-bip39", version = "2.0.1", features = ["rand"] }
tokio = { version = "1.22.0", features = ["parking_lot", "rt-multi-thread", "signal"] }
sc-client-api = { path = "../api" }
sc-client-db = { path = "../db", default-features = false }
Expand Down
2 changes: 1 addition & 1 deletion substrate/client/cli/src/commands/generate.rs
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ impl GenerateCmd {
let password = self.keystore_params.read_password()?;
let output = self.output_scheme.output_type;

let phrase = mnemonic.word_iter().join(" ");
let phrase = mnemonic.words().join(" ");

with_crypto_scheme!(
self.crypto_scheme.scheme,
Expand Down
12 changes: 12 additions & 0 deletions substrate/primitives/application-crypto/check-features-variants.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
#!/usr/bin/env -S bash -eux

michalkucharczyk marked this conversation as resolved.
Show resolved Hide resolved
export RUSTFLAGS="-Cdebug-assertions=y -Dwarnings"
T=wasm32-unknown-unknown
cargo check --release
cargo check --release --target=$T --no-default-features
cargo check --release --target=$T --no-default-features --features="full_crypto"
cargo check --release --target=$T --no-default-features --features="serde"
cargo check --release --target=$T --no-default-features --features="serde,full_crypto"
cargo check --release --target=$T --no-default-features --features="bandersnatch-experimental"
cargo check --release --target=$T --no-default-features --features="bls-experimental"
cargo check --release --target=$T --no-default-features --features="bls-experimental,full_crypto"
5 changes: 2 additions & 3 deletions substrate/primitives/application-crypto/src/bls377.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,13 @@
use crate::{KeyTypeId, RuntimePublic};

pub use sp_core::bls::bls377::*;
use sp_std::vec::Vec;

mod app {
crate::app_crypto!(super, sp_core::testing::BLS377);
}

#[cfg(feature = "full_crypto")]
pub use app::Pair as AppPair;
pub use app::{Public as AppPublic, Signature as AppSignature};
pub use app::{Pair as AppPair, Public as AppPublic, Signature as AppSignature};

impl RuntimePublic for Public {
type Signature = Signature;
Expand Down
4 changes: 1 addition & 3 deletions substrate/primitives/application-crypto/src/ecdsa.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,7 @@ mod app {
crate::app_crypto!(super, sp_core::testing::ECDSA);
}

#[cfg(feature = "full_crypto")]
pub use app::Pair as AppPair;
pub use app::{Public as AppPublic, Signature as AppSignature};
pub use app::{Pair as AppPair, Public as AppPublic, Signature as AppSignature};

impl RuntimePublic for Public {
type Signature = Signature;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
//! ECDSA and BLS12-377 paired crypto applications.

use crate::{KeyTypeId, RuntimePublic};
use sp_std::vec::Vec;

pub use sp_core::paired_crypto::ecdsa_bls377::*;

Expand Down
4 changes: 1 addition & 3 deletions substrate/primitives/application-crypto/src/ed25519.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,7 @@ mod app {
crate::app_crypto!(super, sp_core::testing::ED25519);
}

#[cfg(feature = "full_crypto")]
pub use app::Pair as AppPair;
pub use app::{Public as AppPublic, Signature as AppSignature};
pub use app::{Pair as AppPair, Public as AppPublic, Signature as AppSignature};

impl RuntimePublic for Public {
type Signature = Signature;
Expand Down
59 changes: 40 additions & 19 deletions substrate/primitives/application-crypto/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,9 @@
#![warn(missing_docs)]
#![cfg_attr(not(feature = "std"), no_std)]

pub use sp_core::crypto::{key_types, CryptoTypeId, KeyTypeId};
pub use sp_core::crypto::{key_types, CryptoTypeId, DeriveJunction, KeyTypeId, Ss58Codec};
#[doc(hidden)]
#[cfg(feature = "full_crypto")]
pub use sp_core::crypto::{DeriveError, Pair, SecretStringError};
#[cfg(any(feature = "full_crypto", feature = "serde"))]
pub use sp_core::crypto::{DeriveJunction, Ss58Codec};
#[doc(hidden)]
pub use sp_core::{
self,
Expand Down Expand Up @@ -85,7 +82,7 @@ macro_rules! app_crypto {
$module::CRYPTO_ID
);
$crate::app_crypto_signature_common!($module::Signature, $key_type);
$crate::app_crypto_pair!($module::Pair, $key_type, $module::CRYPTO_ID);
$crate::app_crypto_pair_common!($module::Pair, $key_type, $module::CRYPTO_ID);
};
}

Expand Down Expand Up @@ -116,13 +113,15 @@ macro_rules! app_crypto {
$module::CRYPTO_ID
);
$crate::app_crypto_signature_common!($module::Signature, $key_type);
$crate::app_crypto_pair_common!($module::Pair, $key_type, $module::CRYPTO_ID);
};
}

/// Declares `Pair` type which is functionally equivalent to `$pair`, but is
/// new application-specific type whose identifier is `$key_type`.
/// It is a common part shared between full_crypto and non full_crypto environments.
#[macro_export]
macro_rules! app_crypto_pair {
macro_rules! app_crypto_pair_common {
($pair:ty, $key_type:expr, $crypto_type:expr) => {
$crate::wrap! {
/// A generic `AppPublic` wrapper type over $pair crypto; this has no specific App.
Expand All @@ -140,7 +139,14 @@ macro_rules! app_crypto_pair {
type Signature = Signature;

$crate::app_crypto_pair_functions_if_std!($pair);
$crate::app_crypto_pair_functions_if_full_crypto!($pair);

fn from_phrase(
phrase: &str,
password: Option<&str>,
) -> Result<(Self, Self::Seed), $crate::SecretStringError> {
<$pair>::from_phrase(phrase, password).map(|r| (Self(r.0), r.1))
}
fn derive<Iter: Iterator<Item = $crate::DeriveJunction>>(
&self,
path: Iter,
Expand All @@ -154,9 +160,6 @@ macro_rules! app_crypto_pair {
fn from_seed_slice(seed: &[u8]) -> Result<Self, $crate::SecretStringError> {
<$pair>::from_seed_slice(seed).map(Self)
}
fn sign(&self, msg: &[u8]) -> Self::Signature {
Signature(self.0.sign(msg))
}
fn verify<M: AsRef<[u8]>>(
sig: &Self::Signature,
message: M,
Expand Down Expand Up @@ -203,13 +206,6 @@ macro_rules! app_crypto_pair_functions_if_std {
let r = <$pair>::generate_with_phrase(password);
(Self(r.0), r.1, r.2)
}

fn from_phrase(
phrase: &str,
password: Option<&str>,
) -> Result<(Self, Self::Seed), $crate::SecretStringError> {
<$pair>::from_phrase(phrase, password).map(|r| (Self(r.0), r.1))
}
};
}

Expand All @@ -220,6 +216,25 @@ macro_rules! app_crypto_pair_functions_if_std {
($pair:ty) => {};
}

/// Implements functions for the `Pair` trait when `feature = "full_crypto"` is enabled.
#[doc(hidden)]
#[cfg(feature = "full_crypto")]
#[macro_export]
macro_rules! app_crypto_pair_functions_if_full_crypto {
($pair:ty) => {
fn sign(&self, msg: &[u8]) -> Self::Signature {
Signature(self.0.sign(msg))
}
};
}

#[doc(hidden)]
#[cfg(not(feature = "full_crypto"))]
#[macro_export]
macro_rules! app_crypto_pair_functions_if_full_crypto {
($pair:ty) => {};
}

/// Declares `Public` type which is functionally equivalent to `$public` but is
/// new application-specific type whose identifier is `$key_type`.
/// For full functionality, `app_crypto_public_common!` must be called too.
Expand Down Expand Up @@ -267,7 +282,7 @@ macro_rules! app_crypto_public_not_full_crypto {
$crate::wrap! {
/// A generic `AppPublic` wrapper type over $public crypto; this has no specific App.
#[derive(
Clone, Eq, PartialEq, Ord, PartialOrd,
Clone, Eq, Hash, PartialEq, Ord, PartialOrd,
$crate::codec::Encode,
$crate::codec::Decode,
$crate::RuntimeDebug,
Expand All @@ -277,10 +292,13 @@ macro_rules! app_crypto_public_not_full_crypto {
pub struct Public($public);
}

impl $crate::CryptoType for Public {}
impl $crate::CryptoType for Public {
type Pair = Pair;
}

impl $crate::AppCrypto for Public {
type Public = Public;
type Pair = Pair;
type Signature = Signature;
const ID: $crate::KeyTypeId = $key_type;
const CRYPTO_ID: $crate::CryptoTypeId = $crypto_type;
Expand Down Expand Up @@ -452,10 +470,13 @@ macro_rules! app_crypto_signature_not_full_crypto {
pub struct Signature($sig);
}

impl $crate::CryptoType for Signature {}
impl $crate::CryptoType for Signature {
type Pair = Pair;
}

impl $crate::AppCrypto for Signature {
type Public = Public;
type Pair = Pair;
type Signature = Signature;
const ID: $crate::KeyTypeId = $key_type;
const CRYPTO_ID: $crate::CryptoTypeId = $crypto_type;
Expand Down
4 changes: 1 addition & 3 deletions substrate/primitives/application-crypto/src/sr25519.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,7 @@ mod app {
crate::app_crypto!(super, sp_core::testing::SR25519);
}

#[cfg(feature = "full_crypto")]
pub use app::Pair as AppPair;
pub use app::{Public as AppPublic, Signature as AppSignature};
pub use app::{Pair as AppPair, Public as AppPublic, Signature as AppSignature};

impl RuntimePublic for Public {
type Signature = Signature;
Expand Down
14 changes: 1 addition & 13 deletions substrate/primitives/application-crypto/src/traits.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,7 @@
use codec::Codec;
use scale_info::TypeInfo;

#[cfg(feature = "full_crypto")]
use sp_core::crypto::Pair;
use sp_core::crypto::{CryptoType, CryptoTypeId, IsWrappedBy, KeyTypeId, Public};
use sp_core::crypto::{CryptoType, CryptoTypeId, IsWrappedBy, KeyTypeId, Pair, Public};
use sp_std::{fmt::Debug, vec::Vec};

/// Application-specific cryptographic object.
Expand All @@ -45,24 +43,14 @@ pub trait AppCrypto: 'static + Sized + CryptoType {
type Signature: AppSignature;

/// The corresponding key pair type in this application scheme.
#[cfg(feature = "full_crypto")]
type Pair: AppPair;
}

/// Type which implements Hash in std, not when no-std (std variant).
#[cfg(any(feature = "std", feature = "full_crypto"))]
pub trait MaybeHash: sp_std::hash::Hash {}
#[cfg(any(feature = "std", feature = "full_crypto"))]
impl<T: sp_std::hash::Hash> MaybeHash for T {}

/// Type which implements Hash in std, not when no-std (no-std variant).
#[cfg(all(not(feature = "std"), not(feature = "full_crypto")))]
pub trait MaybeHash {}
#[cfg(all(not(feature = "std"), not(feature = "full_crypto")))]
impl<T> MaybeHash for T {}

/// Application-specific key pair.
#[cfg(feature = "full_crypto")]
pub trait AppPair:
AppCrypto + Pair<Public = <Self as AppCrypto>::Public, Signature = <Self as AppCrypto>::Signature>
{
Expand Down
Loading
Loading