Support using P-256 keys for block and transaction signing #491

swatanabe · 2023-09-22T22:07:33Z

Motivation

The NIST curves are much more widely supported than secp256k1. Adding them will give us better compatibility with third party software and hardware.

Data formats

The signature is encoded in compact form (64 bytes, r+s, big endian)
The public key is encoded as an X.509 Subject Public Key Info
The private key is encoded using PKCS #8

Verify Service

The VerifySys service (verify-sys) uses Botan. Botan supports picking out only the required components much better than openssl does and compiles to WASM with only minimal tweaks. The main patch I needed to make was to add a stub exception library that simply aborts when an exception is thrown. VerifySys supports both named curves and arbitrary curves with explicit domain parameters. However, only 256-bit curves are useful right now, because SHA256 is applied by native.

Block Signing

Native block signing is supported using OpenSSL. Any private key supported by OpenSSL can be used to sign, however key generation is not configurable and always generates an r1 key. As with VerifySys, the use of SHA256 is hard-coded.

psibase command line support

All key command line arguments can be either base58 encoded keys as before or the name of a file containing a PEM or DER encoded key. A base58 key uses verifyec-sys/auth-ec-sys. ASN.1 encodings use verify-sys/auth-sys. Only k1 keys are supported right now. The public key must be in compressed form and must use the named curve representation of the parameters.

js frontend support

Keys can be either PEM or Base58 encoded. When creating an account, account-sys always uses auth-sys. When signing or importing keys, the auth service will be deduced by querying the chain.

James-Mart · 2023-09-25T16:06:39Z

VerifySys supports both named curves and arbitrary curves with explicit domain parameters. However, only 256-bit curves are useful right now, because SHA256 is applied by native.

Am I understanding correctly that verify-sys can be used to verify nist and k1 curves?

swatanabe · 2023-09-25T16:31:13Z

Am I understanding correctly that verify-sys can be used to verify nist and k1 curves?

Yes. Also brainpool and other more obscure curves.

James-Mart · 2023-09-25T16:55:43Z

Yes. Also brainpool and other more obscure curves

Any reason not to switch everything to this and delete verify-ec-sys?

swatanabe · 2023-09-25T18:17:23Z

Any reason not to switch everything to this and delete verify-ec-sys?

I haven't checked performance. secp256k1 is more optimized.

James-Mart · 2023-09-25T22:03:09Z

programs/psinode/main.cpp

+         }
+         else if (key.service.str() == "verify-sys")
+         {
+            result = std::make_shared<OpenSSLProver>(key.service, key.rawData);


Shouldn't the key data be fracpacked?

Why? It's already in a binary format.

Just noticed that the k1 key is expected to be packed, why is it different?

The k1 key is actually a variant<k1_key, r1_key>, so it needs additional encoding. The BER encoded key already contains a discriminator to identify the type of key. I don't see any reason to add another layer of wrapping.

James-Mart · 2023-09-25T22:06:11Z

rust/psibase/src/boot.rs

@@ -52,6 +52,7 @@ const ACCOUNTS: [AccountNumber; 27] = [
    account!("token-sys"),
    transaction_sys::SERVICE,
    account!("verifyec-sys"),
+    account!("verify-sys"),


The names are a bit misleading now, since nist curves are ec, but can only be verified with verify-sys. Maybe rename verifyec-sys to verifyk1-sys?

- Botan requires a larger stack to initialize the oid tables - raw_hash needs to be available

- Change account-sys and common-sys to use auth-sys instead of auth-ec-sys and adjust key and signature formats

swatanabe · 2023-10-13T19:01:24Z

The current performance numbers don't look good. I'm going to check how much of that is global initialization.

ns	verify-sys	verifyec-sys
	43013901	733819
	42598396	2903760
	45292236	778158
	66910510	2885490
	42495448	1679233

swatanabe · 2023-10-16T18:32:53Z

The current performance numbers don't look good. I'm going to check how much of that is global initialization.

Most of the cost appears to be in initializing tables for EC group operations.

…'t overwrite them.

swatanabe · 2023-10-17T15:56:01Z

Tests are failing because binaryen 105 (from ubuntu 22.04) doesn't fully support v128.

…can't handle it.

James-Mart · 2023-10-18T19:24:12Z

Can't we just change the 22.04 image to use the version of binaryen currently installed in 20.04?

swatanabe · 2023-10-18T19:44:22Z

Can't we just change the 22.04 image to use the version of binaryen currently installed in 20.04?

We could, but I'd rather minimize our dependence on non-system packages. I didn't see any significant performance difference for this service. We can reconsider minimum versions in six months when 24.04 ships.

Limitations: - psibase chooses the auth service based on whether the key is base58 (auth-ec-sys) or PEM (auth-sys). Since it does not query the chain, there is no other way to deduce the auth service. - The PEM encoded keys must be put directly on the command line. Reading a file is not supported. - The private key always signs using a compressed public key, but the public keys are not normalized. This can lead to signatures not being recognized unless the user is careful about managing encodings.

…nd line

swatanabe added 7 commits September 22, 2023 12:27

Add support for r1 block signing

aaca895

Add r1 signature verification using Botan

8aa5882

Move stub __cxa_throw into a separate library

63a1e08

start needs to call __wasm_call_ctors

0f1ffeb

Add missing dependency and add some explanations about the Botan build

1d62c0f

Use compact ECDSA signatures instead of DER

5297277

Handle reading r1 keys

6541795

swatanabe requested a review from James-Mart September 22, 2023 22:07

Avoid functions added in OpenSSL 3.0

47f62b9

EVP_PKEY_generate was also added in OpenSSL 3.0.

9a42962

James-Mart reviewed Sep 25, 2023

View reviewed changes

James-Mart added the Node infra Related to the necessary infrastructure provided by all psibase infrastructure providers label Sep 26, 2023

swatanabe added 3 commits September 28, 2023 13:22

Fix VerifySys

8953040

- Botan requires a larger stack to initialize the oid tables - raw_hash needs to be available

Print exception type when aborting on exception

2bdecd9

Add auth-sys to match verify-sys

30708ac

- Change account-sys and common-sys to use auth-sys instead of auth-ec-sys and adjust key and signature formats

James-Mart changed the title ~~Support P-256 keys~~ Support using P-256 curves for block and transaction signing Oct 2, 2023

James-Mart changed the title ~~Support using P-256 curves for block and transaction signing~~ Support using P-256 keys for block and transaction signing Oct 2, 2023

James-Mart mentioned this pull request Oct 2, 2023

Phase 2: Simplify psinode admin #497

Open

Allow importing and signing for accounts that use the old auth-ec-sys

264358c

swatanabe added 3 commits October 16, 2023 14:03

pre-evaluate expensive setup

3f91419

Multiple applets can be loaded between renders. Make sure that we don…

e58c067

…'t overwrite them.

Merge branch 'main' into r1-support

745b784

Disable simd in botan because the version of wasm-ctor-eval in 22.04 …

d94cc0b

…can't handle it.

swatanabe added 3 commits October 23, 2023 15:22

Add missing file

9f569d8

Read PEM and DER files instead of expecting PEM directly on the comma…

7415fa9

…nd line

swatanabe requested a review from James-Mart October 24, 2023 16:38

James-Mart approved these changes Oct 24, 2023

View reviewed changes

swatanabe merged commit da03175 into main Oct 24, 2023
3 checks passed

swatanabe deleted the r1-support branch October 24, 2023 20:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support using P-256 keys for block and transaction signing #491

Support using P-256 keys for block and transaction signing #491

swatanabe commented Sep 22, 2023 •

edited

Loading

James-Mart commented Sep 25, 2023 •

edited

Loading

swatanabe commented Sep 25, 2023

James-Mart commented Sep 25, 2023

swatanabe commented Sep 25, 2023

James-Mart Sep 25, 2023

swatanabe Sep 25, 2023

James-Mart Sep 26, 2023

swatanabe Sep 26, 2023

James-Mart Sep 25, 2023

swatanabe commented Oct 13, 2023

swatanabe commented Oct 16, 2023

swatanabe commented Oct 17, 2023

James-Mart commented Oct 18, 2023

swatanabe commented Oct 18, 2023

Support using P-256 keys for block and transaction signing #491

Support using P-256 keys for block and transaction signing #491

Conversation

swatanabe commented Sep 22, 2023 • edited Loading

Motivation

Data formats

Verify Service

Block Signing

psibase command line support

js frontend support

James-Mart commented Sep 25, 2023 • edited Loading

swatanabe commented Sep 25, 2023

James-Mart commented Sep 25, 2023

swatanabe commented Sep 25, 2023

James-Mart Sep 25, 2023

Choose a reason for hiding this comment

swatanabe Sep 25, 2023

Choose a reason for hiding this comment

James-Mart Sep 26, 2023

Choose a reason for hiding this comment

swatanabe Sep 26, 2023

Choose a reason for hiding this comment

James-Mart Sep 25, 2023

Choose a reason for hiding this comment

swatanabe commented Oct 13, 2023

swatanabe commented Oct 16, 2023

swatanabe commented Oct 17, 2023

James-Mart commented Oct 18, 2023

swatanabe commented Oct 18, 2023

swatanabe commented Sep 22, 2023 •

edited

Loading

James-Mart commented Sep 25, 2023 •

edited

Loading