Member recovery shares encryption with RSA-OAEP-256 (#1841)

CHANGELOG.md

@@ -6,12 +6,18 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
 
 ## Unreleased
+
 ### Changed
-- CCF now depends on [Open Enclave 0.12](https://github.com/openenclave/openenclave/releases/tag/v0.12.0).
+- CCF now depends on [Open Enclave 0.12](https://github.com/openenclave/openenclave/releases/tag/v0.12.0) (#1830).
 - `/app/user_id` now takes `{"cert": user_cert_as_pem_string}` rather than `{"cert": user_cert_as_der_list_of_bytes}` (#278).
+- Members' recovery shares are now encrypted using [RSA-OAEP-256](https://docs.microsoft.com/en-gb/azure/key-vault/keys/about-keys#wrapkeyunwrapkey-encryptdecrypt) (#1841). This has the following implications:
+  - Network's encryption key is no longer output by the first node of a CCF service is no longer required to decrypt recovery shares.
+  - The latest version of the `submit_recovery_share.sh` script should be used.
+  - The latest version of the `proposal_generator.py` should be used (please upgrade the [ccf python package](https://microsoft.github.io/CCF/master/quickstart/install.html#python-package)).
+- `submit_recovery_share.sh` script's `--rpc-address` argument has been removed. The node's address (e.g. `https://127.0.0.1:8000`) should be used directly as the first argument instead (#1841).
 
 ### Fixed
-- Added `tools.cmake` to the install , which `ccf_app.cmake` depends on and was missing from the previous release. 
+- Added `tools.cmake` to the install , which `ccf_app.cmake` depends on and was missing from the previous release.
 
 ## [0.14.2]
 ### Changed

doc/members/accept_recovery.rst

@@ -55,15 +55,15 @@ The recovery share retrieval, decryption and submission steps are conveniently p
 
 .. code-block:: bash
 
-    $ ./submit_recovery_share.sh --rpc-address <ccf-node-address> --member-enc-privk member0_enc_privk.pem --network-enc-pubk network_enc_pubk --cert member0_cert
+    $ submit_recovery_share.sh https://<ccf-node-address> --member-enc-privk member0_enc_privk.pem --cert member0_cert
     --key member0_privk --cacert network_cert
     HTTP/1.1 200 OK
     content-type: text/plain
     x-ccf-tx-seqno: 28
     x-ccf-tx-view: 4
     1/2 recovery shares successfully submitted.
 
-    $ ./submit_recovery_share.sh --rpc-address <ccf-node-address> --member-enc-privk member1_enc_privk.pem --network-enc-pubk network_enc_pubk --cert member1_cert
+    $ submit_recovery_share.sh https://<ccf-node-address> --member-enc-privk member1_enc_privk.pem --cert member1_cert
     --key member1_privk --cacert network_cert
     HTTP/1.1 200 OK
     content-type: text/plain

doc/members/adding_member.rst

@@ -19,7 +19,7 @@ The ``keygenerator.sh`` script can be used to generate the member’s certificat
     Identity curve: secp384r1
     Identity private key generated at:   member_name_privk.pem
     Identity certificate generated at:   member_name_cert.pem (to be registered in CCF)
-    -- Generating encryption key pair for participant "member_name"...
+    -- Generating RSA encryption key pair for participant "member_name"...
     Encryption private key generated at:  member_name_enc_privk.pem
     Encryption public key generated at:   member_name_enc_pubk.pem (to be registered in CCF)
 

doc/members/proposals.rst

@@ -40,25 +40,25 @@ Some of these subcommands require additional arguments, such as the node ID or u
     SUCCESS | Writing proposal to ./trust_node_proposal.json
     SUCCESS | Wrote vote to ./trust_node_vote_for.json
 
-    $ cat trust_node_proposal.json 
+    $ cat trust_node_proposal.json
     {"script": {"text": "tables, args = ...; return Calls:call(\"trust_node\", args)"}, "parameter": "5"}
 
-    $ cat trust_node_vote_for.json 
+    $ cat trust_node_vote_for.json
     {"ballot": {"text": "tables, calls = ...; if not #calls == 1 then return false end; call = calls[1]; if not call.func == \"trust_node\" then return false end; args = call.args; if args == nil then return false end; if not args == [====[5]====] then return false end; return true"}}
 
-    $ python -m ccf.proposal_generator --pretty-print --proposal-output-file add_pedro.json --vote-output-file vote_for_pedro.json new_user pedro_cert.pem 
+    $ python -m ccf.proposal_generator --pretty-print --proposal-output-file add_pedro.json --vote-output-file vote_for_pedro.json new_user pedro_cert.pem
     SUCCESS | Writing proposal to ./add_pedro.json
     SUCCESS | Wrote vote to ./vote_for_pedro.json
-    
-    $ cat add_pedro.json 
+
+    $ cat add_pedro.json
     {
       "script": {
         "text": "tables, args = ...; return Calls:call(\"new_user\", args)"
       },
       "parameter": "-----BEGIN CERTIFICATE-----\nMIIBrzCCATSgAwIBAgIUJY+H0OzuFQWz/udd+WCD7Cv+cgwwCgYIKoZIzj0EAwMw\nDjEMMAoGA1UEAwwDYm9iMB4XDTIwMDcyNDE1MzYyOFoXDTIxMDcyNDE1MzYyOFow\nDjEMMAoGA1UEAwwDYm9iMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAE7h75Xd1+0QDD\nWF2edGphgryHcDoBXdRowq6ciYH2++ilXXagi5Rybai7ewgV0YuvrDm+WfGyJ9CC\n5HbT6C/z5GCJQnLH2t3LaZrw9MQDF3bH6XOHGmaJh6m7rfpZZljpo1MwUTAdBgNV\nHQ4EFgQUN/LhCyVExERjt5f1RZx7820934wwHwYDVR0jBBgwFoAUN/LhCyVExERj\nt5f1RZx7820934wwDwYDVR0TAQH/BAUwAwEB/zAKBggqhkjOPQQDAwNpADBmAjEA\n5MsDNvjEMSgYXy+bPbE2nxOlmH6OhP375IVZxNQALJGzTfgHu+IbpyvDF0/VrMrW\nAjEA723VxgMgpuxB5SszN6eZuz8EW51DsgRIVWMSbBZYYBYyQmu5x3T+Hx/Cs7TD\nu4Ee\n-----END CERTIFICATE-----\n"
     }
-    
-    $ cat vote_for_pedro.json 
+
+    $ cat vote_for_pedro.json
     {
       "ballot": {
         "text": "tables, calls = ...; if not #calls == 1 then return false end; call = calls[1]; if not call.func == \"new_user\" then return false end; args = call.args; if args == nil then return false end; if not args == [====[-----BEGIN CERTIFICATE-----\nMIIBrzCCATSgAwIBAgIUJY+H0OzuFQWz/udd+WCD7Cv+cgwwCgYIKoZIzj0EAwMw\nDjEMMAoGA1UEAwwDYm9iMB4XDTIwMDcyNDE1MzYyOFoXDTIxMDcyNDE1MzYyOFow\nDjEMMAoGA1UEAwwDYm9iMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAE7h75Xd1+0QDD\nWF2edGphgryHcDoBXdRowq6ciYH2++ilXXagi5Rybai7ewgV0YuvrDm+WfGyJ9CC\n5HbT6C/z5GCJQnLH2t3LaZrw9MQDF3bH6XOHGmaJh6m7rfpZZljpo1MwUTAdBgNV\nHQ4EFgQUN/LhCyVExERjt5f1RZx7820934wwHwYDVR0jBBgwFoAUN/LhCyVExERj\nt5f1RZx7820934wwDwYDVR0TAQH/BAUwAwEB/zAKBggqhkjOPQQDAwNpADBmAjEA\n5MsDNvjEMSgYXy+bPbE2nxOlmH6OhP375IVZxNQALJGzTfgHu+IbpyvDF0/VrMrW\nAjEA723VxgMgpuxB5SszN6eZuz8EW51DsgRIVWMSbBZYYBYyQmu5x3T+Hx/Cs7TD\nu4Ee\n-----END CERTIFICATE-----\n]====] then return false end; return true"
@@ -102,11 +102,12 @@ For example, ``member1`` may submit a proposal to add a new member (``member4``)
     $ cat new_member.json
     {
       "parameter": {
-        "cert": "-----BEGIN CERTIFICATE-----\nMIIBrzCCATSgAwIBAgIUTu47sG/Ziz4hgoeMhKzs/alrEYcwCgYIKoZIzj0EAwMw\nDjEMMAoGA1UEAwwDYm9iMB4XDTIwMDcwOTE0NTc0OFoXDTIxMDcwOTE0NTc0OFow\nDjEMMAoGA1UEAwwDYm9iMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAENhB3M5fWT5YQ\n+vBOl0T9xt29CvYBsJyLCGeflqLAFA4YDs7Bb3mMH46EiJg+BFT0HmIPtGW91qE5\nZEPMINQ2zuU0IU6uomPBi76pQ5vhm/2HDy3SLDwRytrSDNqTXZXfo1MwUTAdBgNV\nHQ4EFgQUBchpeGuTHjy4XuwdgQqC3pOqOdEwHwYDVR0jBBgwFoAUBchpeGuTHjy4\nXuwdgQqC3pOqOdEwDwYDVR0TAQH/BAUwAwEB/zAKBggqhkjOPQQDAwNpADBmAjEA\nmNPNpZvqn3piEepKGFJtqKtq+bZxUZuWZxxXILj4/qnC1fLatJaMQ/DHRtCxwcU/\nAjEAtZe3LAQ6NtVIrn4qFG3ruuEgFL9arCpFGEBLFkVdkL2nFIBTp1L4C1/aJBqk\nK9d9\n-----END CERTIFICATE-----\n",
-        "keyshare": "-----BEGIN PUBLIC KEY-----\nMCowBQYDK2VuAyEAO63rFGghBlp4DUvFQ6437ZGBlB8LNHnzgNEjW5hRPHM=\n-----END PUBLIC KEY-----\n"
+        "cert": "-----BEGIN CERTIFICATE-----\nMIIBdzCCARygAwIBAgIURwD6S1/rcb2TbHhQLnTNh/7WyYYwCgYIKoZIzj0EAwIw\nEjEQMA4GA1UEAwwHbWVtYmVyNDAeFw0yMDEwMjkxNjI2NTNaFw0yMTEwMjkxNjI2\nNTNaMBIxEDAOBgNVBAMMB21lbWJlcjQwVjAQBgcqhkjOPQIBBgUrgQQACgNCAARG\nwqj2ZD7vA+h4KoTdh3if3tVO/yks+xtLU1tXAFsbeWSQfDxK3nnA65uX6n/25A20\nJcAQMDHYH2NdLOLra9lxo1MwUTAdBgNVHQ4EFgQUQQDC71N60r/a9c+EGXrzr5l6\nIDQwHwYDVR0jBBgwFoAUQQDC71N60r/a9c+EGXrzr5l6IDQwDwYDVR0TAQH/BAUw\nAwEB/zAKBggqhkjOPQQDAgNJADBGAiEAkvP0AuAU7y0b3z4rhvoOkCBKoH4G3vh/\nPJpLFdWcEu4CIQCSnEYpDaDTP2zoWTheqchZ+/BdTzM2j2s9ILpvSVYMxg==\n-----END CERTIFICATE-----\n",
+        "encryption_pub_key": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAvYKesV5xoT2XnGhLkeqZ\neSC2KsjNUvdjqPrTERk/hp64Xd30SGjdj2HytG3hfCy5hBhc9muQMXoOAOBgxwMA\nQRu7KCANPZNCLEWKR5DZc8YzE+rHX1/8WxhhtV/bvr90selV0BfLWLLJYDxnyo3D\nyioYXNw6Ij2sYBt8MTPNPti3jRJ7LmMow/VrJD9Ww1FKWCyxa7/iCxSsbmrwdv8m\nBVf/+d3p+ivxb6gBvtTimj+fj1OdRkGHElZSaBFWmQISga3Ki4vnP4W1iw/ujaza\n3gItLPrEnD0lxGBaCSs+XVm2l8nsn3HJDZYMP5u3jWB3MWsBwna0o+KUon4KaS1k\nlwIDAQAB\n-----END PUBLIC KEY-----\n",
+        "member_data": null
       },
       "script": {
-        "text": "tables, args = ...; return Calls:call(\"new_member\", args)"
+        "text": "\n    tables, args = ...\n    return Calls:call(\"new_member\", args)\n    "
       }
     }
 

doc/operators/recovery.rst

@@ -31,9 +31,9 @@ To initiate the first phase of the recovery procedure, one or several nodes shou
     recover
     --network-cert-file /path/to/network_certificate
 
-Each node will then immediately restore the public entries of its ledger (``--ledger-dir``). Because deserialising the public entries present in the ledger may take some time, operators can query the progress of the public recovery by calling ``node/state`` which returns the version of the last signed recovered ledger entry. Once the public ledger is fully recovered, the recovered node automatically becomes part of the public network, allowing other nodes to join the network.
+Each node will then immediately restore the public entries of its ledger (``--ledger-dir``). Because deserialising the public entries present in the ledger may take some time, operators can query the progress of the public recovery by calling ``/node/state`` which returns the version of the last signed recovered ledger entry. Once the public ledger is fully recovered, the recovered node automatically becomes part of the public network, allowing other nodes to join the network.
 
-.. note:: If more than one node were started in ``recover`` mode, the node with the highest signed index (as per the response to the ``node/state`` RPC) should be preferred to start the new network. Other nodes should be shutdown and new nodes restarted with the ``join`` option.
+.. note:: If more than one node were started in ``recover`` mode, the node with the highest signed sequence number (as per the response to the ``/node/state`` RPC) should be preferred to start the new network. Other nodes should be shutdown and new nodes restarted with the ``join`` option.
 
 Similarly to the normal join protocol (see :ref:`operators/start_network:Adding a New Node to the Network`), other nodes are then able to join the network.
 
@@ -48,10 +48,10 @@ Similarly to the normal join protocol (see :ref:`operators/start_network:Adding
         Node 2-->>Operators: Network Certificate
         Note over Node 2: Reading Public Ledger...
 
-        Operators->>+Node 2: node/state
+        Operators->>+Node 2: /node/state
         Node 2-->>Operators: {"last_signed_seqno": 50, "state": "readingPublicLedger"}
         Note over Node 2: Finished Reading Public Ledger, now Part of Public Network
-        Operators->>Node 2: node/state
+        Operators->>Node 2: /node/state
         Node 2-->>Operators: {"last_signed_seqno": 243, "state": "partOfPublicNetwork"}
 
         Note over Operators, Node 2: Operators select Node 2 to start the new network

doc/operators/start_network.rst

@@ -25,14 +25,13 @@ To create a new CCF network, the first node of the network should be invoked wit
     [--sig-ms-interval number_of_milliseconds]
     start
     --network-cert-file /path/to/network_certificate
-    --network-enc-pubk-file /path/to/network_encryption_pubk
     --member-info /path/to/member1_cert,/path/to/member1_enc_pub[,</path/to/member1_data>]
     [--member-info /path/to/member2_cert,/path/to/member2_enc_pub ...]
     --gov-script /path/to/lua/governance_script
 
 CCF nodes can be started by using IP Addresses (both IPv4 and IPv6 are supported) or by specifying a fully qualified domain name. If an FQDN is used then ``--domain`` should be passed to the node at startup. Once a DNS has been setup it will be possible to connect to the node over TLS by using the node's domain name.
 
-When starting up, the node generates its own key pair and outputs the certificate associated with its public key at the location specified by ``--node-cert-file``. The certificate of the freshly-created CCF network is also output at the location specified by ``--network-cert-file`` as well as the network encryption public key used by members during recovery via ``--network-enc-pubk-file``.
+When starting up, the node generates its own key pair and outputs the certificate associated with its public key at the location specified by ``--node-cert-file``. The certificate of the freshly-created CCF network is also output at the location specified by ``--network-cert-file``.
 
 .. note:: The network certificate should be distributed to users and members to be used as the certificate authority (CA) when establishing a TLS connection with any of the nodes part of the CCF network. When using curl, this is passed as the ``--cacert`` argument.
 

doc/quickstart/test_network.rst

@@ -42,7 +42,7 @@ In a different terminal, using the local IP address and port of the CCF nodes di
 Recovering a Service
 --------------------
 
-The ``sandbox.sh`` script can also be used to automatically recover a defunct network, as per the steps described :ref:`here <members/accept_recovery:Accepting Recovery and Submitting Shares>`. The ledger to be recovered (``--ledger-dir``) , the defunct network encryption public key (``--network-enc-pubk``) and the directory containing the members and users identities and the network encryption public key (``--common-dir``) should be passed as arguments to the script.
+The ``sandbox.sh`` script can also be used to automatically recover a defunct network, as per the steps described :ref:`here <members/accept_recovery:Accepting Recovery and Submitting Shares>`. The ledger to be recovered (``--ledger-dir``) and the directory containing the members and users identities (``--common-dir``) should be passed as arguments to the script.
 
 Additionally, if snapshots were generated by the defunct service (using the ``--snapshot-tx-interval <interval>`` option), the recovery procedure can be significantly sped up by re-starting from the latest available snapshot (``--snapshot-dir``).
 
@@ -51,13 +51,11 @@ Additionally, if snapshots were generated by the defunct service (using the ``--
     $ cd CCF/build
     $ cp -r ./workspace/sandbox_0/0.ledger .
     $ cp -r ./workspace/sanbox_0/snapshots . # Optional, only if snapshots are available
-    $ cp ./workspace/sandbox_0/network_enc_pubk.pem .
-    $ ./sandbox.sh -e release -p liblogging.enclave.so.signed --recover --ledger-dir 0.ledger --network-enc-pubk network_enc_pubk.pem --common-dir ./workspace/sandbox_common/ [--snapshot-dir snapshots]
+    $ ./sandbox.sh -e release -p liblogging.enclave.so.signed --recover --ledger-dir 0.ledger --common-dir ./workspace/sandbox_common/ [--snapshot-dir snapshots]
     Setting up Python environment...
     Python environment successfully setup
     [16:24:29.563] Starting 1 CCF nodes...
     [16:24:29.563] Recovering network from:
-    [16:24:29.563]  - Defunct network public encryption key: network_enc_pubk.pem
     [16:24:29.563]  - Common directory: ./workspace/sandbox_common/
     [16:24:29.563]  - Ledger: 0.ledger
     [16:24:29.563] No available snapshot to recover from. Entire transaction history will be replayed.

doc/schemas/gov_openapi.json

@@ -95,21 +95,6 @@
         ],
         "type": "object"
       },
-      "GetEncryptedRecoveryShare": {
-        "properties": {
-          "encrypted_recovery_share": {
-            "$ref": "#/components/schemas/string"
-          },
-          "nonce": {
-            "$ref": "#/components/schemas/string"
-          }
-        },
-        "required": [
-          "encrypted_recovery_share",
-          "nonce"
-        ],
-        "type": "object"
-      },
       "GetMetrics__HistogramResults": {
         "properties": {
           "buckets": {
@@ -1020,7 +1005,7 @@
             "content": {
               "application/json": {
                 "schema": {
-                  "$ref": "#/components/schemas/GetEncryptedRecoveryShare"
+                  "$ref": "#/components/schemas/string"
                 }
               }
             },

doc/users/index.rst

@@ -10,7 +10,7 @@ To generate the certificate and private key of trusted users should be generated
     Identity private key generated at:   user1_privk.pem
     Identity certificate generated at:   user1_cert.pem (to be registered in CCF)
 
-.. note:: The ``keygenerator.sh`` script is installed as part of the :ref:`CCF Python package <users/python_tutorial:Python Client Tutorial>` or can be found in the CCF repo under ``python/utils``. This is a simple wrapper around openssl. CCF will accept key may generated by other tools, as long as they are in the correct format and on a supported cryptographic curve. See :ref:`design/cryptography:Algorithms and Curves` for the list of supported cryptographic curves.
+.. note:: The ``keygenerator.sh`` script is installed as part of the :ref:`CCF Python package <users/python_tutorial:Python Client Tutorial>` or can be found in the CCF installation under ``bin``. This is a simple wrapper around ``openssl``. CCF will accept key may generated by other tools, as long as they are in the correct format and on a supported cryptographic curve. See :ref:`design/cryptography:Algorithms and Curves` for the list of supported cryptographic curves.
 
 Before issuing business transactions to CCF, the certificates of trusted users need to be voted in by the consortium of members (see :ref:`members/open_network:Adding Users`).
 

edl/ccf.edl

@@ -20,9 +20,6 @@ enclave {
             [out, count=network_cert_size] uint8_t * network_cert,
             size_t network_cert_size,
             [out] size_t* network_cert_len,
-            [out, count=network_enc_pubk_size] uint8_t * network_enc_pubk,
-            size_t network_enc_pubk_size,
-            [out] size_t* network_enc_pubk_len,
             StartType start_type,
             ConsensusType consensus_type,
             size_t num_worker_thread,

getting_started/setup_vm/roles/ccf_build/tasks/install.yml

@@ -59,9 +59,4 @@
   command: make install
   args:
     chdir: "{{ workspace }}/mbedtls-{{ mbedtls_dir }}/build"
-  become: true
-
-- name: Install step-cli
-  apt:
-    deb: "{{ step_cli }}"
-  become: true
+  become: true

getting_started/setup_vm/roles/ccf_build/vars/common.yml

@@ -25,6 +25,4 @@ debs:
 
 mbedtls_ver: "2.16.6"
 mbedtls_dir: "mbedtls-{{ mbedtls_ver }}"
-mbedtls_src: "{{ mbedtls_dir }}.tar.gz"
-step_cli_ver: "0.15.2"
-step_cli: "https://github.com/smallstep/cli/releases/download/v{{ step_cli_ver }}/step-cli_{{ step_cli_ver }}_amd64.deb"
+mbedtls_src: "{{ mbedtls_dir }}.tar.gz"

python/ccf/proposal_generator.py

@@ -199,7 +199,7 @@ def new_member(
 
     # Read certs
     member_cert = open(member_cert_path).read()
-    member_keyshare_encryptor = open(member_enc_pubk_path).read()
+    encryption_pub_key = open(member_enc_pubk_path).read()
 
     # Script which proposes adding a new member
     proposal_script_text = """
@@ -211,7 +211,7 @@ def new_member(
     proposal = {
         "parameter": {
             "cert": member_cert,
-            "keyshare": member_keyshare_encryptor,
+            "encryption_pub_key": encryption_pub_key,
             "member_data": member_data,
         },
         "script": {"text": proposal_script_text},
@@ -239,8 +239,8 @@ def new_member(
     return false
     end
 
-    expected_keyshare = [====[{member_keyshare_encryptor}]====]
-    if not call.args.keyshare == expected_keyshare then
+    expected_enc_pub_key = [====[{encryption_pub_key}]====]
+    if not call.args.encryption_pub_key == expected_enc_pub_key then
     return false
     end