Renew node certificate proposal

.daily_canary

@@ -1 +1 @@
-PSW!
+.

CHANGELOG.md

@@ -13,6 +13,11 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 - `get_metrics_v1` API to `BaseEndpointRegistry` for applications that do not make use of builtins and want to version or customise metrics output.
 - Slow ledger IO operations will now be logged at level FAIL. The threshold over which logging will activate can be adjusted by the `--io-logging-threshold` CLI argument to cchost (#3067).
 - Snapshot files now include receipt of evidence transaction. Nodes can now join or recover a service from a standalone snapshot file. 2.x nodes can still make use of snapshots created by a 1.x node, as long as the ledger suffix containing the proof of evidence is also specified at start-up (#2998).
+- Nodes certificates validity period is no longer hardcoded and can instead be set by operators and renewed by members (#2924):
+  - The new `--initial-node-cert-validity-days` (defaults to 1 day) CLI argument to cchost lets operators set the initial validity period for the node certificate (valid from the current system time).
+  - The new `--max-allowed-node-cert-validity-days` (defaults to 365 days) CLI argument to cchost sets the maximum validity period allowed for node certificates.
+  - The new `set_node_certificate_validity` proposal action allows members to renew a node certificate (or `set_all_nodes_certificate_validity` equivalent action to renew _all_ trusted nodes certificates).
+  - The existing `transition_node_to_trusted` proposal action now requires a new `valid_from` argument (and optional `validity_period_days`, which defaults to the value of ``--max-allowed-node-cert-validity-days`).
 - `ccf.historical.getStateRange` / `ccf.historical.dropCachedStates` JavaScript APIs to manually retrieve historical state in endpoints declared as `"mode": "readonly"` (#3033).
 
 ### Changed

doc/governance/common_member_operations.rst

@@ -136,4 +136,33 @@ The number of member shares required to restore the private ledger (``recovery_t
         "state": "Accepted"
     }
 
-.. note:: The new recovery threshold has to be in the range between 1 and the current number of active recovery members.
+.. note:: The new recovery threshold has to be in the range between 1 and the current number of active recovery members.
+
+Renewing Node Certificate
+-------------------------
+
+.. note:: Renewing the certificate of a node does not change the identity (public key) of that node but only its validity period.
+
+To renew the soon-to-be-expired certificate of a node, members should issue a ``set_node_certificate_validity`` proposal, specifying the date at which the validity period of the renewed certificate should start (``valid_from``), as well as its validity period in days (``validity_period_days``).
+
+The ``valid_from`` date argument should be a ASN1 UTCTime string, i.e. ``"YYMMDDhhmmssZ"``. The ``validity_period_days`` should be less than the validity period set by operators (see :ref:`operations/certificates:Node Certificates`).
+
+A sample proposal is:
+
+.. code-block:: bash
+
+    $ cat set_node_certificate_validity.json
+    {
+        "actions": [
+            {
+                "name": "set_node_certificate_validity",
+                "args": {
+                    "node_id": "86c0ccfab4b869abbc779937c51158c9dd2a130d58323643a3119e83b33dcf5c"
+                    "valid_from": "211019154318Z",
+                    "validity_period_days": 365
+                }
+            }
+        ]
+    }
+
+.. tip:: All currently trusted nodes certificates can be renewed at once using the ``set_all_nodes_certificate_validity`` proposal (same arguments minus ``node_id``).

doc/governance/proposals.rst

@@ -69,12 +69,12 @@ Some of these subcommands require additional arguments, such as the node ID or u
 
 .. code-block:: bash
 
-    $ python -m ccf.proposal_generator transition_node_to_trusted 6d566123a899afaea977c5fc0f7a2a9fef33f2946fbc4abefbc3e10ee597343f
+    $ python -m ccf.proposal_generator transition_node_to_trusted 6d566123a899afaea977c5fc0f7a2a9fef33f2946fbc4abefbc3e10ee597343f 211019154318Z
     SUCCESS | Writing proposal to ./trust_node_proposal.json
     SUCCESS | Wrote vote to ./trust_node_vote_for.json
 
     $ cat trust_node_proposal.json
-    {"actions": [{"name": "transition_node_to_trusted", "args": {"node_id": "6d566123a899afaea977c5fc0f7a2a9fef33f2946fbc4abefbc3e10ee597343f"}}]}
+    {"actions": [{"name": "transition_node_to_trusted", "args": {"node_id": "6d566123a899afaea977c5fc0f7a2a9fef33f2946fbc4abefbc3e10ee597343f", "valid_from": "211019154318Z"}}]}
 
     $ python -m ccf.proposal_generator --pretty-print --proposal-output-file add_pedro.json --vote-output-file vote_for_pedro.json set_user pedro_cert.pem
     SUCCESS | Writing proposal to ./add_pedro.json

doc/operations/certificates.rst

@@ -0,0 +1,15 @@
+Certificates
+============
+
+Since 2.x releases, the validity period of certificates is no longer hardcoded. This page describes how the validity period can instead be set by operators, and renewed by members.
+
+.. note:: The granularity for the validity period of nodes certificates is one day.
+
+Node Certificates
+-----------------
+
+At startup, operators can set the validity period for a node using the ``--initial-node-cert-validity-days`` CLI argument. The default value is set to 1 day and it is expected that members will issue a proposal to renew the certificate before it expires, when the service is open. Initial nodes certificates are valid from the current system time when the ``cchost`` executable is launched.
+
+The ``--max-allowed-node-cert-validity-days`` CLI argument (defaults to 365 days) can be used to set the maximum allowed validity period for nodes certificates when they are renewed by members. It is used as the default value for the validity period when a node certificate is renewed but the validity period is omitted.
+
+.. tip:: Once a node certificate has expired, clients will no longer trust the node serving their request. It is expected that operators and members will monitor the certificate validity dates with regards to current time and renew the node certificate before expiration. See :ref:`governance/common_member_operations:Renewing Node Certificate` for more details.

doc/operations/index.rst

@@ -26,6 +26,13 @@ This section describes how :term:`Operators` manage the different nodes constitu
 
     ---
 
+    :fa:`stamp` :doc:`certificates`
+    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+    Set and renew nodes and service x509 certificates.
+
+    ---
+
     :fa:`helicopter` :doc:`recovery`
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 

include/ccf/endpoint_registry.h

@@ -9,7 +9,6 @@
 #include "ds/json_schema.h"
 #include "ds/openapi.h"
 #include "http/http_consts.h"
-#include "node/certs.h"
 #include "node/endpoint_metrics.h"
 #include "node/rpc/serialization.h"
 

python/ccf/proposal_generator.py

@@ -226,8 +226,13 @@ def refresh_js_app_bytecode_cache(**kwargs):
 
 
 @cli_proposal
-def transition_node_to_trusted(node_id: str, **kwargs):
-    return build_proposal("transition_node_to_trusted", {"node_id": node_id}, **kwargs)
+def transition_node_to_trusted(
+    node_id: str, valid_from: str, validity_period_days: Optional[int] = None, **kwargs
+):
+    args = {"node_id": node_id, "valid_from": valid_from}
+    if validity_period_days is not None:
+        args["validity_period_days"] = validity_period_days
+    return build_proposal("transition_node_to_trusted", args, **kwargs)
 
 
 @cli_proposal
@@ -325,6 +330,26 @@ def set_jwt_public_signing_keys(issuer: str, jwks_path: str, **kwargs):
     return build_proposal("set_jwt_public_signing_keys", args, **kwargs)
 
 
+@cli_proposal
+def set_node_certificate_validity(
+    node_id: str, valid_from: str, validity_period_days: Optional[int] = None, **kwargs
+):
+    args = {"node_id": node_id, "valid_from": valid_from}
+    if validity_period_days is not None:
+        args["validity_period_days"] = validity_period_days
+    return build_proposal("set_node_certificate_validity", args, **kwargs)
+
+
+@cli_proposal
+def set_all_nodes_certificate_validity(
+    valid_from: str, validity_period_days: Optional[int] = None, **kwargs
+):
+    args = {"valid_from": valid_from}
+    if validity_period_days is not None:
+        args["validity_period_days"] = validity_period_days
+    return build_proposal("set_all_nodes_certificate_validity", args, **kwargs)
+
+
 if __name__ == "__main__":
     parser = argparse.ArgumentParser()
 

src/crypto/certs.h

@@ -0,0 +1,76 @@
+// Copyright (c) Microsoft Corporation. All rights reserved.
+// Licensed under the Apache 2.0 License.
+#pragma once
+
+#include "key_pair.h"
+#include "openssl/x509_time.h"
+#include "pem.h"
+
+#include <string>
+
+namespace crypto
+{
+  static std::string compute_cert_valid_to_string(
+    const std::string& valid_from, size_t validity_period_days)
+  {
+    // Note: As per RFC 5280, the validity period runs until "notAfter"
+    // _inclusive_ so substract one second from the validity period.
+    auto valid_to = OpenSSL::adjust_time(valid_from, validity_period_days, -1);
+    return OpenSSL::to_x509_time_string(OpenSSL::to_time_t(valid_to));
+  }
+
+  static Pem create_self_signed_cert(
+    const KeyPairPtr& key_pair,
+    const CertificateSubjectIdentity& csi,
+    const std::string& valid_from,
+    size_t validity_period_days)
+  {
+    return key_pair->self_sign(
+      csi,
+      true /* CA */,
+      valid_from,
+      compute_cert_valid_to_string(valid_from, validity_period_days));
+  }
+
+  static Pem create_endorsed_cert(
+    const Pem& csr,
+    const std::string& valid_from,
+    const std::string& valid_to,
+    const Pem& issuer_key_pair,
+    const Pem& issuer_cert)
+  {
+    return make_key_pair(issuer_key_pair)
+      ->sign_csr(issuer_cert, csr, false /* Not CA */, valid_from, valid_to);
+  }
+
+  static Pem create_endorsed_cert(
+    const Pem& csr,
+    const std::string& valid_from,
+    size_t validity_period_days,
+    const Pem& issuer_key_pair,
+    const Pem& issuer_cert)
+  {
+    return create_endorsed_cert(
+      csr,
+      valid_from,
+      compute_cert_valid_to_string(valid_from, validity_period_days),
+      issuer_key_pair,
+      issuer_cert);
+  }
+
+  static Pem create_endorsed_cert(
+    const KeyPairPtr& subject_key_pair,
+    const CertificateSubjectIdentity& csi,
+    const std::string& valid_from,
+    size_t validity_period_days,
+    const Pem& issuer_key_pair,
+    const Pem& issuer_cert)
+  {
+    return create_endorsed_cert(
+      subject_key_pair->create_csr(csi),
+      valid_from,
+      validity_period_days,
+      issuer_key_pair,
+      issuer_cert);
+  }
+}

src/crypto/key_pair.h

@@ -55,24 +55,32 @@ namespace crypto
     virtual Pem sign_csr(
       const Pem& issuer_cert,
       const Pem& signing_request,
-      bool ca = false) const = 0;
+      bool ca = false,
+      const std::optional<std::string>& valid_from = std::nullopt,
+      const std::optional<std::string>& valid_to = std::nullopt) const = 0;
 
     Pem self_sign(
       const std::string& name,
       const std::optional<SubjectAltName> subject_alt_name = std::nullopt,
-      bool ca = true) const
+      bool ca = true,
+      const std::optional<std::string>& valid_from = std::nullopt,
+      const std::optional<std::string>& valid_to = std::nullopt) const
     {
       std::vector<SubjectAltName> sans;
       if (subject_alt_name.has_value())
         sans.push_back(subject_alt_name.value());
       auto csr = create_csr({name, sans});
-      return sign_csr(Pem(0), csr, ca);
+      return sign_csr(Pem(0), csr, ca, valid_from, valid_to);
     }
 
-    Pem self_sign(const CertificateSubjectIdentity& csi, bool ca = true) const
+    Pem self_sign(
+      const CertificateSubjectIdentity& csi,
+      bool ca = true,
+      const std::optional<std::string>& valid_from = std::nullopt,
+      const std::optional<std::string>& valid_to = std::nullopt) const
     {
       auto csr = create_csr(csi);
-      return sign_csr(Pem(0), csr, ca);
+      return sign_csr(Pem(0), csr, ca, valid_from, valid_to);
     }
   };
 

src/crypto/mbedtls/key_pair.cpp

@@ -273,7 +273,11 @@ namespace crypto
   }
 
   Pem KeyPair_mbedTLS::sign_csr(
-    const Pem& issuer_cert, const Pem& signing_request, bool ca) const
+    const Pem& issuer_cert,
+    const Pem& signing_request,
+    bool ca,
+    const std::optional<std::string>& valid_from,
+    const std::optional<std::string>& valid_to) const
   {
     auto entropy = create_entropy();
     auto csr = mbedtls::make_unique<mbedtls::X509Csr>();
@@ -321,8 +325,13 @@ namespace crypto
 
     // Note: 825-day validity range
     // https://support.apple.com/en-us/HT210176
+    // Note: For the mbedtls implementation, we do not check that valid_from and
+    // valid_to are valid or chronological. See OpenSSL equivalent call for a
+    // safer implementation.
     MCHK(mbedtls_x509write_crt_set_validity(
-      crt.get(), "20210311000000", "20230611235959"));
+      crt.get(),
+      valid_from.value_or("20210311000000").c_str(),
+      valid_to.value_or("20230611235959").c_str()));
 
     MCHK(mbedtls_x509write_crt_set_basic_constraints(crt.get(), ca ? 1 : 0, 0));
     MCHK(mbedtls_x509write_crt_set_subject_key_identifier(crt.get()));

src/crypto/mbedtls/key_pair.h

@@ -55,6 +55,8 @@ namespace crypto
     virtual Pem sign_csr(
       const Pem& issuer_cert,
       const Pem& signing_request,
-      bool ca = false) const override;
+      bool ca = false,
+      const std::optional<std::string>& valid_from = std::nullopt,
+      const std::optional<std::string>& valid_to = std::nullopt) const override;
   };
 }

src/crypto/mbedtls/verifier.cpp

@@ -21,6 +21,19 @@ namespace crypto
     "-----BEGIN CERTIFICATE-----\n";
   static constexpr auto PEM_CERTIFICATE_FOOTER = "-----END CERTIFICATE-----\n";
 
+  static inline std::string to_x509_time_string(const mbedtls_x509_time& time)
+  {
+    // Returns ASN1 time string (YYYYMMDDHHMMSSZ)
+    return fmt::format(
+      "{:02}{:02}{:02}{:02}{:02}{:02}Z",
+      time.year,
+      time.mon,
+      time.day,
+      time.hour,
+      time.min,
+      time.sec);
+  }
+
   MDType Verifier_mbedTLS::get_md_type(mbedtls_md_type_t mdt) const
   {
     switch (mdt)
@@ -196,4 +209,11 @@ namespace crypto
     }
     return buf;
   }
+
+  std::pair<std::string, std::string> Verifier_mbedTLS::validity_period() const
+  {
+    return std::make_pair(
+      to_x509_time_string(cert->valid_from),
+      to_x509_time_string(cert->valid_to));
+  }
 }

src/crypto/mbedtls/verifier.h

@@ -29,5 +29,8 @@ namespace crypto
     virtual bool is_self_signed() const override;
 
     virtual std::string serial_number() const override;
+
+    virtual std::pair<std::string, std::string> validity_period()
+      const override;
   };
 }

src/crypto/openssl/key_pair.cpp

@@ -7,6 +7,7 @@
 #include "crypto/openssl/public_key.h"
 #include "hash.h"
 #include "openssl_wrappers.h"
+#include "x509_time.h"
 
 #define FMT_HEADER_ONLY
 #include <fmt/format.h>
@@ -215,7 +216,11 @@ namespace crypto
   }
 
   Pem KeyPair_OpenSSL::sign_csr(
-    const Pem& issuer_cert, const Pem& signing_request, bool ca) const
+    const Pem& issuer_cert,
+    const Pem& signing_request,
+    bool ca,
+    const std::optional<std::string>& valid_from,
+    const std::optional<std::string>& valid_to) const
   {
     X509* icrt = NULL;
     Unique_BIO mem(signing_request);
@@ -256,17 +261,19 @@ namespace crypto
 
     // Note: 825-day validity range
     // https://support.apple.com/en-us/HT210176
-    ASN1_TIME *before = NULL, *after = NULL;
-    OpenSSL::CHECKNULL(before = ASN1_TIME_new());
-    OpenSSL::CHECKNULL(after = ASN1_TIME_new());
-    OpenSSL::CHECK1(ASN1_TIME_set_string(before, "20210311000000Z"));
-    OpenSSL::CHECK1(ASN1_TIME_set_string(after, "20230611235959Z"));
-    OpenSSL::CHECK1(ASN1_TIME_normalize(before));
-    OpenSSL::CHECK1(ASN1_TIME_normalize(after));
-    OpenSSL::CHECK1(X509_set1_notBefore(crt, before));
-    OpenSSL::CHECK1(X509_set1_notAfter(crt, after));
-    ASN1_TIME_free(before);
-    ASN1_TIME_free(after);
+    Unique_X509_TIME not_before(valid_from.value_or("20210311000000Z"));
+    Unique_X509_TIME not_after(valid_to.value_or("20230611235959Z"));
+    if (!validate_chronological_times(not_before, not_after))
+    {
+      throw std::logic_error(fmt::format(
+        "Certificate cannot be created with not_before date {} > not_after "
+        "date {}",
+        to_x509_time_string(not_before),
+        to_x509_time_string(not_after)));
+    }
+
+    OpenSSL::CHECK1(X509_set1_notBefore(crt, not_before));
+    OpenSSL::CHECK1(X509_set1_notAfter(crt, not_after));
 
     X509_set_subject_name(crt, X509_REQ_get_subject_name(csr));
     X509_set_pubkey(crt, req_pubkey);