Merge pull request #1677 from NCEAS/feature-1665-x509-certs

Client certs & cross-signing

docker/docker-entrypoint.sh

@@ -145,6 +145,7 @@ elif [[ $1 = "catalina.sh" ]]; then
     # Make sure all default directories are available
     mkdir -p \
         /var/metacat/.metacat    \
+        /var/metacat/certs       \
         /var/metacat/config      \
         /var/metacat/data        \
         /var/metacat/documents   \

helm/README.md

@@ -310,64 +310,88 @@ ingress:
 
 ### Setting up Certificates for DataONE Replication
 
-DataONE Replication relies on mutual authentication with x509 client-side certs. For full details
-on becoming part of the DataONE network, see the
-[Metacat Administrator's Guide](https://knb.ecoinformatics.org/knb/docs/dataone.html) and
-[Authentication and Authorization in DataONE](https://releases.dataone.org/online/api-documentation-v2.0.1/design/AuthorizationAndAuthentication.html).
-To set up the certificates for each Metacat Member Node (MN) or Coordinating Node (CN):
-
+For full details on becoming part of the DataONE network, see the [Metacat Administrator's Guide
+](https://knb.ecoinformatics.org/knb/docs/dataone.html) and
+[Authentication and Authorization in DataONE
+](https://releases.dataone.org/online/api-documentation-v2.0.1/design/AuthorizationAndAuthentication.html)
+.
+
+DataONE Replication relies on mutual authentication with x509 client-side certs. As a DataONE
+Member Node (MN) or Coordinating Node (CN), your metacat instance will act as both a server and
+as a client, at different times during the replication process. It is therefore necessary to
+configure certificates and settings for both these roles.
+
+#### Prerequisites
 1. First make sure you have the Kubernetes version of the
    [nginx ingress installed](#networking-and-x509-certificates)
 1. Ensure [HTTPS access is set up](#setting-up-a-tls-certificates-for-https-traffic) and
-   working correctly
+   working correctly. This allows other nodes, acting as "clients" to verify your server's identity
+   during mutual authentication.
 1. From the DataONE administrators ([support@dataone.org](mailto:support@dataone.org)), obtain:
 
-   1. a copy of the DataONE *Certificate Authority (CA) certificate*, in order to validate client
-      certificates signed by that authority.
+   1. a copy of the **DataONE Certificate Authority (CA) certificate chain**. This enables your node
+      (when acting as server) to validate other nodes' client certificates signed by that authority.
+   1. a **Client Certificate**, that uniquely identifies your Metacat instance. This allows another
+      node (acting as server) to verify your node's identity (acting as "client") during mutual
+      authentication.
 
-   1. a *client certificate*, that uniquely identifies your Metacat instance.
+#### Install the CA Chain
 
-1. Create the Kubernetes Secret (named `ca-secret`) to hold the ca cert (assuming it's in a file named
-   `DataONECAChain.crt`):
+1. Create the Kubernetes Secret (named `ca-secret`) to hold the ca chain (e.g. assuming it's in
+    a file named `DataONECAChain.crt`):
 
     ```shell
-        kubectl create secret generic ca-secret --from-file=ca.crt=DataONECAChain.crt
-        # (don't forget to define a non-default namespace if necessary, using `-n myNameSpace`)
+    kubectl create secret generic ca-secret --from-file=ca.crt=DataONECAChain.crt
+    # (don't forget to define a non-default namespace if necessary, using `-n myNameSpace`)
     ```
 
 1. Run the [`configure-nginx-mutual-auth.sh` script](./admin/configure-nginx-mutual-auth.sh).
-   This will configure your nginx ingress controller to add a shared secret header that Metacat
-   requires for added security. 
+  This will configure your nginx ingress controller to add a shared secret header that Metacat
+  requires for added security. Ensure you have already defined a value for this shared secret,
+  named `METACAT_DATAONE_CERT_FROM_HTTP_HEADER_PROXY_KEY`, [in metacat Secrets](#secrets).
+
+#### Install the Client Certificate
+
+   1. Create the Kubernetes Secret (named `<yourReleaseName>-d1-client-secret`) to hold the Client
+      Certificate (e.g. assuming it's in a file named `urn_node_TestNAME.pem`):
 
-1. Set the correct parameters in `values.yaml`:
+      ```shell
+      kubectl create secret generic <yourReleaseName>-d1-client-secret \
+                                    --from-file=d1client.crt=urn_node_TestNAME.pem
+      # (don't forget to define a non-default namespace if necessary, using `-n myNameSpace`)
+      ```
+
+#### Set the correct parameters in `values.yaml`
+
+1. Enable the shared secret header
 
     ```yaml
     metacat:
-
       dataone.certificate.fromHttpHeader.enabled: true
+    ```
 
+1. set the CA secret name
 
+    ```yaml
     ingress:
-
       className: "nginx"
-
       d1CaCertSecretName: ca-secret
     ```
 
-1. re-install or upgrade to apply the changes
+1. Finally, re-install or upgrade to apply the changes
 
-See Appendix 3 for help with troubleshooting
+See [Appendix 3](#appendix-3-troubleshooting) for help with troubleshooting
 
 ---
 
 ## Appendices
 
 ## Appendix 1: Self-Signing TLS Certificates for HTTPS Traffic
 
-Also see the [Kubernetes nginx documentation](https://kubernetes.github.
-io/ingress-nginx/user-guide/tls)
-
 > **NOTE: For development and testing purposes only!**
+>
+> Also see the [Kubernetes nginx documentation](<https://kubernetes.github>.
+> io/ingress-nginx/user-guide/tls)
 
 You can create your own self-signed certificate as follows:
 
@@ -397,12 +421,15 @@ Whatever hostname you are using, don't forget to set the
 
 ## Appendix 2: Self-Signing Certificates for Testing Mutual Authentication
 
-Also see the [Kubernetes nginx documentation
-](https://kubernetes.github.io/ingress-nginx/examples/PREREQUISITES/#client-certificate-authentication)
-
 > **NOTE: For development and testing purposes only!**
+>
+> Also see the [Kubernetes nginx documentation
+> ](https://kubernetes.github.io/ingress-nginx/examples/PREREQUISITES/#client-certificate-authentication)
 
-You can create your own self-signed certificate as follows:
+Assuming you already have a [server certificate installed
+](#setting-up-a-tls-certificates-for-https-traffic) (either signed by a trusted CA or [self-signed
+](#appendix-1-self-signing-tls-certificates-for-https-traffic) for development & testing), you can
+create your own self-signed Mutual Auth Client certificate and CA certificate as follows:
 
 1. Generate the CA Key and Certificate:
 
@@ -411,22 +438,16 @@ You can create your own self-signed certificate as follows:
             -subj '/CN=My Cert Authority'
     ```
 
-1. Generate the Server Key, and Certificate and Sign with the CA Certificate:
+1. Generate the Client Key and Certificate Signing Request:
 
     ```shell
-    openssl req -new -newkey rsa:4096 -keyout server.key -out server.csr -nodes \
-            -subj '/CN=mydomain.com'
-
-    openssl x509 -req -sha256 -days 365 -in server.csr -CA ca.crt -CAkey ca.key \
-            -set_serial 01 -out server.crt
+    openssl req -new -newkey rsa:4096 -keyout client.key -out client.csr -nodes \
+            -subj '/CN=My Client'
     ```
 
-1. Generate the Client Key, and Certificate and Sign with the CA Certificate:
+1. Sign with the CA Key:
 
     ```shell
-    openssl req -new -newkey rsa:4096 -keyout client.key -out client.csr -nodes \
-            -subj '/CN=My Client'
-
     openssl x509 -req -sha256 -days 365 -in client.csr -CA ca.crt -CAkey ca.key \
             -set_serial 02 -out client.crt
     ```
@@ -471,21 +492,24 @@ You can check the configuration as follows:
           # more lines above
           nginx.ingress.kubernetes.io/auth-tls-pass-certificate-to-upstream: "true"
           nginx.ingress.kubernetes.io/auth-tls-secret: default/ca-secret
-          ## above may differ for you: <namespace>/<ingress.d1CaCertSecretName>
+          ## above may differ for you. Format is: <namespace>/<ingress.d1CaCertSecretName>
           nginx.ingress.kubernetes.io/auth-tls-verify-client: optional_no_ca
           nginx.ingress.kubernetes.io/auth-tls-verify-depth: "10"
     ```
+
     If you don't see these, or they are incorrect, check values.yaml for:
 
     ```yaml
       metacat:
         dataone.certificate.fromHttpHeader.enabled: #true for mutual auth
 
       ingress:
-        tls: # needs to have been set up properly - see above
+        tls: # needs to have been set up properly [ref 1]
 
-        d1CaCertSecretName: # needs to match secret name holding your ca cert
+        d1CaCertSecretName: # needs to match secret name holding your ca cert chain [ref 2]
     ```
+    - *[[ref 1]](#setting-up-a-tls-certificates-for-https-traffic)*
+    - *[[ref 2]](#install-the-ca-chain)*
 
 1. then check the configmaps for the ingress controller:
 

helm/config/metacat-site.properties

@@ -61,6 +61,9 @@ server.internalPort=8080
 # Need to override this because HazelcastIndexEventLog tries to write a file here.
 solr.homeDir=/var/metacat/solr-home-legacy
 
+# Override the Member Node client certificate filename to match secret setup - see ../README
+D1Client.certificate.file=/var/metacat/certs/d1client.crt
+
 # Pretend the config is all done.
 # TODO - handle this more elegantly? See: https://github.com/NCEAS/metacat/issues/1638
 #

helm/templates/statefulset.yaml

@@ -49,6 +49,9 @@ spec:
             - name: {{ .Release.Name }}-volume
               mountPath: /var/metacat
               readOnly: false
+            - name: {{ .Release.Name }}-d1-cert-volume
+              mountPath: /var/metacat/certs/
+              readOnly: true
           env:
             - name: METACAT_IS_RUNNING_IN_A_CONTAINER
               value: "true"
@@ -84,6 +87,10 @@ spec:
         configMap:
           name: {{ .Release.Name }}-configfiles
           defaultMode: 0644
+      - name: {{ .Release.Name }}-d1-cert-volume
+        secret:
+          secretName: {{ .Release.Name }}-d1-client-secret
+          optional: true
 {{- if not (.Values.persistence.enabled) }}
       - name: {{ .Release.Name }}-volume
         emptyDir: {}