From 72592889248ae65cec1b45caf720e903bad78651 Mon Sep 17 00:00:00 2001 From: Giuseppe Capizzi Date: Thu, 22 Dec 2022 16:26:57 +0000 Subject: [PATCH] Document global.containerRegistryCACertSecret --- INSTALL.md | 105 ++++++++++-------- ...-container-registry-signed-by-custom-ca.md | 16 --- 2 files changed, 59 insertions(+), 62 deletions(-) delete mode 100644 docs/using-container-registry-signed-by-custom-ca.md diff --git a/INSTALL.md b/INSTALL.md index adb09cc37..2928e15e7 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -7,28 +7,28 @@ The following lines will guide you through the process of deploying a [released ## Prerequisites -- Tools: - - [kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl); - - [cf](https://docs.cloudfoundry.org/cf-cli/install-go-cli.html) CLI version 8.5 or greater; - - [Helm](https://helm.sh/docs/intro/install/). -- Resources: - - Kubernetes cluster of one of the [upstream releases](https://kubernetes.io/releases/); - - Container Registry on which you have write permissions. +- Tools: + - [kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl); + - [cf](https://docs.cloudfoundry.org/cf-cli/install-go-cli.html) CLI version 8.5 or greater; + - [Helm](https://helm.sh/docs/intro/install/). +- Resources: + - Kubernetes cluster of one of the [upstream releases](https://kubernetes.io/releases/); + - Container Registry on which you have write permissions. This document was tested on: -- [EKS](https://aws.amazon.com/eks/), using AWS' [Elastic Container Registry (ECR)](https://aws.amazon.com/ecr/) (see [_Install Korifi on EKS_](./INSTALL.EKS.md)); -- [GKE](https://cloud.google.com/kubernetes-engine), using GCP's [Artifact Registry](https://cloud.google.com/artifact-registry); -- [kind](https://kind.sigs.k8s.io/), using [DockerHub](https://hub.docker.com/) (see [_Install Korifi on kind_](./INSTALL.kind.md)). +- [EKS](https://aws.amazon.com/eks/), using AWS' [Elastic Container Registry (ECR)](https://aws.amazon.com/ecr/) (see [_Install Korifi on EKS_](./INSTALL.EKS.md)); +- [GKE](https://cloud.google.com/kubernetes-engine), using GCP's [Artifact Registry](https://cloud.google.com/artifact-registry); +- [kind](https://kind.sigs.k8s.io/), using [DockerHub](https://hub.docker.com/) (see [_Install Korifi on kind_](./INSTALL.kind.md)). ## Initial setup The following environment variables will be needed throughout this guide: -- `ROOT_NAMESPACE`: the namespace at the root of the Korifi org and space hierarchy. The default value is `cf`. -- `KORIFI_NAMESPACE`: the namespace in which Korifi will be installed. -- `ADMIN_USERNAME`: the name of the Kubernetes user who will have CF admin privileges on the Korifi installation. For security reasons, you should choose or create a user that is different from your cluster admin user. To provision new users, follow the user management instructions specific for your cluster's [authentication configuration](https://kubernetes.io/docs/reference/access-authn-authz/authentication/) or create a [new (short-lived) client certificate for user authentication](https://kubernetes.io/docs/reference/access-authn-authz/certificate-signing-requests/#normal-user). -- `BASE_DOMAIN`: the base domain used by both the Korifi API and, by default, all apps running on Korifi. +- `ROOT_NAMESPACE`: the namespace at the root of the Korifi org and space hierarchy. The default value is `cf`. +- `KORIFI_NAMESPACE`: the namespace in which Korifi will be installed. +- `ADMIN_USERNAME`: the name of the Kubernetes user who will have CF admin privileges on the Korifi installation. For security reasons, you should choose or create a user that is different from your cluster admin user. To provision new users, follow the user management instructions specific for your cluster's [authentication configuration](https://kubernetes.io/docs/reference/access-authn-authz/authentication/) or create a [new (short-lived) client certificate for user authentication](https://kubernetes.io/docs/reference/access-authn-authz/certificate-signing-requests/#normal-user). +- `BASE_DOMAIN`: the base domain used by both the Korifi API and, by default, all apps running on Korifi. Here are the example values we'll use in this guide: @@ -39,10 +39,6 @@ ADMIN_USERNAME="cf-admin" BASE_DOMAIN="korifi.example.org" ``` -### Registries with Custom CA - -See [_Using container registry signed by custom CA_](docs/using-container-registry-signed-by-custom-ca.md). - ### Free Dockerhub accounts DockerHub allows only one private repository per free account. In case the DockerHub account you configure Korifi with has the `private` [default repository privacy](https://hub.docker.com/settings/default-privacy) enabled, then Korifi would only be able to create a single repository and would get `UNAUTHORIZED: authentication required` error when trying to push to a subsequent repository. This could either cause build errors during `cf push`, or the Kpack cluster builder may never become ready. Therefore you should either set the default repository privacy to `public`, or upgrade your DockerHub subscription plan. As of today, the `Pro` subscription plan provides unlimited private repositories. @@ -116,14 +112,40 @@ kubectl --namespace "$ROOT_NAMESPACE" create secret docker-registry image-regist Make sure the value of `--docker-server` is a valid [URI authority](https://datatracker.ietf.org/doc/html/rfc3986#section-3.2). -- If using **DockerHub**: - - `--docker-server` should be `https://index.docker.io/v1/`; - - `--docker-username` should be your DockerHub user; - - `--docker-password` can be either your DockerHub password or a [generated personal access token](https://hub.docker.com/settings/security?generateToken=true). -- If using **GCR**: - - `--docker-server` should be `gcr.io`; - - `--docker-username` should be `_json_key`; - - `--docker-password` should be the JSON-formatted access token for a service account that has permission to manage images in GCR. +- If using **DockerHub**: + - `--docker-server` should be `https://index.docker.io/v1/`; + - `--docker-username` should be your DockerHub user; + - `--docker-password` can be either your DockerHub password or a [generated personal access token](https://hub.docker.com/settings/security?generateToken=true). +- If using **GCR**: + - `--docker-server` should be `gcr.io`; + - `--docker-username` should be `_json_key`; + - `--docker-password` should be the JSON-formatted access token for a service account that has permission to manage images in GCR. + +### TLS certificates + +Self-signed TLS certificates are generated automatically by the installation if `global.generateIngressCertificates` has been set to `true`. + +If you want to generate certificates yourself, you should not set the `global.generateIngressCertificates` value, and instead provide your certificates to Korifi by creating two TLS secrets in `$KORIFI_NAMESPACE`: + +1. `korifi-api-ingress-cert`; +1. `korifi-workloads-ingress-cert`. + +### Container registry Certificate Authority + +Korifi can be configured to use a custom Certificate Authority when contacting the container registry. To do so, first create a `Secret` containing the CA certificate: + +```sh +kubectl --namespace "$KORIFI_NAMESPACE" create secret generic \ + --from-file=ca.crt= +``` + +You can then specify the `` using the `global.containerRegistryCACertSecret`. + +> **Warning** +> Kpack does not support self-signed/internal CA configuration out of the box (see [pivotal/kpack#207](https://github.com/pivotal/kpack/issues/207)). +> In order to make Kpack trust your CA certificate, you will have to inject it in both the Kpack controller and the Kpack build pods. +> * The [`kpack-controller` `Deployment`](https://github.com/pivotal/kpack/blob/main/config/controller.yaml) can be modified to mount a `Secret` similar to the one created above: see the [Korifi API `Deployment`](https://github.com/cloudfoundry/korifi/blob/registry-ca/helm/api/templates/deployment.yaml) for an example of how to do this. +> * For the build pods you can use the [cert-injection-webhook](https://github.com/vmware-tanzu/cert-injection-webhook), configured on the `kpack.io/build` label. ## Install Korifi @@ -147,13 +169,13 @@ helm install korifi https://github.com/cloudfoundry/korifi/releases/download/v.azurecr.io/foo/bar/korifi- | \.azurecr.io/foo/bar/korifi-\-packages | Repositories are created dynamically during push by ACR | -| DockerHub | index.docker.io/\/ | index.docker.io/\/\-packages | Docker does not support nested repositories | -| ECR | \.dkr.ecr.\.amazonaws.com/foo/bar/korifi- | \.dkr.ecr.\.amazonaws.com/foo/bar/korifi-\-packages | Korifi will create the repository before pushing, as dynamic repository creation is not posssible on ECR | -| GAR | \-docker.pkg.dev/\/foo/bar/korifi- | \-docker.pkg.dev/\/foo/bar/korifi-\-packages | The `foo` repository must already exist in GAR | -| GCR | gcr.io/\/foo/bar/korifi- | gcr.io/\/foo/bar/korifi-\-packages | Repositories are created dynamically during push by GCR | +| Registry | containerRepositoryPrefix | Resultant Image Ref | Notes | +| --------- | ------------------------------------------------------------ | ------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------- | +| ACR | `.azurecr.io/foo/bar/korifi-` | `.azurecr.io/foo/bar/korifi--packages` | Repositories are created dynamically during push by ACR | +| DockerHub | `index.docker.io//` | `index.docker.io//-packages` | Docker does not support nested repositories | +| ECR | `.dkr.ecr..amazonaws.com/foo/bar/korifi-` | `.dkr.ecr..amazonaws.com/foo/bar/korifi--packages` | Korifi will create the repository before pushing, as dynamic repository creation is not posssible on ECR | +| GAR | `-docker.pkg.dev//foo/bar/korifi-` | `-docker.pkg.dev//foo/bar/korifi--packages` | The `foo` repository must already exist in GAR | +| GCR | `gcr.io//foo/bar/korifi-` | `gcr.io//foo/bar/korifi--packages` | Repositories are created dynamically during push by GCR | The chart provides various other values that can be set. See [`README.helm.md`](./README.helm.md) for details. @@ -161,26 +183,17 @@ The chart provides various other values that can be set. See [`README.helm.md`]( If you are using an authentication proxy with your cluster to enable SSO, you must set the following chart values: -- `api.authProxy.host`: IP address of your cluster's auth proxy; -- `api.authProxy.caCert`: CA certificate of your cluster's auth proxy. +- `api.authProxy.host`: IP address of your cluster's auth proxy; +- `api.authProxy.caCert`: CA certificate of your cluster's auth proxy. ## Post-install Configuration -### TLS certificates - -Self-signed TLS certificates are generated automatically by the installation if `global.generateIngressCertificates` has been set to `true`. - -If you want to generate certificates yourself, you should not set the `global.generateIngressCertificates` value, and instead provide your certificates to Korifi by creating two TLS secrets in `$KORIFI_NAMESPACE`: - -1. `korifi-api-ingress-cert`; -1. `korifi-workloads-ingress-cert`. - ### DNS Create DNS entries for the Korifi API and for the apps running on Korifi. They should match the Helm values used to [deploy Korifi](#deploy-korifi): -- The Korifi API entry should match the `api.apiServer.url` value. In our example, that would be `api.korifi.example.org`. -- The apps entry should be a wildcard matching the `global.defaultAppDomainName` value. In our example, `*.apps.korifi.example.org`. +- The Korifi API entry should match the `api.apiServer.url` value. In our example, that would be `api.korifi.example.org`. +- The apps entry should be a wildcard matching the `global.defaultAppDomainName` value. In our example, `*.apps.korifi.example.org`. The DNS entries should point to the load balancer endpoint created by Contour when installed. To discover your endpoint, run: diff --git a/docs/using-container-registry-signed-by-custom-ca.md b/docs/using-container-registry-signed-by-custom-ca.md deleted file mode 100644 index c728205c9..000000000 --- a/docs/using-container-registry-signed-by-custom-ca.md +++ /dev/null @@ -1,16 +0,0 @@ -# Use container registry signed by custom CAs -Korifi can be configured to use a custom registry that has a self-signed certicate. -Korifi accepts the `REGISTRY_CA_FILE` environment variable to point to a mounted secret containing the registry's CA cert. - -## Configure korifi-api - -- Create a secret containing the registry's CA cert in the `korifi-api-system` namespace. (see instructions) - For example: `kubectl -n korifi-api-system create secret generic registry-ca-cert --from-file=cacerts.pem=./ca-cert.pem` -- Update [deployment definition](https://github.com/cloudfoundry/korifi/blob/main/api/config/base/deployment.yaml) to mount the secret (see [instructions](https://kubernetes.io/docs/concepts/configuration/secret/#using-secrets-as-files-from-a-pod)) -- Set `REGISTRY_CA_FILE` environment variable on the deployment to point to the path of the mounted ca.crt file. - -## Configure kpack-image-builder -Use the same instructions as above, but create secret in the `korifi-kpack-build-system` namespace and use this [deployment definition](https://github.com/cloudfoundry/korifi/blob/main/kpack-image-builder/config/manager/manager.yaml) - -## Kpack -Kpack does not support self-signed/internal CA configuration out of the box [see: pivotal/kpack#207](https://github.com/pivotal/kpack/issues/207) and providing support for that is not something in scope for us to take on. Operators can modify the Kpack deployment, using something like the [cert-injection-webhook](https://github.com/vmware-tanzu/cert-injection-webhook) on the kpack Pods, or bring their own build reconciler in these cases. \ No newline at end of file