chore: handle upgrade path for functional layers, add doc for usage (#…

…896)

## Description

Changes included:
1. Functional layers upgrade was not successful when I tested initially.
There were ownership issues with the `manifests` we use for istio. To
resolve that problem I moved the manifests into a chart and added an
action to update the ownership before the upgrade. I also removed the
pepr action that we have had for several releases since it was needed as
a one-time upgrade step (similar to this one).
2. Added a lightweight doc on the usage of functional layers with a very
brief explanation and warning as well as a full example of a bundle
pulling in the layers.
3. A few misc link fixes and other things to follow the astro docs
switch.

## Related Issue

Fixes #868

Fixes #900

## Type of change

- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [x] Other (security config, docs update, etc)

## Checklist before merging

- [x] Test, docs, adr added or updated as needed
- [x] [Contributor
Guide](https://github.com/defenseunicorns/uds-template-capability/blob/main/CONTRIBUTING.md)
followed

---------

Co-authored-by: Noah <40781376+noahpb@users.noreply.github.com>

README.md

@@ -1,6 +1,6 @@
 # Unicorn Delivery Service - Core (UDS Core)
 
-## [UDS Core Docs](https://uds.defenseunicorns.com/core/)
+## [UDS Core Overview](https://uds.defenseunicorns.com/reference/uds-core/overview/)
 
 UDS Core establishes a secure baseline for cloud-native systems and ships with compliance documentation and first-class support for airgap/egress-limited systems. Based on the work of [Platform One](https://p1.dso.mil), UDS Core expands on the security posture of [Big Bang](https://repo1.dso.mil/big-bang/bigbang) while providing advanced automation with the [UDS Operator](./src/pepr/operator/README.md) and [UDS Policy Engine](./src/pepr/policies/README.md). UDS Core is a collection of several individual applications combined into a single [Zarf](https://zarf.dev) package and we recommend using [UDS CLI](https://github.com/defenseunicorns/uds-cli?tab=readme-ov-file#install) to deploy it as a [UDS Bundle](#using-uds-core-in-production).
 

docs/reference/UDS Core/dns.md

@@ -1,24 +1,23 @@
 ---
 title: DNS Configuration
-type: docs
-weight: 2
 ---
 
-UDS Core deploys two Gateways by default - a Tenant Gateway for end-user applications and an Admin Gateway for administrative applications. You can read more about Istio configuration in UDS Core [here](https://uds.defenseunicorns.com/core/configuration/istio/ingress/). This section covers how to configure DNS for these Gateways.
+UDS Core deploys two Gateways by default - a Tenant Gateway for end-user applications and an Admin Gateway for administrative applications. You can read more about Istio configuration in UDS Core [here](https://uds.defenseunicorns.com/reference/configuration/ingress/). This section covers how to configure DNS for these Gateways.
 
 ### Domain Configuration
 Each Gateway is associated to a wildcard DNS entry that is derived from the `DOMAIN` [variable](https://github.com/defenseunicorns/uds-core/blob/e624d73f79bd6739b6808fbdbf5ca75ebb7c1d3c/src/istio/zarf.yaml#L8) in the UDS Core Istio package. When deploying UDS Core, you can expect two Gateways to be created that match the following domain names:
 - `*.<DOMAIN>` / Tenant Gateway
 - `*.admin.<DOMAIN>` / Admin Gateway
 
-{{% alert-note %}}
-The default value for `DOMAIN` is `uds.dev`, which is intended for development purposes only. For non-development purposes, you should override this value by specifying a value for `domain` in your `uds-config.yaml`. You can find instructions on how to do so [here](https://uds.defenseunicorns.com/core/configuration/istio/ingress/#configure-domain-name-and-tls-for-istio-gateways). 
-{{% /alert-note %}}
+:::note
+The default value for `DOMAIN` is `uds.dev`, which is intended for development purposes only. For non-development purposes, you should override this value by specifying a value for `domain` in your `uds-config.yaml`. You can find instructions on how to do so [here](https://uds.defenseunicorns.com/reference/configuration/ingress/#configure-domain-name-and-tls-for-istio-gateways). 
+:::
 
 ### Bundle Configuration
-{{% alert-note %}}
+
+:::note
 UDS Core does not include any cloud provider specific configuration by default. Additional overrides are required to deploy UDS Core on a given provider. This section will refer to AWS, but values can be substituted as needed for other providers.
-{{% /alert-note %}}
+:::
 
 The Admin and Tenant Gateways will be each be bound to an external Load Balancer that is exposed on TCP ports 80 and 443 by default. The Admin Gateway should be configured to use an internal facing Load Balancer and the Tenant Gateway should be configured to use an external facing Load Balancer. Below is an example of overrides that would accomplish this:
 ```yaml
@@ -71,4 +70,4 @@ istio-admin-gateway         admin-ingressgateway                             Loa
 istio-tenant-gateway        tenant-ingressgateway                            LoadBalancer   10.43.47.182    k8s-istioten-tenant...elb.us-east-1.amazonaws.com  15021:31222/TCP,80:30456/TCP,443:32508/TCP  1h
 ```
 
-From here, you can register your domain and/or create DNS records for your environment that point to the appropriate Gateways/Load Balancers. Refer to your DNS provider's documentation. 
+From here, you can register your domain and/or create DNS records for your environment that point to the appropriate Gateways/Load Balancers. Refer to your DNS provider's documentation. 

docs/reference/UDS Core/functional-layers.md

@@ -0,0 +1,51 @@
+---
+title: Functional Layers
+---
+
+## Background
+
+Context on the inclusion of "functional layers" can be viewed in our [ADR](https://github.com/defenseunicorns/uds-core/blob/main/docs/adrs/0002-uds-core-functional-layers.md). In short, UDS Core publishes smaller Zarf packages that contain subsets of core's capabilities, grouped by their function (such as monitoring, logging, backup/restore, etc) to allow more flexibility in deployment. This helps to support resource constrained environments (edge deployments) and other situations where an environment has different needs than the default core stack.
+
+Each layer is published as an individual OCI Zarf package. Package sources can be viewed under the [`packages` directory](https://github.com/defenseunicorns/uds-core/tree/main/packages), with each folder containing a readme detailing the contents and any dependencies. All layers assume the requirement of the base layer which provides Istio, the UDS Operator, and UDS Policy Engine.
+
+:::caution
+By removing pieces of core from your deployment you may affect your security and compliance posture as well as reduce functionality of the stack. Deploying core using these layers should be the exception in most cases and only done after carefully weighing needs for your environment.
+:::
+
+## Example Usage
+
+Functional layers are designed to be combined into a UDS bundle for deployment. The example below shows all layers in the correct order. Keep in mind that 'base' must always be the first layer, and any other layers should follow based on their dependency order. When building your bundle, you can skip layers that don't fit your deployment needs and apply overrides to individual layers as needed. Ensure all layers are using the same version for compatibility.
+
+```yaml
+kind: UDSBundle
+metadata:
+  name: functional-layer-core-bundle
+  description: An example bundle for deploying all of core using functional layers
+  version: "0.1.0"
+
+packages:
+  - name: core-base
+    repository: ghcr.io/defenseunicorns/packages/uds/core-base
+    ref: 0.29.0-upstream
+  - name: core-identity-authorization
+    repository: ghcr.io/defenseunicorns/packages/uds/core-identity-authorization
+    ref: 0.29.0-upstream
+  - name: core-metrics-server
+    repository: ghcr.io/defenseunicorns/packages/uds/core-metrics-server
+    ref: 0.29.0-upstream
+  - name: core-runtime-security
+    repository: ghcr.io/defenseunicorns/packages/uds/core-runtime-security
+    ref: 0.29.0-upstream
+  - name: core-logging
+    repository: ghcr.io/defenseunicorns/packages/uds/core-logging
+    ref: 0.29.0-upstream
+  - name: core-monitoring
+    repository: ghcr.io/defenseunicorns/packages/uds/core-monitoring
+    ref: 0.29.0-upstream
+  - name: core-ui
+    repository: ghcr.io/defenseunicorns/packages/uds/core-ui
+    ref: 0.29.0-upstream
+  - name: core-backup-restore
+    repository: ghcr.io/defenseunicorns/packages/uds/core-backup-restore
+    ref: 0.29.0-upstream
+```

docs/reference/UDS Core/prerequisites.md

@@ -1,5 +1,7 @@
 ---
-title: UDS Core Prerequisites
+title: Prerequisites
+sidebar:
+    order: 2
 ---
 
 `UDS Core` can run in any [CNCF conformant Kubernetes distribution](https://www.cncf.io/training/certification/software-conformance/), but sometimes customizations are needed based on environments. This is an attempt to document and link to relevant information to aid in setting up your Kubernetes environment and hosts to ensure a successful `UDS Core` installation.  

docs/reference/configuration/ingress.md

@@ -36,7 +36,7 @@ packages:
 
 By default, the UDS Core Istio Gateways are set up to use the `uds.dev` domain and have a valid TLS certificate packaged.  You will want to change the domain name for your environment and provide a valid TLS certificate for this domain.
 
-You can set the TLS certs via overrides in a [UDS Bundle](https://uds.defenseunicorns.com/bundles/) (see below). UDS Core Istio Gateways default to only supporting TLS v1.3, but this can also be overridden per gateway if clients use TLS 1.2 (as seen in the tenant gateway example `value` below).
+You can set the TLS certs via overrides in a [UDS Bundle](https://uds.defenseunicorns.com/structure/bundles/) (see below). UDS Core Istio Gateways default to only supporting TLS v1.3, but this can also be overridden per gateway if clients use TLS 1.2 (as seen in the tenant gateway example `value` below).
 
 ```yaml
 kind: UDSBundle
@@ -75,7 +75,7 @@ packages:
 
 You can then either use environment variables (`UDS_ADMIN_TLS_CERT`, `UDS_ADMIN_TLS_KEY`, `UDS_TENANT_TLS_CERT`, and `UDS_TENANT_TLS_KEY`) or a config file to configure the certs for each gateway. These values should be base64 encoded strings of the TLS certificate and key for the admin and tenant gateways respectively.
 
-Domain should be set via your [uds-config](https://uds.defenseunicorns.com/cli/quickstart-and-usage/#variables-and-configuration) file using the shared key to override the Zarf Domain Variable (see example `uds-config.yaml` below).
+Domain should be set via your [uds-config](https://uds.defenseunicorns.com/reference/cli/quickstart-and-usage/#variables-and-configuration) file using the shared key to override the Zarf Domain Variable (see example `uds-config.yaml` below).
 
 ```yaml
 shared:
@@ -91,5 +91,5 @@ variables:
 ```
 
 :::note
-If you are using Private PKI or self-signed certificates for your tenant certificates it is necessary to additionally configure `UDS_CA_CERT` with additional [trusted certificate authorities](https://uds.defenseunicorns.com/core/configuration/uds-operator/#trusted-certificate-authority).
+If you are using Private PKI or self-signed certificates for your tenant certificates it is necessary to additionally configure `UDS_CA_CERT` with additional [trusted certificate authorities](https://uds.defenseunicorns.com/reference/configuration/uds-operator/#trusted-certificate-authority).
 :::

docs/reference/configuration/uds-operator.md

@@ -181,7 +181,7 @@ variables:
     CA_CERT: <base64 encoded certificate authority>
 ```
 
-See [configuring Istio Ingress](https://uds.defenseunicorns.com/core/configuration/istio/ingress/#configure-domain-name-and-tls-for-istio-gateways) for the relevant documentation on configuring ingress certificates.
+See [configuring Istio Ingress](https://uds.defenseunicorns.com/reference/configuration/ingress/#configure-domain-name-and-tls-for-istio-gateways) for the relevant documentation on configuring ingress certificates.
 
 ### Creating a UDS Package with a Device Flow client
 

src/istio/common/chart/.helmignore

@@ -0,0 +1,23 @@
+# Patterns to ignore when building packages.
+# This supports shell glob matching, relative path matching, and
+# negation (prefixed with !). Only one pattern per line.
+.DS_Store
+# Common VCS dirs
+.git/
+.gitignore
+.bzr/
+.bzrignore
+.hg/
+.hgignore
+.svn/
+# Common backup files
+*.swp
+*.bak
+*.tmp
+*.orig
+*~
+# Various IDEs
+.project
+.idea/
+*.tmproj
+.vscode/

src/istio/common/chart/Chart.yaml

@@ -0,0 +1,19 @@
+# SPDX-License-Identifier: AGPL-3.0-or-later OR Commercial
+apiVersion: v2
+name: uds-global-istio-config
+description: Global Istio configuration for UDS
+
+# A chart can be either an 'application' or a 'library' chart.
+#
+# Application charts are a collection of templates that can be packaged into versioned archives
+# to be deployed.
+#
+# Library charts provide useful utilities or functions for the chart developer. They're included as
+# a dependency of application charts to inject those utilities and functions into the rendering
+# pipeline. Library charts do not define any templates and therefore cannot be deployed.
+type: application
+
+# This is the chart version. This version number should be incremented each time you make changes
+# to the chart and its templates, including the app version.
+# Versions are expected to follow Semantic Versioning (https://semver.org/)
+version: 0.1.0

src/istio/common/chart/values.yaml

@@ -0,0 +1 @@
+# SPDX-License-Identifier: AGPL-3.0-or-later OR Commercial

src/istio/common/zarf.yaml

@@ -19,15 +19,22 @@ components:
         namespace: istio-system
         valuesFiles:
           - "../values/values.yaml"
-    manifests:
       - name: uds-global-istio-config
         namespace: istio-system
-        files:
-          - "manifests/envoy-filters.yaml"
-          - "manifests/peer-authentication.yaml"
-          - "manifests/pepr-istio-config.yaml"
+        version: 0.1.0
+        localPath: chart
     actions:
       onDeploy:
+        before:
+          - description: "Fix helm ownership if necessary for clean helm upgrade"
+            mute: true
+            cmd: |
+              ./zarf tools kubectl annotate EnvoyFilter misdirected-request -n istio-system meta.helm.sh/release-name=uds-global-istio-config --overwrite || true
+              ./zarf tools kubectl annotate EnvoyFilter remove-server-header -n istio-system meta.helm.sh/release-name=uds-global-istio-config --overwrite || true
+              ./zarf tools kubectl annotate PeerAuthentication default-istio-system -n istio-system meta.helm.sh/release-name=uds-global-istio-config --overwrite || true
+              ./zarf tools kubectl annotate PeerAuthentication permissive-pepr-webhook -n pepr-system meta.helm.sh/release-name=uds-global-istio-config --overwrite || true
+              ./zarf tools kubectl annotate PeerAuthentication permissive-pepr-webhook-watcher -n pepr-system meta.helm.sh/release-name=uds-global-istio-config --overwrite || true
         after:
           - description: "Ensure istio-injection is enabled for Pepr"
+            mute: true
             cmd: "./zarf tools kubectl label namespace pepr-system istio-injection=enabled --overwrite"

src/pepr/zarf.yaml

@@ -46,23 +46,3 @@ components:
       - name: module
         valuesFiles:
           - values.yaml
-    actions:
-      onDeploy:
-        before:
-          - mute: true
-            description: "Update helm ownership for Pepr resources if necessary during the upgrade"
-            cmd: |
-              ./zarf tools kubectl annotate secret -n pepr-system pepr-uds-core-api-token meta.helm.sh/release-name=module --overwrite || true
-              ./zarf tools kubectl annotate secret -n pepr-system pepr-uds-core-module meta.helm.sh/release-name=module --overwrite || true
-              ./zarf tools kubectl annotate secret -n pepr-system pepr-uds-core-tls meta.helm.sh/release-name=module --overwrite || true
-              ./zarf tools kubectl annotate serviceaccount -n pepr-system pepr-uds-core meta.helm.sh/release-name=module --overwrite || true
-              ./zarf tools kubectl annotate clusterrolebinding pepr-uds-core meta.helm.sh/release-name=module --overwrite || true
-              ./zarf tools kubectl annotate clusterrole pepr-uds-core meta.helm.sh/release-name=module --overwrite || true
-              ./zarf tools kubectl annotate role -n pepr-system pepr-uds-core-store meta.helm.sh/release-name=module --overwrite || true
-              ./zarf tools kubectl annotate rolebinding -n pepr-system pepr-uds-core-store meta.helm.sh/release-name=module --overwrite || true
-              ./zarf tools kubectl annotate service -n pepr-system pepr-uds-core meta.helm.sh/release-name=module --overwrite || true
-              ./zarf tools kubectl annotate service -n pepr-system pepr-uds-core-watcher meta.helm.sh/release-name=module --overwrite || true
-              ./zarf tools kubectl annotate deployment -n pepr-system pepr-uds-core meta.helm.sh/release-name=module --overwrite || true
-              ./zarf tools kubectl annotate deployment -n pepr-system pepr-uds-core-watcher meta.helm.sh/release-name=module --overwrite || true
-              ./zarf tools kubectl annotate mutatingwebhookconfiguration -n pepr-system pepr-uds-core meta.helm.sh/release-name=module --overwrite || true
-              ./zarf tools kubectl annotate validatingwebhookconfiguration -n pepr-system pepr-uds-core meta.helm.sh/release-name=module --overwrite || true