Support Azure Workload Identity

With an update to github.com/fluxcd/pkg/oci v0.22.0. This includes a pin of `github.com/docker/docker` to `v20.10.x`, to prevent Oras from complaining. Co-authored-by: Hidde Beydals <hidde@hhh.computer> Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
fluxcd · Mar 30, 2023 · 4995cff · 4995cff
1 parent 6b235f0
commit 4995cff
Show file tree

Hide file tree

Showing 7 changed files with 283 additions and 78 deletions.
diff --git a/docs/spec/v1beta2/buckets.md b/docs/spec/v1beta2/buckets.md
@@ -280,6 +280,7 @@ Without a [Secret reference](#secret-reference), authentication using a chain
 with:
 
 - [Environment credentials](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azidentity#EnvironmentCredential)
+- [Workload Identity](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azidentity@v1.3.0-beta.4#WorkloadIdentityCredential)
 - [Managed Identity](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azidentity#ManagedIdentityCredential)
   with the `AZURE_CLIENT_ID`
 - Managed Identity with a system-assigned identity
@@ -436,16 +437,97 @@ data:
   accountKey: <BASE64>
 ```
 
-#### Managed Identity with AAD Pod Identity
+##### Workload Identity
 
-If you are using [aad pod identity](https://azure.github.io/aad-pod-identity/docs), you can create an identity that has access to Azure Storage.
+If you have [Workload Identity mutating webhook](https://azure.github.io/azure-workload-identity/docs/installation/managed-clusters.html)
+installed on your cluster. You would need to create an Azure Identity and
+give it access to Azure Blob Storage.
+
+```shell
+export IDENTITY_NAME="blob-access"
+
+az role assignment create --role "Storage Blob Data Reader" \
+--assignee-object-id "$(az identity show -n $IDENTITY_NAME  -o tsv --query principalId  -g $RESOURCE_GROUP)" \
+--scope "/subscriptions/<SUBSCRIPTION-ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.Storage/storageAccounts/<account-name>/blobServices/default/containers/<container-name>"
+```
+
+Establish federated identity between the identity and the source-controller
+ServiceAccount.
+
+```shell
+export SERVICE_ACCOUNT_ISSUER="$(az aks show --resource-group <RESOURCE_GROUP> --name <CLUSTER-NAME> --query "oidcIssuerProfile.issuerUrl" -otsv)"
+
+az identity federated-credential create \
+  --name "kubernetes-federated-credential" \
+  --identity-name "${IDENTITY_NAME}" \
+  --resource-group "${RESOURCE_GROUP}" \
+  --issuer "${SERVICE_ACCOUNT_ISSUER}" \
+  --subject "system:serviceaccount:flux-system:source-controller"
+```
+
+Add a patch to label and annotate the source-controller Pods and ServiceAccount
+correctly so that it can match an identity binding:
+
+```yaml
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - gotk-components.yaml
+  - gotk-sync.yaml
+patches:
+  - patch: |-
+      apiVersion: v1
+      kind: ServiceAccount
+      metadata:
+        name: source-controller
+        namespace: flux-system
+        annotations:
+          azure.workload.identity/client-id: <AZURE_CLIENT_ID>
+        labels:
+          azure.workload.identity/use: "true"
+  - patch: |-
+      apiVersion: apps/v1
+      kind: Deployment
+      metadata:
+        name: source-controller
+        namespace: flux-system
+        labels:
+          azure.workload.identity/use: "true"
+      spec:
+        template:
+          metadata:
+            labels:
+              azure.workload.identity/use: "true"
+```
+
+If you have set Workload Identity up correctly and labeled the source-controller
+Pod and ServiceAccount, then you don't need to reference a Secret. For more information,
+please see [documentation](https://azure.github.io/azure-workload-identity/docs/quick-start.html).
+
+```yaml
+apiVersion: source.toolkit.fluxcd.io/v1beta2
+kind: Bucket
+metadata:
+  name: azure-bucket
+  namespace: flux-system
+spec:
+  interval: 5m0s
+  provider: azure
+  bucketName: testsas
+  endpoint: https://testfluxsas.blob.core.windows.net
+```
+
+##### Managed Identity with AAD Pod Identity
+
+If you are using [aad pod identity](https://azure.github.io/aad-pod-identity/docs),
+you can create an identity that has access to Azure Storage.
 
 ```sh
 export IDENTITY_NAME="blob-access"
 
-az role assignment create --role "Storage Blob Data Contributor"  \
---assignee-object-id "$(az identity show -n blob-access  -o tsv --query principalId  -g $RESOURCE_GROUP)" \
---scope "/subscriptions/<SUBSCRIPTION-ID>/resourceGroups/aks-somto/providers/Microsoft.Storage/storageAccounts/<account-name>/blobServices/default/containers/<container-name>"
+az role assignment create --role "Storage Blob Data Reader"  \
+--assignee-object-id "$(az identity show -n $IDENTITY_NAME -o tsv --query principalId  -g $RESOURCE_GROUP)" \
+--scope "/subscriptions/<SUBSCRIPTION-ID>/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.Storage/storageAccounts/<account-name>/blobServices/default/containers/<container-name>"
 
 export IDENTITY_CLIENT_ID="$(az identity show -n ${IDENTITY_NAME} -g ${RESOURCE_GROUP} -otsv --query clientId)"
 export IDENTITY_RESOURCE_ID="$(az identity show -n ${IDENTITY_NAME} -otsv --query id)"
@@ -493,7 +575,8 @@ spec:
         aadpodidbinding: ${IDENTITY_NAME}  # match the AzureIdentity name
 ```
 
-If you have set aad-pod-identity up correctly and labeled the source-controller pod, then you don't need to reference a secret.
+If you have set aad-pod-identity up correctly and labeled the source-controller
+pod, then you don't need to reference a secret.
 
 ```yaml
 apiVersion: source.toolkit.fluxcd.io/v1beta2

diff --git a/docs/spec/v1beta2/helmrepositories.md b/docs/spec/v1beta2/helmrepositories.md
@@ -190,9 +190,13 @@ The `aws` provider can be used to authenticate automatically using the EKS worke
 node IAM role or IAM Role for Service Accounts (IRSA), and by extension gain access
 to ECR.
 
+##### EKS Worker Node IAM Role
+
 When the worker node IAM role has access to ECR, source-controller running on it
 will also have access to ECR.
 
+##### IAM Role for Service Accounts (IRSA)
+
 When using IRSA to enable access to ECR, add the following patch to your bootstrap
 repository, in the `flux-system/kustomization.yaml` file:
 
@@ -224,9 +228,56 @@ The `azure` provider can be used to authenticate automatically using kubelet man
 identity or Azure Active Directory pod-managed identity (aad-pod-identity), and 
 by extension gain access to ACR.
 
+##### Kubelet Managed Identity
+
 When the kubelet managed identity has access to ACR, source-controller running on 
 it will also have access to ACR.
 
+##### Workload Identity
+
+When using Workload Identity to enable access to ACR, add the following patch to
+your bootstrap repository, in the `flux-system/kustomization.yaml` file:
+
+```yaml
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - gotk-components.yaml
+  - gotk-sync.yaml
+patches:
+  - patch: |-
+      apiVersion: v1
+      kind: ServiceAccount
+      metadata:
+        name: source-controller
+        namespace: flux-system
+        annotations:
+          azure.workload.identity/client-id: <AZURE_CLIENT_ID>
+        labels:
+          azure.workload.identity/use: "true"
+  - patch: |-
+      apiVersion: apps/v1
+      kind: Deployment
+      metadata:
+        name: source-controller
+        namespace: flux-system
+        labels:
+          azure.workload.identity/use: "true"
+      spec:
+        template:
+          metadata:
+            labels:
+              azure.workload.identity/use: "true"
+```
+
+To use Workload Identity, you have to install the Workload Identity
+mutating webhook and create an identity that has access to ACR. Next, establish
+a federated identity between the source-controller ServiceAccount and the
+identity. Patch the source-controller Pod and ServiceAccount as shown in the patch
+above. Please take a look at this [guide](https://azure.github.io/azure-workload-identity/docs/quick-start.html#6-establish-federated-identity-credential-between-the-identity-and-the-service-account-issuer--subject).
+
+##### AAD Pod Identity
+
 When using aad-pod-identity to enable access to ACR, add the following patch to
 your bootstrap repository, in the `flux-system/kustomization.yaml` file:
 
@@ -261,9 +312,13 @@ if you want to use AKS pod-managed identities add-on that is in preview.
 The `gcp` provider can be used to authenticate automatically using OAuth scopes or
 Workload Identity, and by extension gain access to GCR or Artifact Registry.
 
+#### Access Scopes
+
 When the GKE nodes have the appropriate OAuth scope for accessing GCR and Artifact Registry,
 source-controller running on it will also have access to them.
 
+#### Workload Identity
+
 When using Workload Identity to enable access to GCR or Artifact Registry, add the
 following patch to your bootstrap repository, in the `flux-system/kustomization.yaml`
 file:

diff --git a/docs/spec/v1beta2/ocirepositories.md b/docs/spec/v1beta2/ocirepositories.md
@@ -161,9 +161,55 @@ The `azure` provider can be used to authenticate automatically using kubelet
 managed identity or Azure Active Directory pod-managed identity (aad-pod-identity),
 and by extension gain access to ACR.
 
+##### Kubelet Managed Identity
+
 When the kubelet managed identity has access to ACR, source-controller running
 on it will also have access to ACR.
 
+##### Workload Identity
+
+When using Workload Identity to enable access to ACR, add the following patch to
+your bootstrap repository, in the `flux-system/kustomization.yaml` file:
+
+```yaml
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - gotk-components.yaml
+  - gotk-sync.yaml
+patches:
+  - patch: |-
+      apiVersion: v1
+      kind: ServiceAccount
+      metadata:
+        name: source-controller
+        namespace: flux-system
+        annotations:
+          azure.workload.identity/client-id: <AZURE_CLIENT_ID>
+        labels:
+          azure.workload.identity/use: "true"
+  - patch: |-
+      apiVersion: apps/v1
+      kind: Deployment
+      metadata:
+        name: source-controller
+        namespace: flux-system
+        labels:
+          azure.workload.identity/use: "true"
+      spec:
+        template:
+          metadata:
+            labels:
+              azure.workload.identity/use: "true"
+```
+
+To use Workload Identity, you have to install the Workload Identity
+mutating webhook and create an identity that has access to ACR. Next, establish 
+a federated identity between the source-controller ServiceAccount and the 
+identity. Patch the source-controller Pod and ServiceAccount as shown in the patch
+above. Please take a look at this [guide](https://azure.github.io/azure-workload-identity/docs/quick-start.html#6-establish-federated-identity-credential-between-the-identity-and-the-service-account-issuer--subject).
+
+##### AAD Pod Identity
 When using aad-pod-identity to enable access to ACR, add the following patch to
 your bootstrap repository, in the `flux-system/kustomization.yaml` file:
 

diff --git a/go.mod b/go.mod
@@ -8,11 +8,18 @@ replace github.com/fluxcd/source-controller/api => ./api
 // xref: https://github.com/opencontainers/go-digest/pull/66
 replace github.com/opencontainers/go-digest => github.com/opencontainers/go-digest v1.0.1-0.20220411205349-bde1400a84be
 
+// Required to keep oras.land/oras-go happy, as it will otherwise fail with
+// "assignment mismatch: 3 variables but registry.PingV2Registry returns 2 values"
+//
+// Check again when oras.land/oras-go is updated to >=v2.0.0, which is a
+// dependency of Helm.
+replace github.com/docker/docker => github.com/docker/docker v20.10.23+incompatible
+
 require (
 	cloud.google.com/go/storage v1.30.1
 	github.com/AdaLogics/go-fuzz-headers v0.0.0-20230106234847-43070de90fa1
-	github.com/Azure/azure-sdk-for-go/sdk/azcore v1.4.0
-	github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.2.1
+	github.com/Azure/azure-sdk-for-go/sdk/azcore v1.5.0-beta.1
+	github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.3.0-beta.4
 	github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.0.0
 	github.com/Masterminds/semver/v3 v3.2.0
 	github.com/cyphar/filepath-securejoin v0.2.3
@@ -28,7 +35,7 @@ require (
 	github.com/fluxcd/pkg/helmtestserver v0.12.0
 	github.com/fluxcd/pkg/lockedfile v0.1.0
 	github.com/fluxcd/pkg/masktoken v0.2.0
-	github.com/fluxcd/pkg/oci v0.21.1
+	github.com/fluxcd/pkg/oci v0.22.0
 	github.com/fluxcd/pkg/runtime v0.35.0
 	github.com/fluxcd/pkg/sourceignore v0.3.3
 	github.com/fluxcd/pkg/ssh v0.7.3
@@ -38,7 +45,7 @@ require (
 	github.com/fluxcd/source-controller/api v0.36.1
 	github.com/go-git/go-billy/v5 v5.4.1
 	github.com/go-logr/logr v1.2.3
-	github.com/google/go-containerregistry v0.13.0
+	github.com/google/go-containerregistry v0.14.0
 	github.com/google/go-containerregistry/pkg/authn/k8schain v0.0.0-20230307034325-57f010d26af8
 	github.com/google/uuid v1.3.0
 	github.com/minio/minio-go/v7 v7.0.50
@@ -85,7 +92,7 @@ require (
 	github.com/Azure/go-autorest/autorest/date v0.3.0 // indirect
 	github.com/Azure/go-autorest/logger v0.2.1 // indirect
 	github.com/Azure/go-autorest/tracing v0.6.0 // indirect
-	github.com/AzureAD/microsoft-authentication-library-for-go v0.8.1 // indirect
+	github.com/AzureAD/microsoft-authentication-library-for-go v0.9.0 // indirect
 	github.com/BurntSushi/toml v1.2.1 // indirect
 	github.com/MakeNowJust/heredoc v1.0.0 // indirect
 	github.com/Masterminds/goutils v1.1.1 // indirect
@@ -109,19 +116,19 @@ require (
 	github.com/alibabacloud-go/tea-xml v1.1.2 // indirect
 	github.com/aliyun/credentials-go v1.2.3 // indirect
 	github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d // indirect
-	github.com/aws/aws-sdk-go-v2 v1.17.5 // indirect
-	github.com/aws/aws-sdk-go-v2/config v1.18.15 // indirect
-	github.com/aws/aws-sdk-go-v2/credentials v1.13.15 // indirect
-	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.23 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.29 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.23 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/ini v1.3.30 // indirect
-	github.com/aws/aws-sdk-go-v2/service/ecr v1.18.5 // indirect
+	github.com/aws/aws-sdk-go-v2 v1.17.7 // indirect
+	github.com/aws/aws-sdk-go-v2/config v1.18.19 // indirect
+	github.com/aws/aws-sdk-go-v2/credentials v1.13.18 // indirect
+	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.13.1 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.31 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.25 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/ini v1.3.32 // indirect
+	github.com/aws/aws-sdk-go-v2/service/ecr v1.18.7 // indirect
 	github.com/aws/aws-sdk-go-v2/service/ecrpublic v1.13.17 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.23 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sso v1.12.4 // indirect
-	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.4 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sts v1.18.5 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.25 // indirect
+	github.com/aws/aws-sdk-go-v2/service/sso v1.12.6 // indirect
+	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.6 // indirect
+	github.com/aws/aws-sdk-go-v2/service/sts v1.18.7 // indirect
 	github.com/aws/smithy-go v1.13.5 // indirect
 	github.com/awslabs/amazon-ecr-credential-helper/ecr-login v0.0.0-20221004211355-a250ad2ca1e3 // indirect
 	github.com/benbjohnson/clock v1.1.0 // indirect
@@ -144,7 +151,7 @@ require (
 	github.com/common-nighthawk/go-figure v0.0.0-20210622060536-734e95fb86be // indirect
 	github.com/containerd/containerd v1.6.18 // indirect
 	github.com/containerd/continuity v0.3.0 // indirect
-	github.com/containerd/stargz-snapshotter/estargz v0.12.1 // indirect
+	github.com/containerd/stargz-snapshotter/estargz v0.14.3 // indirect
 	github.com/coreos/go-oidc/v3 v3.5.0 // indirect
 	github.com/coreos/go-semver v0.3.0 // indirect
 	github.com/coreos/go-systemd/v22 v22.3.2 // indirect
@@ -153,7 +160,7 @@ require (
 	github.com/davecgh/go-spew v1.1.1 // indirect
 	github.com/dimchansky/utfbom v1.1.1 // indirect
 	github.com/docker/distribution v2.8.1+incompatible // indirect
-	github.com/docker/docker v20.10.21+incompatible // indirect
+	github.com/docker/docker v23.0.1+incompatible // indirect
 	github.com/docker/docker-credential-helpers v0.7.0 // indirect
 	github.com/docker/go-connections v0.4.0 // indirect
 	github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c // indirect
@@ -198,7 +205,7 @@ require (
 	github.com/gofrs/uuid v4.2.0+incompatible // indirect
 	github.com/gogo/protobuf v1.3.2 // indirect
 	github.com/golang-jwt/jwt v3.2.2+incompatible // indirect
-	github.com/golang-jwt/jwt/v4 v4.4.2 // indirect
+	github.com/golang-jwt/jwt/v4 v4.4.3 // indirect
 	github.com/golang/glog v1.0.0 // indirect
 	github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
 	github.com/golang/mock v1.6.0 // indirect
@@ -233,7 +240,7 @@ require (
 	github.com/huandu/xstrings v1.4.0 // indirect
 	github.com/imdario/mergo v0.3.13 // indirect
 	github.com/in-toto/in-toto-golang v0.3.4-0.20220709202702-fa494aaa0add // indirect
-	github.com/inconshreveable/mousetrap v1.0.1 // indirect
+	github.com/inconshreveable/mousetrap v1.1.0 // indirect
 	github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
 	github.com/jedisct1/go-minisign v0.0.0-20211028175153-1c139d1cc84b // indirect
 	github.com/jhump/protoreflect v1.14.0 // indirect