From fce04b503a746a214ca7397d9f72ad0ff96bb791 Mon Sep 17 00:00:00 2001 From: Sulochan Acharya Date: Mon, 5 Aug 2024 17:24:51 +0100 Subject: [PATCH] Kustomize: Using environment variables for configuration Using kustomize with env vars. --- docs/blog/.authors.yml | 7 + ...24-08-02-kustomize-environment-specific.md | 149 ++++++++++++++++++ 2 files changed, 156 insertions(+) create mode 100644 docs/blog/posts/2024-08-02-kustomize-environment-specific.md diff --git a/docs/blog/.authors.yml b/docs/blog/.authors.yml index 791da67..d33b328 100644 --- a/docs/blog/.authors.yml +++ b/docs/blog/.authors.yml @@ -41,3 +41,10 @@ authors: avatar: https://avatars.githubusercontent.com/pratik705 github: pratik705 linkedin: pratik.bandarkar + sulochan: + name: Sulochan M Acharya + description: Cloud Architect @ Rackspace Technology + avatar: https://avatars.githubusercontent.com/sulochan + github: sulochan + twitter: sulo + linkedin: sulochan diff --git a/docs/blog/posts/2024-08-02-kustomize-environment-specific.md b/docs/blog/posts/2024-08-02-kustomize-environment-specific.md new file mode 100644 index 0000000..7568172 --- /dev/null +++ b/docs/blog/posts/2024-08-02-kustomize-environment-specific.md @@ -0,0 +1,149 @@ +--- +date: 2024-08-02 +title: Kustomize - Using environment variables for configuration +authors: + - sulochan +description: > + Kustomize - Using environment variables for configuration +categories: + - Kubernetes + - Kustomize +--- +# Kustomize: Using environment variables for configuration + +[Kustomize](https://kustomize.io) is a widely used tool for Kuberenetes config management that provides a template free way to change your manifests during application deployment. It uses a kustomization.yaml file to define the actions taken during the build process. The file itself can be seen a collection of optional ordered processes: resources, generators, transformers, validators, configMapGenerator, patches and so on. + +We wont dive too deep into these but look at a specific way of using the transformer property to use configuration values (as environment variables) from a file to drive your configuration. + +Lets look at a typical layout of the kustomization files from the genstack repo: + +```shell + +$ ls genestack/base-kustomize/ + +. +.. +argocd/base +bacups/etcd +barbican +... +``` + +and + +```shell + +$ ls genestack/base-kustomize/glance + +.. +aio +base +``` + +We are familiar with this style of providing variance. A base configuration and different varient to go with base configuration folders. + +The problem that we face in this scenario if you would like to deploy different values for your base configuration let’s say depending on the region you are on or host you are on is difficult since it’s not templated. + +One way to handle this is to use configuration file or environment variables to drive the values. This is possible to do since configMapGenerator takes in both envs and literal values. For that to work we need to adopt a slightly different way of doing things: + +Take this deployment for example : + +```shell +apiVersion: gateway.networking.k8s.io/v1 +kind: Gateway +metadata: + name: flex-gateway + namespace: nginx-gateway + annotations: # This is the name of the ClusterIssuer created in the previous step + cert-manager.io/cluster-issuer: flex-gateway-issuer + acme.cert-manager.io/http01-edit-in-place: "true" +spec: + gatewayClassName: $(CLASSNAME) + listeners: + - name: cluster-http + port: 80 + protocol: HTTP + hostname: $(HOST1) + allowedRoutes: + namespaces: + from: All + - name: cluster-tls + port: 443 + protocol: HTTPS + hostname: $(HOST2) + allowedRoutes: + namespaces: + from: All + tls: + mode: Terminate + certificateRefs: + - kind: Secret + name: wildcard-cluster-tls-secret +``` + +Instead of hardcoding the gatewayClassName and hostname for the listners we have defined variables $(CLASSNAME) and $(HOST1) and so on. +We then modify our kustomization to include these variables in the config. + +```shell +$ cat kustomization.yaml + +resources: + - deployment.yaml + +configurations: + - env-var-transformer.yaml + +configMapGenerator: +- name: environment-variables + envs: [environment-properties.env] + behavior: create + +vars: +- name: HOST1 + objref: + kind: ConfigMap + name: environment-variables + apiVersion: v1 + fieldref: + fieldpath: data.HOST1 +- name: HOST2 + objref: + kind: ConfigMap + name: environment-variables + apiVersion: v1 + fieldref: + fieldpath: data.HOST2 +- name: CLASSNAME + objref: + kind: ConfigMap + name: environment-variables + apiVersion: v1 + fieldref: + fieldpath: data.CLASSNAME +``` + +and the corresponding configuration file look like so + +```shell +$ cat env-var-transformer.yaml + +varReference: + - kind: Gateway + path: spec/gatewayClassName + - kind: Gateway + path: spec/listeners/hostname +``` + +Finally, we want these valus to come from our config files, so we define these values in a file (keep it separate from the repo in a separate secure place and pull during deploy only) + +```shell +$ cat environment-properties.env + +HOST1=host1.cluster.local +HOST2=host2.cluster.local +CLASSNAME=nginx +``` + +Now, a ```kubectl kustomize .``` will result in the expected values being replaced with our config values. What this allows us to do is to have a set of base config be applied with different context specific variables. + +Note: Once drawback of this is that if for some reason the value is undefined there is no default value replaced and the build will have empty or $var, so make sure you check your kustomize build to ensure smooth operation.