Skip to content

Commit

Permalink
feat: Add ability to read full config from env var
Browse files Browse the repository at this point in the history
Setting environment variable CONFIG_JSON containing the config will override any config file on disk
Updated documentation.
  • Loading branch information
nielm committed May 23, 2024
1 parent e7cd290 commit 8b0e4d5
Show file tree
Hide file tree
Showing 5 changed files with 106 additions and 5 deletions.
72 changes: 71 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,77 @@ Use the
to understand how to configure your Google Cloud Platform project to use Cloud
Run and Eventarc.

## Changes
## Using Environment variables in the configuration

The tutorial above uses a configuration file `config.json` built into the Docker
container for the configuration of the unscanned, clean, quarantined and CVD
updater cloud storage buckets.

Environment variables can be used to vary the deployment in 2 ways:

### Expansion of environment variables

Any environment variables specified using shell-format within the `config.json`
file will be expanded using
[`envsubst`](https://manpages.debian.org/bookworm/gettext-base/envsubst.1.en.html).

### Passing entire configuration as environment variable

An alternative to building the configuration file into the container is to use
environmental variables to contain the configuration of the service, so that
multiple deployments can use the same container, and configuration updates do
not need a container rebuild.

This can be done by setting the environmental variable `CONFIG_JSON` containing
the JSON configuration, which will override any config in the `config.json`
file.

If using the `gcloud run deploy` command line, this environment variable must be
set using the
[`--env-vars-file`](https://cloud.google.com/sdk/gcloud/reference/run/deploy#--env-vars-file)
argument, specifying a YAML file containing the environment variable definitions
(This is because the commas in JSON would break the parsing of `--set-env-vars`)

Take care when embedding JSON in YAML - it is recommended to use the
[Literal Block Scalar style](https://yaml-multiline.info/) using `|`, as this
preserves newlines and quotes

For example, the `CONFIG_JSON` environment variable could be defined in a file
`config-env.yaml` as follows:

```yaml
CONFIG_JSON: |
{
"buckets": [
{
"unscanned": "unscanned-bucket-name",
"clean": "clean-bucket-name",
"quarantined": "quarantined-bucket-name"
}
],
"ClamCvdMirrorBucket": "cvd-mirror-bucket-name"
}
```
An example commandline using this file to specify the environment:
```sh
gcloud beta run deploy "${SERVICE_NAME}" \
--source . \
--region "${REGION}" \
--no-allow-unauthenticated \
--memory 4Gi \
--cpu 1 \
--concurrency 20 \
--min-instances 1 \
--max-instances 5 \
--no-cpu-throttling \
--cpu-boost \
--service-account="${SERVICE_ACCOUNT}" \
--env-vars-file=config-env.yaml
```

## Change history

See [CHANGELOG.md](cloudrun-malware-scanner/CHANGELOG.md)

Expand Down
1 change: 0 additions & 1 deletion cloudrun-malware-scanner/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
FROM node:22
WORKDIR /app
COPY . /app
COPY config.json /app

# Install apt and pip3 requirements.
# GCloud SDK install taken from
Expand Down
12 changes: 10 additions & 2 deletions cloudrun-malware-scanner/bootstrap.sh
Original file line number Diff line number Diff line change
Expand Up @@ -51,10 +51,18 @@ pipx install cvdupdate
service clamav-daemon stop &
service clamav-freshclam stop &

CONFIG_FILE=./config.json

# Check for config from environment
#
if [[ "${CONFIG_JSON}" ]] ; then
Log INFO main "Using config from environment variable CONFIG_JSON"
echo "${CONFIG_JSON}" > ${CONFIG_FILE}
fi

# Check and perform shell-varable substitution on config file, copying it to /etc
#
Log INFO main "Perfoming env var substitution on config file"
CONFIG_FILE=./config.json
if [[ ! -e "${CONFIG_FILE}" ]] ; then
Log ERROR main "${CONFIG_FILE} does not exist"
exit 1
Expand All @@ -77,7 +85,7 @@ fi

# Start the reverse proxy which adds authentication
# to requests to GCS REST API, allowing freshclam to access the GCS
# CVD mirror bucket as if it was an unauthenticated HTPP server
# CVD mirror bucket as if it was an unauthenticated HTTP server
#
export PROXY_PORT=${PROXY_PORT:-8888}
PROXY_SERVER_ADDRESS=127.0.0.1:${PROXY_PORT}
Expand Down
19 changes: 19 additions & 0 deletions cloudrun-malware-scanner/config-env-example.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Example file for specifying the configuration using environment variables.
#
# use with the --env-vars-file argument to gcloud run deploy.
#
# The CONFIG_JSON variable contains the JSON configuration which will
# override any deployed config.json file. Literal Block Scaler style (|)
# is used to preserve quotes and newlines without needing escaping.
#
CONFIG_JSON: |
{
"buckets": [
{
"unscanned": "unscanned-bucket-name",
"clean": "clean-bucket-name",
"quarantined": "quarantined-bucket-name"
}
],
"ClamCvdMirrorBucket": "cvd-mirror-bucket-name"
}
7 changes: 6 additions & 1 deletion cloudrun-malware-scanner/config.json.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,12 @@
"and being rate limited/blacklisted. Its contents are maintained by the updateCvdMirror.sh script",
"",
"Shell environmental variable substitution is supported in this file.",
"At runtime, it will be copied to /etc"
"At runtime, it will be copied to /etc",
"",
"As an alternative to including this file in the container the contents can be passed as an enviroment variable CONFIG_JSON on",
"Cloud Run startup",
"",
"Note: The comments property is optional and can be removed."
],
"buckets": [
{
Expand Down

0 comments on commit 8b0e4d5

Please sign in to comment.