Skip to content

Commit

Permalink
Feast Helm charts and build script (#289)
Browse files Browse the repository at this point in the history
* Update Dockerfiles for core and serving for Feast 0.3

* Add example job store options for Redis in application.yaml

* Add script to publish feast helm chart

* Update default jvm options in CMD instruction in Feast Core Dockerfile

* Update installation path of grpc-health-probe in Feast serving Dockerfile

* Update feast charts for installing Feast with Helm
- Refer to README.md for more details how feast chart is structured and how to use it
- Feast Core and Feast Serving are subcharts that the parent Feast chart uses to install Feast. These subcharts optionally contains other dependencies such as Kafka, Postgresql or Redis. All these components can be enabled/disabled using the enabled flag in the Helm values.

* Update configmap.yaml for feast core
Override bootstrapServer value only if kafka is enabled and kafka.external is disabled

* Update configmap template for feast-serving
Support overriding Redis store host with LoadBalancer IP

* Set lower default delay time for readiness probes
- So pod scaling and startup is faster
- In most cases, Core and Feast should not need more than 15 seconds to start up

* Update Dockerfile for Feast Core
Include exploded/unpacked jar in the production image
To be used when staging files when submitting Dataflow jobs

* Allow users to specify nodePort in Helm values
  • Loading branch information
davidheryanto authored and feast-ci-bot committed Nov 9, 2019
1 parent 004e0e6 commit d8cf0ad
Show file tree
Hide file tree
Showing 52 changed files with 1,197 additions and 1,104 deletions.
57 changes: 57 additions & 0 deletions .prow/scripts/sync-helm-charts.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
#!/usr/bin/env bash

# Script to sync local charts to remote helm repository in Google Cloud Storage
# Copied from: https://github.com/helm/charts/blob/master/test/repo-sync.sh

set -o errexit
set -o nounset
set -o pipefail

log_error() {
printf '\e[31mERROR: %s\n\e[39m' "$1" >&2
}

# Assume working directory is Feast repository root folder
repo_dir=infra/charts
bucket=gs://feast-charts
repo_url=https://feast-charts.storage.googleapis.com/
sync_dir=/tmp/syncdir
index_dir=/tmp/indexdir

rm -rf $sync_dir $index_dir

echo "Syncing repo '$repo_dir'..."

mkdir -p "$sync_dir"
if ! gsutil cp "$bucket/index.yaml" "$index_dir/index.yaml"; then
log_error "Exiting because unable to copy index locally. Not safe to proceed."
exit 1
fi

exit_code=0

for dir in "$repo_dir"/*; do
if helm dependency build "$dir"; then
helm package --destination "$sync_dir" "$dir"
else
log_error "Problem building dependencies. Skipping packaging of '$dir'."
exit_code=1
fi
done

if helm repo index --url "$repo_url" --merge "$index_dir/index.yaml" "$sync_dir"; then
# Move updated index.yaml to sync folder so we don't push the old one again
mv -f "$sync_dir/index.yaml" "$index_dir/index.yaml"

gsutil -m rsync "$sync_dir" "$bucket"

# Make sure index.yaml is synced last
gsutil cp "$index_dir/index.yaml" "$bucket"
else
log_error "Exiting because unable to update index. Not safe to push update."
exit 1
fi

ls -l "$sync_dir"

exit "$exit_code"
9 changes: 0 additions & 9 deletions Dockerfile

This file was deleted.

119 changes: 0 additions & 119 deletions infra/charts/README.md

This file was deleted.

5 changes: 0 additions & 5 deletions infra/charts/feast-core/Chart.yaml

This file was deleted.

Binary file removed infra/charts/feast-core/charts/postgresql-3.17.0.tgz
Binary file not shown.
6 changes: 0 additions & 6 deletions infra/charts/feast-core/requirements.lock

This file was deleted.

5 changes: 0 additions & 5 deletions infra/charts/feast-core/requirements.yaml

This file was deleted.

48 changes: 0 additions & 48 deletions infra/charts/feast-core/templates/core-configmap.yaml

This file was deleted.

110 changes: 0 additions & 110 deletions infra/charts/feast-core/templates/core-deploy.yaml

This file was deleted.

Loading

0 comments on commit d8cf0ad

Please sign in to comment.