Skip to content

Commit

Permalink
Automated cherry pick of #1498: fix kustomize manifests for kubeflow #…
Browse files Browse the repository at this point in the history
…1514: Create workflow for Go Cherry pick of #1498 #1514 on release-0.11. #1498: fix kustomize manifests for kubeflow #1514: Create workflow for Go (#1515)

* fix kustomize manifests for kubeflow

* fix standalone and external-db manifests

* remove old namespace file

* remove PV from kubeflow manifest

* fix katib-external-db reference outside of root

* fix katib-with-kubeflow-cert-manager

* Move image tags to katib-config.yaml and remove patches

* use common namespace kustomization

* Make kubeflow-cert use kubeflow as a base

* Remove katib-cert-generator job from kubeflow-cert-generator manifests

* Move pv-patch to patches folder

* Create katib-cert-manager and make kubeflowuse this as base

* Fix release and CI scripts for new layout

* Remove unnecessary cert-generator images from kustomization.yaml

* Remove unnecessary SA, CR and CRB from katib-cert-manager

* Remove commonLabel from katib-with-kubeflow

* Separate cert-generator from webhook kustomization

* Create workflow for Go

* Add GOPATH env

* Move check up

* Add env

* Add go mod download

* Add ls command

* Add path

* Change path for run

* Change GOPATH

* Add kubebuilder

* Download coveralls

* Add node test

* Remove Travis

* Add coveralls step

* Change coveralls use

* Add working dir

* Remove run

* Fix the patch

* Remove patch

Co-authored-by: DavidSpek <vanderspek.david@gmail.com>
  • Loading branch information
andreyvelich and davidspek authored Apr 13, 2021
1 parent 2b01bc0 commit 7d7c34c
Show file tree
Hide file tree
Showing 28 changed files with 250 additions and 286 deletions.
20 changes: 10 additions & 10 deletions .github/workflows/test-charmed-katib.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,19 +10,19 @@ jobs:
runs-on: ubuntu-latest

steps:
- name: Check out code
uses: actions/checkout@v2
- name: Check out code
uses: actions/checkout@v2

- name: Install dependencies
run: |
sudo apt-get install python3-setuptools
sudo pip3 install black flake8
- name: Install dependencies
run: |
sudo apt-get install python3-setuptools
sudo pip3 install black flake8
- name: Check black
run: black --check operators
- name: Check black
run: black --check operators

- name: Check flake8
run: cd operators && flake8
- name: Check flake8
run: cd operators && flake8

build:
name: Test
Expand Down
44 changes: 44 additions & 0 deletions .github/workflows/test-go.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: Go Test

on:
- push
- pull_request

jobs:
test:
name: Test
runs-on: ubuntu-latest
env:
GOPATH: ${{ github.workspace }}/go
defaults:
run:
working-directory: ${{ env.GOPATH }}/src/github.com/kubeflow/katib

steps:
- name: Check out code
uses: actions/checkout@v2
with:
path: ${{ env.GOPATH }}/src/github.com/kubeflow/katib

- name: Setup Go
uses: actions/setup-go@v2
with:
go-version: 1.15.8

- name: Run Go test
run: |
go mod download
curl -L -O "https://github.com/kubernetes-sigs/kubebuilder/releases/download/v1.0.7/kubebuilder_1.0.7_linux_amd64.tar.gz"
tar -zxvf kubebuilder_1.0.7_linux_amd64.tar.gz
sudo mv kubebuilder_1.0.7_linux_amd64 /usr/local/kubebuilder
export PATH=$PATH:/usr/local/kubebuilder/bin
make check
make test
- name: Coveralls report
uses: shogo82148/actions-goveralls@v1
with:
path-to-profile: coverage.out
working-directory: ${{ env.GOPATH }}/src/github.com/kubeflow/katib
24 changes: 24 additions & 0 deletions .github/workflows/test-node.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: Frontend Test

on:
- push
- pull_request

jobs:
test:
name: Test
runs-on: ubuntu-latest

steps:
- name: Check out code
uses: actions/checkout@v2

- name: Setup Node
uses: actions/setup-node@v2
with:
node-version: 12.18.1

- name: Run Node test
run: |
npm install --global prettier@2.2.0
make prettier-check
26 changes: 0 additions & 26 deletions .travis.yml

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- cert-generator.yaml
- rbac.yaml
24 changes: 12 additions & 12 deletions manifests/v1beta1/components/controller/katib-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,13 @@ data:
metrics-collector-sidecar: |-
{
"StdOut": {
"image": "docker.io/kubeflowkatib/file-metrics-collector"
"image": "docker.io/kubeflowkatib/file-metrics-collector:v0.11.0"
},
"File": {
"image": "docker.io/kubeflowkatib/file-metrics-collector"
"image": "docker.io/kubeflowkatib/file-metrics-collector:v0.11.0"
},
"TensorFlowEvent": {
"image": "docker.io/kubeflowkatib/tfevent-metrics-collector",
"image": "docker.io/kubeflowkatib/tfevent-metrics-collector:v0.11.0",
"resources": {
"limits": {
"memory": "1Gi"
Expand All @@ -24,38 +24,38 @@ data:
suggestion: |-
{
"random": {
"image": "docker.io/kubeflowkatib/suggestion-hyperopt"
"image": "docker.io/kubeflowkatib/suggestion-hyperopt:v0.11.0"
},
"tpe": {
"image": "docker.io/kubeflowkatib/suggestion-hyperopt"
"image": "docker.io/kubeflowkatib/suggestion-hyperopt:v0.11.0"
},
"grid": {
"image": "docker.io/kubeflowkatib/suggestion-chocolate"
"image": "docker.io/kubeflowkatib/suggestion-chocolate:v0.11.0"
},
"hyperband": {
"image": "docker.io/kubeflowkatib/suggestion-hyperband"
"image": "docker.io/kubeflowkatib/suggestion-hyperband:v0.11.0"
},
"bayesianoptimization": {
"image": "docker.io/kubeflowkatib/suggestion-skopt"
"image": "docker.io/kubeflowkatib/suggestion-skopt:v0.11.0"
},
"cmaes": {
"image": "docker.io/kubeflowkatib/suggestion-goptuna"
"image": "docker.io/kubeflowkatib/suggestion-goptuna:v0.11.0"
},
"enas": {
"image": "docker.io/kubeflowkatib/suggestion-enas",
"image": "docker.io/kubeflowkatib/suggestion-enas:v0.11.0",
"resources": {
"limits": {
"memory": "200Mi"
}
}
},
"darts": {
"image": "docker.io/kubeflowkatib/suggestion-darts"
"image": "docker.io/kubeflowkatib/suggestion-darts:v0.11.0"
}
}
early-stopping: |-
{
"medianstop": {
"image": "docker.io/kubeflowkatib/earlystopping-medianstop"
"image": "docker.io/kubeflowkatib/earlystopping-medianstop:v0.11.0"
}
}
9 changes: 9 additions & 0 deletions manifests/v1beta1/components/controller/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- controller.yaml
- katib-config.yaml
- rbac.yaml
- service.yaml
- trial-templates.yaml
7 changes: 7 additions & 0 deletions manifests/v1beta1/components/crd/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- experiment.yaml
- suggestion.yaml
- trial.yaml
6 changes: 6 additions & 0 deletions manifests/v1beta1/components/db-manager/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- db-manager.yaml
- service.yaml
9 changes: 9 additions & 0 deletions manifests/v1beta1/components/mysql/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- mysql.yaml
- pv.yaml
- pvc.yaml
- secret.yaml
- service.yaml
6 changes: 6 additions & 0 deletions manifests/v1beta1/components/namespace/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: kubeflow
resources:
# Namespace.
- namespace.yaml
File renamed without changes.
7 changes: 7 additions & 0 deletions manifests/v1beta1/components/ui/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- rbac.yaml
- service.yaml
- ui.yaml
5 changes: 5 additions & 0 deletions manifests/v1beta1/components/webhook/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- webhooks.yaml
60 changes: 60 additions & 0 deletions manifests/v1beta1/installs/katib-cert-manager/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: kubeflow
resources:
# Namespace.
- ../../components/namespace
# Katib controller.
- ../../components/controller/
# Katib CRDs.
- ../../components/crd/
# Katib DB manager.
- ../../components/db-manager/
# Katib DB mysql.
- ../../components/mysql/
# Katib UI.
- ../../components/ui/
# Katib webhooks.
- ../../components/webhook/
# Cert-manager certificate for webhooks
- certificate.yaml
images:
- name: docker.io/kubeflowkatib/katib-controller
newName: docker.io/kubeflowkatib/katib-controller
newTag: v0.11.0
- name: docker.io/kubeflowkatib/katib-db-manager
newName: docker.io/kubeflowkatib/katib-db-manager
newTag: v0.11.0
- name: docker.io/kubeflowkatib/katib-ui
newName: docker.io/kubeflowkatib/katib-ui
newTag: v0.11.0

patchesStrategicMerge:
- patches/katib-cert-injection.yaml

vars:
- fieldref:
fieldPath: metadata.namespace
name: KATIB_NAMESPACE
objref:
apiVersion: v1
kind: Service
name: katib-controller
- fieldref:
fieldPath: metadata.name
name: KATIB_SERVICE_NAME
objref:
apiVersion: v1
kind: Service
name: katib-controller
- name: KATIB_CERT_NAME
objref:
kind: Certificate
group: cert-manager.io
version: v1alpha2
name: katib-webhook-cert
fieldref:
fieldpath: metadata.name

configurations:
- params.yaml
34 changes: 12 additions & 22 deletions manifests/v1beta1/installs/katib-external-db/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,43 +3,33 @@ kind: Kustomization
namespace: kubeflow
resources:
# Namespace.
- ../../components/namespace.yaml
- ../../components/namespace/
# Katib controller.
- ../../components/controller/controller.yaml
- ../../components/controller/service.yaml
- ../../components/controller/rbac.yaml
- ../../components/controller/katib-config.yaml
- ../../components/controller/trial-templates.yaml
- ../../components/controller/
# Katib CRDs.
- ../../components/crd/experiment.yaml
- ../../components/crd/suggestion.yaml
- ../../components/crd/trial.yaml
- ../../components/crd/
# Katib DB manager.
- ../../components/db-manager/db-manager.yaml
- ../../components/db-manager/service.yaml
- ../../components/db-manager/
# Katib UI.
- ../../components/ui/ui.yaml
- ../../components/ui/service.yaml
- ../../components/ui/rbac.yaml
- ../../components/ui/
# Katib Cert Generator
- ../../components/cert-generator/
# Katib webhooks.
- ../../components/webhook/cert-generator.yaml
- ../../components/webhook/rbac.yaml
- ../../components/webhook/webhooks.yaml
- ../../components/webhook/
images:
- name: docker.io/kubeflowkatib/katib-controller
newTag: v0.11.0
newName: docker.io/kubeflowkatib/katib-controller
- name: docker.io/kubeflowkatib/katib-db-manager
newTag: v0.11.0
- name: docker.io/kubeflowkatib/katib-db-manager
newName: docker.io/kubeflowkatib/katib-db-manager
- name: docker.io/kubeflowkatib/katib-ui
newTag: v0.11.0
- name: docker.io/kubeflowkatib/katib-ui
newName: docker.io/kubeflowkatib/katib-ui
- name: docker.io/kubeflowkatib/cert-generator
newTag: v0.11.0
- name: docker.io/kubeflowkatib/cert-generator
newName: docker.io/kubeflowkatib/cert-generator
newTag: v0.11.0
patchesStrategicMerge:
- ../katib-standalone/katib-config-patch.yaml
- db-manager-patch.yaml
# Modify katib-mysql-secrets with parameters for the DB.
secretGenerator:
Expand Down
Loading

0 comments on commit 7d7c34c

Please sign in to comment.