This is a repository for using declarative configs and GitOps to managed shared community Kubeflow GCP infrastructure.
The management cluster is setup using the Kubeflow management blueprint.
If you need to create GCP resources for Kubeflow or gain access to GCP resources you do so by creating PRs against this repository.
-
We use ACM to sync the Cloud Config Connector(CNRM) to GKE cluster that will apply those resouces.
-
ACM has an oppinionated layout to the repository which is rooted at "/prod"
-
See the docs for how this repository should be layed out
-
There should be a namespace for every GCP project that is managed
-
-
Follow these steps to create new project. Note that
${PROJECT}
name must be globally unique across all GCP projects.-
Create subfolder
/prod/namespaces/${PROJECT}
. -
Create
/prod/namespaces/${PROJECT}/namespace.yaml
defining a Kubernetes namespace. Namespace name should be equal to${PROJECT}
name. -
Create
/prod/namespaces/${PROJECT}/project.yaml
containing aProject
resource defining your project. -
Create
/prod/namespaces/${PROJECT}/iam-policy-members.yaml
containing aIAMPolicyMember
resource list with necessary IAM permissions to access the project. EachIAMPolicyMember
should have unique name.You can give
roles/editor
to your GCP user account to view created project.If you want to integrate your project with
kubeflow-ci
, you have to give access to this service account:serviceAccount:kubeflow-testing@kubeflow-ci.iam.gserviceaccount.com
.kubeflow-testing
service account should have these permissions:roles/editor
to modify GCP resources.roles/cloudbuild.builds.editor
to create Cloud Builds.roles/container.admin
to manage Kubernetes clusters.
-
-
Wait for the PR to be approved
-
Once the PR is merged the resources should be created automatically and you can access created GCP project. You can run
kubectl describe
on appropriate resource inkf-community-admin
cluster to check status.
-
Follow the management blueprint
- Do not install CNRM; we will use ConfigSync to install CNRM
-
Follow the ACM installation guide
- Create the service account 'cnrm-system' in project
kf-kcc-admin
- Note It looks like when using ACM to install and manage CNRM you can't use workload identity and need to provide a GCP service account key.
- Create the service account 'cnrm-system' in project
-
Make sure the CNRM service account has roles
roles/owner
and project creator on the community folder