Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set the default GKE cluster type for jupyterhub to GKE Autopilot. #599

Merged
merged 1 commit into from
Apr 26, 2024
Merged

Set the default GKE cluster type for jupyterhub to GKE Autopilot. #599

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions applications/jupyter/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,8 @@ This module deploys the following resources, once per user:
- iap.googleapis.com (required when using authentication with Identity Aware Proxy)

2. A functional GKE cluster.
- To create a new standard or autopilot cluster, follow the instructions under `infrastructure/README.md`
- To create a new standard or autopilot cluster, follow the instructions in [`infrastructure/README.md`](https://github.com/GoogleCloudPlatform/ai-on-gke/blob/main/infrastructure/README.md)
- Alternatively, you can set the `create_cluster` variable to true in `workloads.tfvars` to provision a new GKE cluster. This will default to creating a GKE Autopilot cluster; if you want to provision a standard cluster you must also set `autopilot_cluster` to false.

3. This module is configured to use Identity Aware Proxy (IAP) as default authentication method for JupyterHub. It expects the brand & the OAuth consent configured in your org. You can check the details here: [OAuth consent screen](https://console.cloud.google.com/apis/credentials/consent)

Expand Down Expand Up @@ -116,7 +117,7 @@ gcloud auth application-default login
1. Extract the randomly generated password for JupyterHub login

```
terraform output password
terraform output jupyterhub_password
```

2. Setup port forwarding for the frontend: `kubectl port-forward service/proxy-public -n <namespace> 8081:80 &`, and open `localhost:8081` in a browser.
Expand Down Expand Up @@ -193,4 +194,4 @@ This module uses `<ip>.nip.io` as the domain name with a global static ipv4 addr

## Additional Information

For more information about JupyterHub profiles and the preset profiles visit [here](https://github.com/GoogleCloudPlatform/ai-on-gke/blob/main/applications/jupyter/profiles.md)
For more information about JupyterHub profiles and the preset profiles visit [here](https://github.com/GoogleCloudPlatform/ai-on-gke/blob/main/applications/jupyter/profiles.md)
2 changes: 1 addition & 1 deletion applications/jupyter/variables.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@ variable "private_cluster" {

variable "autopilot_cluster" {
type = bool
default = false
default = true
}

variable "cpu_pools" {
Expand Down
15 changes: 10 additions & 5 deletions applications/jupyter/workloads.tfvars
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,18 @@
# limitations under the License.

##common variables
## Need to pull this variables from tf output from previous platform stage
## Need to pull this variables from tf output from previous infrastructure stage
project_id = "<your project ID>"

## this is required for terraform to connect to GKE master and deploy workloads
create_cluster = false # this flag will create a new standard public gke cluster in default network
cluster_name = "ml-cluster"
cluster_location = "us-central1"
## This is required for terraform to connect to GKE cluster and deploy workloads.
cluster_name = "ml-cluster"
cluster_location = "us-central1"

## If terraform should create a new GKE cluster, fill in this section as well.
## By default, a public autopilot GKE cluster will be created in the default network.
## Set the autopilot_cluster variable to false to create a standard cluster instead.
create_cluster = false
autopilot_cluster = true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe move this up to be with create_cluster so the variables are together since they are related.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all of the flags in this section are related to the create cluster (cluster name, location, type, fleet membership). I've moved the flag up one line, but since I expect that it's more likely to change the cluster name or location than the type, I'd prefer to leave it below those two variables.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restructured a bit more so that the terraform lint check would pass.

cluster_membership_id = "" # required for private cluster, defaults to `cluster_name`

#######################################################
Expand Down
7 changes: 4 additions & 3 deletions infrastructure/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Setup Infra

Platform module (to be renamed to Infra), creates the GKE cluster & other related resources for the AI applications / workloads to be deployed on them.
The infrastructure module creates the GKE cluster and other related resources for the AI applications / workloads to be deployed on them.

1) Update the ```platform.tfvars``` file with the required configuration. Kindly refer to ```tfvars_examples``` for sample configuration.

Expand All @@ -21,12 +21,13 @@ Following service APIs are enabled,

if not already enabled, use the following command:
```
gcloud services enable container.googleapis.com gkehub.googleapis.com
gcloud services enable container.googleapis.com gkehub.googleapis.com \
servicenetworking.googleapis.com cloudresourcemanager.googleapis.com
```
## Network Connectivity

### Private GKE Cluster with internal endpoint
Default config in ```platform.tfvars``` creates a private GKE cluster with internal endpoints & cluster is added to project-scoped Anthos fleet.
The default configuration in ```platform.tfvars``` creates a private GKE cluster with internal endpoints and adds the cluster to a project-scoped Anthos fleet.
For admin access to cluster, Anthos Connect Gateway is used.

### Private GKE Cluster with external endpoint
Expand Down