Add AKS, GKE and Helm terraform modules #756

aLekSer · 2019-05-02T13:59:19Z

Add configuration for deploying cluster and installing Agones on AKS.

Next step would be to move common modules into /build/modules/ from /build and change terraform.mk make targets to support this. As you can see there is a small difference between helm.tf for GKE and AKS. So need to create common output to combine them into one helm.tf file.

Tested that this configuration works on Azure free account. AWS does not support EKS on free tier.

For #657 .

agones-bot · 2019-05-02T14:15:16Z

Build Succeeded 👏

Build Id: e059442b-f8db-49dc-9d0a-85375bc3f713

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:0.10.0-d84ff9a
image: gcr.io/agones-images/agones-sdk:0.10.0-d84ff9a
image: gcr.io/agones-images/agones-ping:0.10.0-d84ff9a
Linux C++ SDK (build): agonessdk-0.10.0-d84ff9a-linux-arch_64.tar.gz
SDK Server: agonessdk-server-0.10.0-d84ff9a.zip

A preview of the website (the last 30 builds are retained):

https://d84ff9a-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/GoogleCloudPlatform/agones.git pull/756/head:pr_756 && git checkout pr_756
helm install install/helm/agones --namespace agones-system --name agones --set agones.image.tag=0.10.0-d84ff9a

roberthbailey · 2019-05-02T19:56:33Z

build/modules/aks/aks.tf

@@ -0,0 +1,115 @@
+# Copyright 2019 Google LLC All Rights Reserved.


Have you thought about breaking this into multiple files to separate inputs / resources / outputs as per https://www.terraform-best-practices.com/code-structure#getting-started-with-structuring-of-terraform-configurations?

aLekSer · 2019-05-03T06:46:35Z

Thanks for the update. I will break down tf config files and update this PR. Also I want to create prod and dev configurations, dev for current changes, and prod for stable releases.

aLekSer · 2019-05-03T14:21:22Z

Add more natural variables and outputs which make me able to have one helm terraform configuration used by both GKE and AKS.
Also note that I added gke-local example which is using local module files. https://github.com/GoogleCloudPlatform/agones/blob/9259d65a9accb6a63a775e0a3b099585929a4eba/examples/terraform-submodules/gke-local/main.tf

agones-bot · 2019-05-03T14:32:05Z

Build Succeeded 👏

Build Id: 96342f3f-2da8-45ee-82a7-f3c13d7cf1d3

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:0.10.0-9259d65
image: gcr.io/agones-images/agones-sdk:0.10.0-9259d65
image: gcr.io/agones-images/agones-ping:0.10.0-9259d65
Linux C++ SDK (build): agonessdk-0.10.0-9259d65-linux-arch_64.tar.gz
SDK Server: agonessdk-server-0.10.0-9259d65.zip

A preview of the website (the last 30 builds are retained):

https://9259d65-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/GoogleCloudPlatform/agones.git pull/756/head:pr_756 && git checkout pr_756
helm install install/helm/agones --namespace agones-system --name agones --set agones.image.tag=0.10.0-9259d65

agones-bot · 2019-05-03T15:10:18Z

Build Succeeded 👏

Build Id: 6a485ddd-6005-43bb-b9d9-536801f6e562

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:0.10.0-7c875c1
image: gcr.io/agones-images/agones-sdk:0.10.0-7c875c1
image: gcr.io/agones-images/agones-ping:0.10.0-7c875c1
Linux C++ SDK (build): agonessdk-0.10.0-7c875c1-linux-arch_64.tar.gz
SDK Server: agonessdk-server-0.10.0-7c875c1.zip

A preview of the website (the last 30 builds are retained):

https://7c875c1-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/GoogleCloudPlatform/agones.git pull/756/head:pr_756 && git checkout pr_756
helm install install/helm/agones --namespace agones-system --name agones --set agones.image.tag=0.10.0-7c875c1

agones-bot · 2019-05-03T15:38:08Z

Build Succeeded 👏

Build Id: a785dc23-5214-4af4-9738-697f23d63ad9

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:0.10.0-adfd47f
image: gcr.io/agones-images/agones-sdk:0.10.0-adfd47f
image: gcr.io/agones-images/agones-ping:0.10.0-adfd47f
Linux C++ SDK (build): agonessdk-0.10.0-adfd47f-linux-arch_64.tar.gz
SDK Server: agonessdk-server-0.10.0-adfd47f.zip

A preview of the website (the last 30 builds are retained):

https://adfd47f-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/GoogleCloudPlatform/agones.git pull/756/head:pr_756 && git checkout pr_756
helm install install/helm/agones --namespace agones-system --name agones --set agones.image.tag=0.10.0-adfd47f

agones-bot · 2019-05-03T15:56:37Z

Build Succeeded 👏

Build Id: ab16cc8a-8a86-4cfc-be9c-d57c4aa1ef43

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:0.10.0-b39e682
image: gcr.io/agones-images/agones-sdk:0.10.0-b39e682
image: gcr.io/agones-images/agones-ping:0.10.0-b39e682
Linux C++ SDK (build): agonessdk-0.10.0-b39e682-linux-arch_64.tar.gz
SDK Server: agonessdk-server-0.10.0-b39e682.zip

A preview of the website (the last 30 builds are retained):

https://b39e682-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/GoogleCloudPlatform/agones.git pull/756/head:pr_756 && git checkout pr_756
helm install install/helm/agones --namespace agones-system --name agones --set agones.image.tag=0.10.0-b39e682

agones-bot · 2019-05-03T16:39:54Z

Build Succeeded 👏

Build Id: b355cd8d-1ee2-4abe-83df-c06deb43b176

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:0.10.0-d05469c
image: gcr.io/agones-images/agones-sdk:0.10.0-d05469c
image: gcr.io/agones-images/agones-ping:0.10.0-d05469c
Linux C++ SDK (build): agonessdk-0.10.0-d05469c-linux-arch_64.tar.gz
SDK Server: agonessdk-server-0.10.0-d05469c.zip

A preview of the website (the last 30 builds are retained):

https://d05469c-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/GoogleCloudPlatform/agones.git pull/756/head:pr_756 && git checkout pr_756
helm install install/helm/agones --namespace agones-system --name agones --set agones.image.tag=0.10.0-d05469c

roberthbailey

Sorry for the high latency on reviews. I had a few more comments, but overal lgtm.

roberthbailey · 2019-05-08T20:52:22Z

site/content/en/docs/Installation/terraform.md

@@ -12,10 +12,10 @@ description: >
 - Terraform v0.11.13
 - [Helm](https://docs.helm.sh/helm/) package manager 2.10.0+
 - Access to Google Cloud Kubernetes Engine
- `gcloud` utility installed
+- `gcloud` or `az` utility installed


The above line says "Google Cloud Kubernetes Engine" so I think this could either be written as:

"Access to the the Kubernetes hosting provider you are using (e.g. gcloud or az utility installed)"

or

"Access to Google Cloud Kubernetes Engine (gcloud utilitity installed) or Azure Kubernetes Service (az utility installed)".

I reworded this lines and split specific requirements for subsequent sections.

roberthbailey · 2019-05-08T20:53:54Z

site/content/en/docs/Installation/terraform.md

@@ -12,10 +12,10 @@ description: >
 - Terraform v0.11.13
 - [Helm](https://docs.helm.sh/helm/) package manager 2.10.0+
 - Access to Google Cloud Kubernetes Engine


Regarding the following comment, I'm not sure what this line is supposed to mean. Does this mean that you have a project and the api enabled?

Yes, that would be much better description, I will split this into two parts, one for GKE and one for AKS

roberthbailey · 2019-05-08T21:03:17Z

build/modules/gke/cluster.tf

+    password = "${var.password}"
+  }
+  enable_legacy_abac = "${lookup(var.cluster, "legacyAbac")}"
+  node_pool = [


After reading the docs about how terraform handles updates to node pools, I think it would be better to make each node pool a separate resource.

From https://www.terraform.io/docs/providers/google/r/container_cluster.html#node_pool:

Warning: node pools defined inside a cluster can't be changed (or added/removed) after cluster creation without deleting and recreating the entire cluster. Unless you absolutely need the ability to say "these are the only node pools associated with this cluster", use the google_container_node_pool resource instead of this property.

That would make cluster creation a bit longer, about 10 minutes compared to 3 minutes now.
But regarding this warning it seems to be inevitable.

Note that after this change I faced issue with cluster starts upgrading after I attach 2 additional node_pools to it.
hashicorp/terraform-provider-google#3385

Fixed the issue as proposed in link I posted above with:

remove_default_node_pool = true

And now we have 3 node pools and helm would install Agones right after creation of all node pools.

I agree that cluster creation is much slower. It's really annoying that the terraform implementation for GKE requires jumping through these hoops. :/

roberthbailey · 2019-05-08T21:07:30Z

examples/terraform-submodules/gke-local/main.tf

+  }
+}
+
+/*


Delete these commented out lines?

agones-bot · 2019-05-13T09:45:52Z

Build Succeeded 👏

Build Id: c6d188a5-9f31-46ad-ba10-fe121055dbb2

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:0.10.0-6f30051
image: gcr.io/agones-images/agones-sdk:0.10.0-6f30051
image: gcr.io/agones-images/agones-ping:0.10.0-6f30051
Linux C++ SDK (build): agonessdk-0.10.0-6f30051-linux-arch_64.tar.gz
SDK Server: agonessdk-server-0.10.0-6f30051.zip

A preview of the website (the last 30 builds are retained):

https://6f30051-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/GoogleCloudPlatform/agones.git pull/756/head:pr_756 && git checkout pr_756
helm install install/helm/agones --namespace agones-system --name agones --set agones.image.tag=0.10.0-6f30051

agones-bot · 2019-05-13T10:03:48Z

Build Succeeded 👏

Build Id: d1d2f628-a5a8-407c-b311-2608faa54ebf

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:0.10.0-fc6f1ee
image: gcr.io/agones-images/agones-sdk:0.10.0-fc6f1ee
image: gcr.io/agones-images/agones-ping:0.10.0-fc6f1ee
Linux C++ SDK (build): agonessdk-0.10.0-fc6f1ee-linux-arch_64.tar.gz
SDK Server: agonessdk-server-0.10.0-fc6f1ee.zip

A preview of the website (the last 30 builds are retained):

https://fc6f1ee-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/GoogleCloudPlatform/agones.git pull/756/head:pr_756 && git checkout pr_756
helm install install/helm/agones --namespace agones-system --name agones --set agones.image.tag=0.10.0-fc6f1ee

agones-bot · 2019-05-13T10:34:33Z

Build Succeeded 👏

Build Id: 659a23b9-d648-4ddf-b9d0-9ee7cf73dc81

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:0.10.0-753e133
image: gcr.io/agones-images/agones-sdk:0.10.0-753e133
image: gcr.io/agones-images/agones-ping:0.10.0-753e133
Linux C++ SDK (build): agonessdk-0.10.0-753e133-linux-arch_64.tar.gz
SDK Server: agonessdk-server-0.10.0-753e133.zip

A preview of the website (the last 30 builds are retained):

https://753e133-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/GoogleCloudPlatform/agones.git pull/756/head:pr_756 && git checkout pr_756
helm install install/helm/agones --namespace agones-system --name agones --set agones.image.tag=0.10.0-753e133

aLekSer · 2019-05-14T10:04:50Z

@roberthbailey, thanks for the review. Applied all your comments, in a separate PR I am going to remove old not module version of tf config which is left in terraform.mk.

roberthbailey · 2019-05-14T17:08:52Z

Thanks!

@markmandel / @jkowalski - can one of you take a quick look for approval?

Add configuration for deploying cluster and installing Agones on AKS.

agones-bot · 2019-05-15T10:57:38Z

Build Succeeded 👏

Build Id: f7fcaf20-0ffd-4aa4-b076-6aff2d01cfe0

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:0.10.0-6291452
image: gcr.io/agones-images/agones-sdk:0.10.0-6291452
image: gcr.io/agones-images/agones-ping:0.10.0-6291452
Linux C++ SDK (build): agonessdk-0.10.0-6291452-linux-arch_64.tar.gz
SDK Server: agonessdk-server-0.10.0-6291452.zip

A preview of the website (the last 30 builds are retained):

https://6291452-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/GoogleCloudPlatform/agones.git pull/756/head:pr_756 && git checkout pr_756
helm install install/helm/agones --namespace agones-system --name agones --set agones.image.tag=0.10.0-6291452

markmandel

Did a quick look through - LGTM!

agones-bot · 2019-05-16T05:12:58Z

Build Succeeded 👏

Build Id: f6d74c56-a29d-4ca0-acd3-994efddf2444

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:0.10.0-b5cebd9
image: gcr.io/agones-images/agones-sdk:0.10.0-b5cebd9
image: gcr.io/agones-images/agones-ping:0.10.0-b5cebd9
Linux C++ SDK (build): agonessdk-0.10.0-b5cebd9-linux-arch_64.tar.gz
SDK Server: agonessdk-server-0.10.0-b5cebd9.zip

A preview of the website (the last 30 builds are retained):

https://b5cebd9-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/GoogleCloudPlatform/agones.git pull/756/head:pr_756 && git checkout pr_756
helm install install/helm/agones --namespace agones-system --name agones --set agones.image.tag=0.10.0-b5cebd9

roberthbailey reviewed May 2, 2019

View reviewed changes

aLekSer force-pushed the feature/terraform-modules branch from d84ff9a to 9259d65 Compare May 3, 2019 14:16

aLekSer changed the title ~~Add AKS terraform module~~ Add AKS, GKE and Helm terraform modules May 3, 2019

aLekSer force-pushed the feature/terraform-modules branch from 9259d65 to 7c875c1 Compare May 3, 2019 14:55

aLekSer force-pushed the feature/terraform-modules branch 2 times, most recently from adfd47f to b39e682 Compare May 3, 2019 15:35

aLekSer force-pushed the feature/terraform-modules branch from b39e682 to d05469c Compare May 3, 2019 16:25

markmandel added the feature-freeze-do-not-merge Only eligible to be merged once we are out of feature freeze (next full release) label May 8, 2019

roberthbailey reviewed May 8, 2019

View reviewed changes

aLekSer force-pushed the feature/terraform-modules branch from d05469c to 6f30051 Compare May 13, 2019 09:30

aLekSer force-pushed the feature/terraform-modules branch from 6f30051 to fc6f1ee Compare May 13, 2019 09:48

aLekSer force-pushed the feature/terraform-modules branch from fc6f1ee to 753e133 Compare May 13, 2019 10:19

Add AKS, GKE and Helm terraform modules

6291452

Add configuration for deploying cluster and installing Agones on AKS.

aLekSer force-pushed the feature/terraform-modules branch from 753e133 to 6291452 Compare May 15, 2019 10:42

markmandel removed the feature-freeze-do-not-merge Only eligible to be merged once we are out of feature freeze (next full release) label May 16, 2019

markmandel added area/operations Installation, updating, metrics etc kind/feature New features for Agones and removed feature-freeze-do-not-merge Only eligible to be merged once we are out of feature freeze (next full release) labels May 16, 2019

markmandel added this to the 0.11.0 milestone May 16, 2019

markmandel approved these changes May 16, 2019

View reviewed changes

Merge branch 'master' into feature/terraform-modules

b5cebd9

markmandel merged commit fe5450c into googleforgames:master May 16, 2019

aLekSer deleted the feature/terraform-modules branch May 16, 2019 10:00

roberthbailey mentioned this pull request Jul 16, 2019

Approver access for @roberthbailey #914

Closed

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add AKS, GKE and Helm terraform modules #756

Add AKS, GKE and Helm terraform modules #756

aLekSer commented May 2, 2019 •

edited

Loading

agones-bot commented May 2, 2019

roberthbailey May 2, 2019

aLekSer commented May 3, 2019 •

edited

Loading

aLekSer commented May 3, 2019

agones-bot commented May 3, 2019

agones-bot commented May 3, 2019

agones-bot commented May 3, 2019

agones-bot commented May 3, 2019

agones-bot commented May 3, 2019

roberthbailey left a comment

roberthbailey May 8, 2019

aLekSer May 14, 2019

roberthbailey May 8, 2019

aLekSer May 13, 2019

roberthbailey May 8, 2019

aLekSer May 13, 2019 •

edited

Loading

aLekSer May 13, 2019

aLekSer May 13, 2019 •

edited

Loading

roberthbailey May 14, 2019

roberthbailey May 8, 2019

agones-bot commented May 13, 2019

agones-bot commented May 13, 2019

agones-bot commented May 13, 2019

aLekSer commented May 14, 2019

roberthbailey commented May 14, 2019

agones-bot commented May 15, 2019

markmandel left a comment

agones-bot commented May 16, 2019

		@@ -0,0 +1,115 @@
		# Copyright 2019 Google LLC All Rights Reserved.

+                }
+              }
+              /*

Add AKS, GKE and Helm terraform modules #756

Add AKS, GKE and Helm terraform modules #756

Conversation

aLekSer commented May 2, 2019 • edited Loading

agones-bot commented May 2, 2019

Choose a reason for hiding this comment

aLekSer commented May 3, 2019 • edited Loading

aLekSer commented May 3, 2019

agones-bot commented May 3, 2019

agones-bot commented May 3, 2019

agones-bot commented May 3, 2019

agones-bot commented May 3, 2019

agones-bot commented May 3, 2019

roberthbailey left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aLekSer May 13, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aLekSer May 13, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

agones-bot commented May 13, 2019

agones-bot commented May 13, 2019

agones-bot commented May 13, 2019

aLekSer commented May 14, 2019

roberthbailey commented May 14, 2019

agones-bot commented May 15, 2019

markmandel left a comment

Choose a reason for hiding this comment

agones-bot commented May 16, 2019

aLekSer commented May 2, 2019 •

edited

Loading

aLekSer commented May 3, 2019 •

edited

Loading

aLekSer May 13, 2019 •

edited

Loading

aLekSer May 13, 2019 •

edited

Loading