Terraform scripts for Agones

[markmandel]

From my understanding, Terraform can both create a Kubernetes cluster, and install Agones in one go.

Looking at the increase in complexity of the K8s installation (taints for metrics and controller), doing this by hand seems fraught with peril.

I'm thinking we would

1. Create the cluster with the right sizes of nodes with the right taints
2. Sets up appropriate firewall rulels
3. Installs the right version of Agones
4. (Somehow?) works across providers? (I don't know enough about Terraform to know if this is possible).
5. Maybe also replace our development deployment manager scripts with this as well, so we're all using the same codebase.

And maybe more.

What do people think?

[markmandel]

/cc @bbf

[aLekSer]

I will investigate the process of Agones deployment with terraform. I would work on points 1-3 of the above.

[cyriltovena]

4 - need to be separated.
3 - should be supported with helm provider.

[aLekSer]

Also I found that there exist also Kubernetes provider which possibly has more universal nature (cloud agnostic). But I have not tested it.
https://medium.com/devopslinks/how-to-setup-terraform-with-kubernetes-provider-356d0778e771
In order to implement helm config file, this example would be pretty useful:
https://github.com/GoogleCloudPlatform/terraform-google-examples/tree/master/example-gke-k8s-helm

[thisisnotapril]

@Markmande fyil I am reaching out to someone in the k8s community that may be interested in working on this

[aLekSer]

I think we should add here kubectl setup and authorise step.

[aLekSer]

Next step as I see it is to modularize GKE cluster creation config and prepare such modules for other Cloud providers (can also be baremetal one or minikube to start with):
To accomplish next point:

4. (Somehow?) works across providers? (I don't know enough about Terraform to know if this is possible).

[roberthbailey]

@aLekSer - we have some terraform scripts now. Can you document what you think remains to close out this issue?

[aLekSer]

As from the original message we probably going to skip next one, current gcloud utility way looks much smoother:

5. Maybe also replace our development deployment manager scripts with this as well, so we're all using the same codebase.

I would figure out todo list for this topic tomorrow.

[aLekSer]

I think what we should do next is to update Terraform we are using from 0.11.3 to 0.12. Released this May.
Other than that I think we can close this Ticket and create one for updating Terraform in our build system.

Also what would be good to have is CRD deployment (but currently we have Helm for that) this feature was requested in hashicorp/terraform-provider-kubernetes#215 .
Other missing part is AWS support, which I cannot add because the free Tier does not support EKS.

[roberthbailey]

Re Part 5, @markmandel and I were chatting about it and I think it would make sense to replace deployment manager with terraform (although it's perfectly reasonable to do that in a separate issue).

Right now we have declarative config for creating GKE clusters for testing with deployment manager in https://github.com/googleforgames/agones/tree/master/build/gke-test-cluster. And we have declarative config for creating GKE clusters using terraform a couple of places (https://github.com/googleforgames/agones/blob/master/build/cluster.tf and https://github.com/googleforgames/agones/tree/master/build/modules/{aks,gke}).

[roberthbailey]

Upgrading to 0.12 sounds great, since it adds some much needed syntax improvements (I noticed a bunch of places in your original PR where you'd put the 0.12 command in comments and across the board they are better).

[aLekSer]

I need to compare deployment manager and terraform again to give more details of pros and cons of switching to latter. Deployment manager is a bit faster as I remember.

[aLekSer]

We have a check for password length to be over 16 chars as validation step and currently Terraform does not support this.
hashicorp/terraform#15469

[roberthbailey]

@aLekSer - starting with GKE 1.12 we are going to disable basic auth (and client cert auth) so we shouldn't need the password check any longer.

[aLekSer]

ok, I will apply all your comments soon.

[markmandel]

/assign @markmandel to switch out developer tooling to use Terraform.

[aLekSer]

I left a comment in #1375 description.
If we want to support all previous make targets and parameters we need to think how to split deployment of cluster and helm modules

[markmandel]

Do we feel like we should close this ticket, and then create individual tickets for anything that comes up going forward?