Expose maintenance interval as a blueprint setting for node pools in GKE

[annuay-google]

Request

Expose host_maintenance_interval as a user configurable setting in the blueprint

Approach

Expose as an optional terraform variable. If passed, we will set this under node_pool.node_config, using a dynamic block. Possible values are PERIODIC and AS_NEEDED

Test plan

• Update hpc-gke.yaml example with settings.host_maintenance_interval = PERIODIC. Verified this is reflected in Terraform plan and on gcloud cli
• Update hpc-gke.yaml example with settings.host_maintenance_interval = AS_NEEDED. Verified this is reflected in Terraform plan and on gcloud cli
• Keep hpc-gke.yaml as is (no settings.host_maintenance_interval key in yaml). Verified host maintenance interval is unset on gcloud cli

Deployment

Merge to develop. No additional action needed as of now

Submission Checklist

Please take the following actions before submitting this pull request.

• Fork your PR branch from the Toolkit "develop" branch (not main)
• Test all changes with pre-commit in a local branch #
• Confirm that "make tests" passes all tests
• Add or modify unit tests to cover code changes
• Ensure that unit test coverage remains above 80%
• Update all applicable documentation
• Follow Cluster Toolkit Contribution guidelines #

[annuay-google]

I've tested all scenarios (see description for details). Please review @ankitkinra

[arajmane-g]

re: Test Plan

I think, ml-gke would be the right blueprint to use for testing

[annuay-google]

Added maintenance policy to a daily test. Verified that the changes were reflected in the created node pool (ubuntu) @ankitkinra