Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add native support for Google Cloud Storage with GKE #1972

Merged
merged 8 commits into from
Nov 14, 2023

Conversation

nick-stroud
Copy link
Collaborator

You can now configure a cluster and job to use a GCS bucket as a shared file system with GKE.

Note to reviewer:

  • Each commit is self contained and can be reviewed independently
  • First 3 commits are adding loosely related independent features that will be needed downstream
  • Updated storage example job
    • to write to separate files as concurrent file operations was problematic at scale
    • to use both shared file systems

Tested: Deployed storage-gke.yaml blueprint and ran job.

Submission Checklist

Please take the following actions before submitting this pull request.

  • Fork your PR branch from the Toolkit "develop" branch (not main)
  • Test all changes with pre-commit in a local branch #
  • Confirm that "make tests" passes all tests
  • Add or modify unit tests to cover code changes
  • Ensure that unit test coverage remains above 80%
  • Update all applicable documentation
  • Follow Cloud HPC Toolkit Contribution guidelines #

@nick-stroud nick-stroud added the release-key-new-features Added to release notes under the "Key New Features" heading. label Nov 13, 2023
@mr0re1 mr0re1 assigned nick-stroud and unassigned mr0re1 Nov 14, 2023
@nick-stroud
Copy link
Collaborator Author

Will merge pending integration tests.

@nick-stroud nick-stroud merged commit 50ff748 into GoogleCloudPlatform:develop Nov 14, 2023
8 of 35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-key-new-features Added to release notes under the "Key New Features" heading.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants