Version 0.2.0-alpha #25
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Source: [pre-commit-golang](https://github.com/dnephin/pre-commit-golang)
Using a set of precommits for go from [pre-commit-golang](https://github.com/dnephin/pre-commit-golang) and turning on many of the default checks. New dependencies to run will include [gocyclo](https://github.com/fzipp/gocyclo), [goimports](https://pkg.go.dev/golang.org/x/tools/cmd/goimports) and [go-critic](https://github.com/go-critic/go-critic).
Updated error messages throughout to be more descriptive of what is failing and where it's happening.
Update and address strict golang linter pre-commit issues
1. Configure both google and google-beta providers 2. Ensure that google and google-beta providers have version constraints. 3. Ensure that terraform client has minimum version enforced (choose value presently selected by Cloud Foundation Toolkit) 4. Configure each provider with a default zone; the README for generating YAML describes zone as an input variable for all modules. 5. Conform to Terraform standard module structure by creating empty outputs.tf file
Conform to Terraform best practices
Added support for Filestore HighScale. Added support for DDN EXAScaler (Lustre). Renamed hpc-cluster-slurm to hpc-cluster-small and added a hpc-cluster-high-io example, which provisions a filestore Premium for /home, a filestore HighScale for /projects and a lustre file system for /scratch.
To do so, tests were added for PackerWriter and high level functions in reswriter.go. Some minor refactoring was included to aid in testing error cases, more to come in a future PR.
Co-authored-by: heyealex <heyealex@google.com>
Bring test coverage for reswriter above 80%
…nto cboneti/DDN-lustre
Improved file system support.
Add high level function tests for resreader and cover more error handling cases. In addition, add unit tests to pre-commit coverage.
Bring resreader test coverage above 80%
Update test for ExportYamlConfig so that it doesn't log fatal, but rather tests the created files info to validate it was created correctly.
Bring config package unit tests past 80% coverage
Adds more detailed descriptions to each resource README including example where applicable and things to watch out for.
Adding links to the top level resources README to the README's for each resource. Adds a description of common settings for resources. In resource specific README's, warnings and additional examples are added to address reviewer feedback.
Add descriptions to resource READMEs
Adds the high io example description to the examples README including a list of mounted file systems. Moves the Omnia example to a special "experimental" section and adds a warning in the omnia example yaml itself.
Adds a PR temple with a checklist for any new submissions.
Update example README
Add PR template
Added enforce_coverage to make sure engine tests have at least 80% coverage (or none). Added Makefile rule for add-google-license, which can benefit googlers.
It now counts the number of packages with lower coverage and reports that at the end.
All instances have public IPSs by default. Documented current network limitations and requirements.
Fixing a few issues with our documentation.
Changed the path in enforce coverage to /usr/bin/perl. And to be super sure, we call perl in the Makefile, so if perl is in the path, this shoudl work
Workaround for exascaler to work on most networks.
Improved makefile, enforce >= 80% eng converage
- Updted EXAScaler remote version to scripts/2.0.4 - It now works with new VPCs - It no longer creates firewall rules by default - It allows for providing ip address ranges for subnetworks - It might not need private access / public IP addresses (not fully tested). - Added two test files for this config.
Improved EXAScaler
* 1:1 mapping between resource groups and backend configurations * Default backend set with top-level terraform_backend_defaults * If resource group specifies terraform_backend, use that instead Each object has a Type which specifies the type of backend (gcs, s3, etc.) and a Configuration which contains the backend settings as defined by Terraform.
Remote backend
In order to handle certian complex data structures and long or multi-line strings, our current approach of converting to YAML flow style strings before writing to HCL files would not longer suffice. This commit implements writing with the [hcl/hclwrite](https://pkg.go.dev/github.com/hashicorp/hcl/v2/hclwrite) library which gives us more control over how HCL is written and enforces style guidelines actively. As part of this conversion, some of the data in the config needs to be converted to cty.Value objects rather than interface{} for handling dynamic typing. This all happens in the writer currently, however at some point it may be better to rebuild the pipeline to work with cty.Value from the start.
Addresses some reviewer feedback: Pulls most functionality into a single function that will be used by all inheriting struct objects (TFWriter, PackerWriter) rather than having empty functions in both to support the interface. Also removes a step from writeTerraformTfvars that updates passthrough variables, as they should not exist there. Added a test to increase coverage once again past 80%.
Merge remote-tracking branch 'upstream/develop' into hclwrite-resource-writer * Updates tests to bring them back up to 80% in the reswriter * Update writeMain to write terraform block with backend * Handle merge conflicts
Update TFWriter to write files w/hclwrite package
Updating to v.0.2.0
Updating version number for minor release
* Add license * Update to tf-docs generated configs * Fix lint error on DDN-EXAScaler
General cleanup
cboneti
approved these changes
Nov 9, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New Features:
Improvements: