Supported the atmos.d, allowing automatic inclusion of configuration files from the atmos.d directory #808
Conversation
haitham911
requested review from
johncblandii,
RoseSecurity,
aknysh and
osterman
📝 Walkthrough📝 WalkthroughThe changes in this pull request introduce a comprehensive update to the configuration management of the Atmos CLI. A new configuration file, Changes
Assessment against linked issues
Possibly related PRs
Suggested reviewers
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 9
🧹 Outside diff range and nitpick comments (7)
examples/demo-atmos.d/atmos.yaml (3)
4-11: Document security-sensitive settingsCritical settings like
apply_auto_approveandauto_generate_backend_filewould benefit from inline documentation explaining their security implications.Consider adding comments:
components: terraform: base_path: "components/terraform" + # Disable auto-approve for safety in production environments apply_auto_approve: false deploy_run_init: true init_run_reconfigure: true + # Disable auto-generation of backend files to prevent unauthorized state access auto_generate_backend_file: false
12-19: Consider expanding name pattern flexibilityThe current name pattern only supports {stage}. Consider supporting additional variables for better organization.
Consider expanding:
- name_pattern: "{stage}" + name_pattern: "{org}-{stage}-{component}"
20-29: Clean up formatting and enhance documentationThe vendor configuration contains helpful examples but needs formatting cleanup.
Apply these changes:
-vendor: +vendor: # Single file - base_path: "./vendor.yaml" + base_path: "./vendor.yaml" # Default configuration # Directory with multiple files #base_path: "./vendor" # Absolute path #base_path: "vendor.d/vendor1.yaml"🧰 Tools
🪛 yamllint (1.29.0-1)
[error] 20-20: trailing spaces
(trailing-spaces)
[error] 23-23: trailing spaces
(trailing-spaces)
[error] 26-26: trailing spaces
(trailing-spaces)
examples/demo-atmos.d/custom-import/atmos.yaml (2)
26-35: Consider moving configuration examples to documentationWhile the commented examples are helpful, they might be better suited in the documentation, keeping the configuration file cleaner.
- # Single file base_path: "./vendor.yaml" - - # Directory with multiple files - #base_path: "./vendor" - - # Absolute path - #base_path: "vendor.d/vendor1.yaml"🧰 Tools
🪛 yamllint (1.29.0-1)
[error] 26-26: trailing spaces
(trailing-spaces)
[error] 29-29: trailing spaces
(trailing-spaces)
[error] 32-32: trailing spaces
(trailing-spaces)
36-38: Consider environment-specific log configurationsThe current logging setup is good for development, but you might want to add environment-specific configurations for production use.
Example addition:
logs: file: "${ATMOS_LOG_FILE:-/dev/stderr}" level: "${ATMOS_LOG_LEVEL:-Info}"pkg/schema/schema.go (1)
29-29: Add field documentation.Consider adding a documentation comment to describe the purpose and usage of the
Importfield, following Go's documentation conventions.Add this documentation above the field:
+ // Import specifies a list of paths from which to import additional configurations. + // Supports local files, directories (using glob patterns), and remote URLs. Import []string `yaml:"import" json:"import" mapstructure:"import"`pkg/config/config.go (1)
231-232: Combine the return statement for clarityThe return statement is unnecessarily split across two lines.
Consider writing it on a single line:
-return cliConfig, - err +return cliConfig, err
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (10)
examples/demo-atmos.d/atmos.d/configs.d/config1.yaml(1 hunks)examples/demo-atmos.d/atmos.d/configs.d/sub/config2.yaml(1 hunks)examples/demo-atmos.d/atmos.yaml(1 hunks)examples/demo-atmos.d/custom-import/atmos.yaml(1 hunks)examples/demo-atmos.d/custom-import/configs.d/config1.yaml(1 hunks)examples/demo-atmos.d/custom-import/configs.d/sub/config2.yaml(1 hunks)examples/demo-atmos.d/custom-import/extra-config.yaml(1 hunks)pkg/config/config.go(3 hunks)pkg/schema/schema.go(1 hunks)pkg/utils/glob_utils.go(2 hunks)
✅ Files skipped from review due to trivial changes (5)
- examples/demo-atmos.d/atmos.d/configs.d/config1.yaml
- examples/demo-atmos.d/atmos.d/configs.d/sub/config2.yaml
- examples/demo-atmos.d/custom-import/configs.d/config1.yaml
- examples/demo-atmos.d/custom-import/configs.d/sub/config2.yaml
- examples/demo-atmos.d/custom-import/extra-config.yaml
🧰 Additional context used
🪛 yamllint (1.29.0-1)
examples/demo-atmos.d/atmos.yaml
[error] 20-20: trailing spaces
(trailing-spaces)
[error] 23-23: trailing spaces
(trailing-spaces)
[error] 26-26: trailing spaces
(trailing-spaces)
[warning] 38-38: wrong indentation: expected 2 but found 0
(indentation)
[warning] 41-41: wrong indentation: expected 4 but found 2
(indentation)
examples/demo-atmos.d/custom-import/atmos.yaml
[error] 26-26: trailing spaces
(trailing-spaces)
[error] 29-29: trailing spaces
(trailing-spaces)
[error] 32-32: trailing spaces
(trailing-spaces)
[warning] 44-44: wrong indentation: expected 2 but found 0
(indentation)
[warning] 47-47: wrong indentation: expected 4 but found 2
(indentation)
🔇 Additional comments (6)
examples/demo-atmos.d/atmos.yaml (1)
30-33: Add validation for log levels
Consider adding validation to ensure only valid log levels (Debug, Info, Warning, Error) are accepted.
examples/demo-atmos.d/custom-import/atmos.yaml (3)
7-9: Consider adding URL validation for remote configurations
While loading from remote URLs provides flexibility, it could pose security risks. Consider implementing URL validation and HTTPS enforcement.
Would you like me to propose a validation pattern for secure URL handling?
18-24: Strong stack configuration, warrior! 💪
The stack configuration is well-structured with clear inclusion/exclusion patterns and a simple naming convention.
11-16: Verify backend file management strategy
Setting auto_generate_backend_file: false means you'll need to manage backend configurations manually. This is a valid choice but requires careful consideration.
Run this check to ensure consistent backend configuration across components:
✅ Verification successful
Setting auto_generate_backend_file: false is appropriate for this setup
The configuration choice is valid as there's no existing Terraform component directory at the specified path, indicating this is likely a new project setup. Manual backend management is a reasonable approach when starting fresh, allowing for more controlled and explicit backend configuration as the project evolves.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Check for existing backend configurations
fd -e tf -e hcl . components/terraform -x grep -l 'backend'
Length of output: 167
Script:
#!/bin/bash
# First, let's verify if the components/terraform directory exists and check the directory structure
fd --type d terraform
# Then, let's search for backend configurations across all Terraform files in the repository
fd -e tf -e hcl . -x grep -l 'backend'
# Also check for any Terraform component references in the atmos configuration
fd -g "*.yaml" -x grep -A 5 -B 5 "terraform:"
Length of output: 33961
Script:
#!/bin/bash
# Let's check for backend configurations in the actual Terraform components
fd -t f -e tf . examples/demo-atmos.d/custom-import/components/terraform -x grep -l 'backend'
# Also check for any backend configuration files
fd -t f -g "*.backend.tf" -g "backend.tf" examples/demo-atmos.d/custom-import/components/terraform
# Check if there are any existing backend configurations in the component directory
ls -la examples/demo-atmos.d/custom-import/components/terraform/
Length of output: 709
pkg/utils/glob_utils.go (2)
7-7: LGTM! Clean import addition, warrior!
The sort package import is well-placed and necessary for the new functionality.
Line range hint 19-48: Verify caching behavior for dynamic configurations, brave one!
The caching mechanism using sync.Map could prevent picking up new configuration files added to atmos.d during runtime. Additionally, we should verify that the error handling aligns with the requirement to log non-existent paths as warnings.
Let's verify the implementation:
✅ Verification successful
Cache invalidation is not a concern, brave warrior!
The caching mechanism in GetGlobMatches is pattern-based, which means:
- Each unique glob pattern gets its own cache entry
- When a new file is added that matches an existing pattern, it will be discovered on the next process restart
- The cache is process-scoped (using
sync.Map), so it's automatically cleared between runs - Error handling correctly returns both the error and logs the non-existent paths
The implementation aligns perfectly with the requirements:
- Cache is scoped to the current process, preventing stale state between runs
- Error handling returns descriptive errors for non-existent paths:
failed to find a match for the import '%s' ('%s' + '%s')
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Check for warning logs related to non-existent paths and cache invalidation
# Search for warning log implementations
rg -A 2 "failed to find a match for the import" .
# Check if there's any cache invalidation mechanism
rg -A 5 "getGlobMatchesSyncMap" .
# Look for any warning level log statements
rg -A 2 "Warning|WARN|warn" .
Length of output: 47400
There was a problem hiding this comment.
Actionable comments posted: 3
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (2)
pkg/config/config.go(3 hunks)pkg/schema/schema.go(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- pkg/schema/schema.go
🔇 Additional comments (1)
pkg/config/config.go (1)
231-232: LGTM! Clean error handling.
The multi-line error return improves readability.
…correct command structure
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Outside diff range and nitpick comments (3)
examples/demo-atmos.d/custom-import/atmos.yaml (2)
26-34: Clean up trailing spacesRemove trailing spaces from lines 26, 29, and 32 for better maintainability.
-vendor: +vendor: # Single file -base_path: "./vendor.yaml" +base_path: "./vendor.yaml" # Directory with multiple files -#base_path: "./vendor" +#base_path: "./vendor"🧰 Tools
🪛 yamllint (1.29.0-1)
[error] 26-26: trailing spaces
(trailing-spaces)
[error] 29-29: trailing spaces
(trailing-spaces)
[error] 32-32: trailing spaces
(trailing-spaces)
36-38: Logging configuration is good but could be enhancedCurrent configuration is solid. Consider adding:
- Log rotation settings
- Format specification
- Additional log destinations for production environments
pkg/config/config.go (1)
401-444: Consider adding rate limiting and path validationThe import processing is robust, but consider these enhancements:
- Add rate limiting for remote downloads to prevent abuse
- Validate import paths before processing to ensure they're safe
func processImports(cliConfig schema.CliConfiguration, v *viper.Viper) error { + // Add rate limiting for remote downloads + rateLimiter := time.NewTicker(time.Second) + defer rateLimiter.Stop() + for _, importPath := range cliConfig.Import { if importPath == "" { continue } + // Basic path validation + if strings.Contains(importPath, "..") { + u.LogWarning(cliConfig, fmt.Sprintf("Warning: skipping potentially unsafe import path '%s'", importPath)) + continue + }
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (4)
examples/demo-atmos.d/atmos.d/config1.yaml(1 hunks)examples/demo-atmos.d/atmos.d/sub/config2.yaml(1 hunks)examples/demo-atmos.d/custom-import/atmos.yaml(1 hunks)pkg/config/config.go(3 hunks)
✅ Files skipped from review due to trivial changes (2)
- examples/demo-atmos.d/atmos.d/config1.yaml
- examples/demo-atmos.d/atmos.d/sub/config2.yaml
🧰 Additional context used
🪛 yamllint (1.29.0-1)
examples/demo-atmos.d/custom-import/atmos.yaml
[error] 26-26: trailing spaces
(trailing-spaces)
[error] 29-29: trailing spaces
(trailing-spaces)
[error] 32-32: trailing spaces
(trailing-spaces)
🔇 Additional comments (7)
examples/demo-atmos.d/custom-import/atmos.yaml (4)
11-16: Terraform security settings look good!
The Terraform component configuration follows security best practices:
apply_auto_approve: falseprevents unintended infrastructure changesauto_generate_backend_file: falseensures explicit backend configuration
18-24: Stack configuration structure is well-organized!
Good use of:
- Explicit path inclusions/exclusions
- Clear naming pattern
- Logical directory structure
43-47: Command configuration looks good!
The test command is well-defined with clear description and steps.
7-9: Verify security measures for remote configuration imports
The remote URL import capability could pose security risks. Ensure that:
- HTTPS is enforced for remote URLs
- URL validation is implemented
- Content verification is performed before applying configurations
Consider implementing:
- URL allowlist/blocklist
- SSL certificate validation
- Content integrity checks (e.g., checksums)
pkg/config/config.go (3)
231-232: Clean error handling format!
The multi-line error return improves readability while maintaining functionality.
239-250: Validate path to prevent directory traversal
While the implementation correctly handles the atmos.d directory existence, consider adding path validation as suggested in the past review to prevent potential path traversal attacks.
446-464: Add safeguards to remote downloads
The download function needs the safeguards mentioned in the past review:
- Context timeout
- File size limits
- Temporary directory cleanup
…onfig and processImports
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Outside diff range and nitpick comments (4)
pkg/config/config.go (4)
231-232: Consider using idiomatic Go error handling.While splitting the return statement improves readability, consider using the more idiomatic Go style:
- return cliConfig, - err + return cliConfig, fmt.Errorf("failed to unmarshal config: %w", err)
239-256: Strong implementation with robust security measures!The atmos.d directory support is well-implemented with proper path validation and error handling. The path traversal prevention is particularly noteworthy.
Consider adding more context to the error message:
- return cliConfig, fmt.Errorf("invalid atmos.d path: attempted directory traversal") + return cliConfig, fmt.Errorf("invalid atmos.d path '%s': attempted directory traversal outside base path '%s'", atmosDPath, basePath)
257-269: Consider adding debug logging for configuration merging.The import processing logic is sound, but adding debug logs would help troubleshoot configuration issues.
err = v.Unmarshal(&cliConfig) if err != nil { + u.LogTrace(cliConfig, fmt.Sprintf("Failed to unmarshal merged config: %v", err)) return cliConfig, err } + u.LogTrace(cliConfig, "Successfully merged and unmarshaled imported configurations")
407-459: Consider parallel processing for multiple imports.The implementation is robust with proper error handling and path validation. However, processing imports sequentially might be slow for many imports.
Consider using a worker pool for parallel processing:
func processImports(cliConfig schema.CliConfiguration, v *viper.Viper) error { + type importResult struct { + path string + err error + } + workers := make(chan struct{}, 4) // Limit concurrent downloads + results := make(chan importResult) + for _, importPath := range cliConfig.Import { - if importPath == "" { - continue - } + go func(imp string) { + workers <- struct{}{} // Acquire worker + defer func() { <-workers }() // Release worker + + if imp == "" { + results <- importResult{"", nil} + return + } + // ... rest of the import processing logic ... + }(importPath) }
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (1)
pkg/config/config.go(3 hunks)
🔇 Additional comments (1)
pkg/config/config.go (1)
461-479: Past review comment about remote download safeguards is still applicable.
The previous review comment suggesting context timeout, file size limits, and temporary directory cleanup remains valid.
…andle cleanup on error
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Outside diff range and nitpick comments (1)
pkg/config/config.go (1)
232-233: Combine the return statement into a single lineThe return statement is split across two lines. Merging it into one line enhances readability.
Apply this diff:
- return cliConfig, - err + return cliConfig, err
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (1)
pkg/config/config.go(3 hunks)
🔇 Additional comments (4)
pkg/config/config.go (4)
240-257: Good handling of the 'import' key and path security
The logic for setting the default Import path when it's empty is well-implemented. Ensuring the path doesn't escape the intended directory adds a valuable security check.
259-270: Import processing added effectively
The addition of import processing and re-unmarshalling is correctly handled. Error checks are appropriately placed.
417-426: Consider validating remote URLs for security
Currently, any URL starting with "http://" or "https://" is accepted. Validating these URLs against an allowlist can prevent unintended or malicious imports.
464-484: Add a file size limit check in downloadRemoteConfig
While the context timeout is set, the function lacks a file size limit check. This could allow the download of excessively large files, leading to resource exhaustion.
|
Update documentation for Atmos CLI Configuration schema and examples |
|
Update documentation for Atmos CLI Configuration |
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
website/docs/cli/configuration/configuration.mdx (2)
27-27: Add punctuation at the end of the preceding paragraph.Add a period after "and so forth" to properly end the paragraph.
-you'd specify the command to run (e.g. [`opentofu`](/core-concepts/projects/configuration/opentofu)), the base path location of the components, and so forth +you'd specify the command to run (e.g. [`opentofu`](/core-concepts/projects/configuration/opentofu)), the base path location of the components, and so forth.🧰 Tools
🪛 LanguageTool
[grammar] ~27-~27: Please add a punctuation mark at the end of paragraph.
Context: ...f the components, and so forth. ## Base Path The base path defines the root directo...(PUNCTUATION_PARAGRAPH_END)
31-31: Remove unnecessary comma before 'if'.For better readability, remove the comma before the 'if' clause.
-When the `base_path` is explicitly defined, it will always be resolved to the absolute path relative to the current working directory, if an absolute path was not already set. +When the `base_path` is explicitly defined, it will always be resolved to the absolute path relative to the current working directory if an absolute path was not already set.🧰 Tools
🪛 LanguageTool
[typographical] ~31-~31: Usually, there’s no comma before “if”.
Context: ...elative to the current working directory, if an absolute path was not already set. ...(IF_NO_COMMA)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
website/docs/cli/configuration/configuration.mdx(1 hunks)
🧰 Additional context used
🪛 LanguageTool
website/docs/cli/configuration/configuration.mdx
[grammar] ~27-~27: Please add a punctuation mark at the end of paragraph.
Context: ...f the components, and so forth. ## Base Path The base path defines the root directo...
(PUNCTUATION_PARAGRAPH_END)
[typographical] ~31-~31: Usually, there’s no comma before “if”.
Context: ...elative to the current working directory, if an absolute path was not already set. ...
(IF_NO_COMMA)
[style] ~32-~32: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ... an absolute path was not already set. When the base_path is not explicitly provi...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
🔇 Additional comments (1)
website/docs/cli/configuration/configuration.mdx (1)
44-78: Well-structured documentation of configuration loading and imports!
The documentation clearly explains:
- Configuration file loading order and precedence
- Import capabilities and supported formats
- Security considerations for remote imports
|
|
||
| ## Configuration File (`atmos.yaml`) | ||
| The base path defines the root directory for components, stacks, workflows, and Atmos configurations. It can be set using the `ATMOS_BASE_PATH` environment variable or by passing the `--base-path` command-line argument. If both are set, the `--base-path` will take precedence. Both absolute and relative paths are supported. When a relative path is provided, it will be expanded to an absolute path relative to the current working directory, as Atmos executes commands that may change the working directory (e.g., Terraform). |
There was a problem hiding this comment.
| The base path defines the root directory for components, stacks, workflows, and Atmos configurations. It can be set using the `ATMOS_BASE_PATH` environment variable or by passing the `--base-path` command-line argument. If both are set, the `--base-path` will take precedence. Both absolute and relative paths are supported. When a relative path is provided, it will be expanded to an absolute path relative to the current working directory, as Atmos executes commands that may change the working directory (e.g., Terraform). | |
| The base path defines the root directory for components, stacks, workflows, and Atmos configurations. It can be set using the `ATMOS_BASE_PATH` environment variable or by passing the `--base-path` command-line argument. If both are set, the `--base-path` will take precedence. Both absolute and relative paths are supported. When a relative path is provided, it will be expanded to an absolute path relative to the current working directory. This is because Atmos executes commands that may change the working directory (e.g., Terraform). |
There was a problem hiding this comment.
It mostly affects the terraform-provider-utils which reads the atmos configs.
In geodesic, we put atmos.yaml in /usr/local/etc/atmos/atmos.yaml which is a predefined search order places we read from.
There was a problem hiding this comment.
Any relative path is not compatible with the terraform-provider-utils, because terraform changes the directory.
There was a problem hiding this comment.
We hope to remove the for the utils provider in the future versions of our components and refarch.
| When the `base_path` is explicitly defined, it will always be resolved to the absolute path relative to the current working directory, if an absolute path was not already set. | ||
|
|
||
| When the `base_path` is not explicitly provided or is empty, Atmos will attempt to infer the appropriate path using the following strategy, in order of precedence: | ||
| 1. If the current working directory contains any of the following files or directories: `atmos.yaml`, `.atmos.yaml`, `.github/atmos.yaml`, or `atmos.d/**/*`, Atmos will use the directory containing these files as the `base_path`. |
There was a problem hiding this comment.
To be clear, it's the prefix of the filename, not the dirname.
There was a problem hiding this comment.
I think .github/atmos.yaml and .atmos.yaml from the root of the repo, but not any subpath. I think this is moderately better from security standpoint as well.
| When the `base_path` is not explicitly provided or is empty, Atmos will attempt to infer the appropriate path using the following strategy, in order of precedence: | ||
| 1. If the current working directory contains any of the following files or directories: `atmos.yaml`, `.atmos.yaml`, `.github/atmos.yaml`, or `atmos.d/**/*`, Atmos will use the directory containing these files as the `base_path`. | ||
| 2. If a Git repository is detected, Atmos uses the root of the Git repository as the `base_path`. | ||
| 3. If neither of the above applies, Atmos defaults to the absolute path of `./` as the `base_path`. |
There was a problem hiding this comment.
Need to rewrite this for cli path, and when we do, and no config found, then use the default config. I think it would probably tie in to the go:embed file system, but need to look into it more.
| ## Configuration File (`atmos.yaml`) | ||
|
|
||
| The Atmos (CLI) configuration is loaded from the following locations, in order of precedence, and deep-merged: configurations listed earlier have higher priority, meaning they override any conflicting sections defined in subsequent locations: | ||
| - The `--config` parameter is the relative or absolute path to a valid configuration file or directory containing strictly atmos configuration file(s). It has the highest precedence, meaning it supersedes everything. |
There was a problem hiding this comment.
| - The `--config` parameter is the relative or absolute path to a valid configuration file or directory containing strictly atmos configuration file(s). It has the highest precedence, meaning it supersedes everything. | |
| - The `--config` flag is the relative or absolute path to a valid configuration file or directory containing strictly atmos configuration file(s). It has the highest precedence, meaning it supersedes everything. |
There was a problem hiding this comment.
When the --config is specified, all other search paths are ignored. It is the canonical configuration location. For example, no paths in XDG_CONFIG_HOME are loaded. We should be able to specify --config multiple times, and they are deep merged in the order. The first config specified is the lowest config, and the last one is the highest priority config.
|
|
||
| The Atmos (CLI) configuration is loaded from the following locations, in order of precedence, and deep-merged: configurations listed earlier have higher priority, meaning they override any conflicting sections defined in subsequent locations: | ||
| - The `--config` parameter is the relative or absolute path to a valid configuration file or directory containing strictly atmos configuration file(s). It has the highest precedence, meaning it supersedes everything. | ||
| - The `$XDG_CONFIG_HOME/atmos/atmos.yaml` file (if `$XDG_CONFIG_HOME` is set) allows the user to specify their preferences and overrides. It is only superseded by the `--config` flag. |
There was a problem hiding this comment.
Let's see if there's a go XDG library so we support conventional usage.
https://github.com/adrg/xdg
| - `.atmos.yaml` | ||
| - `.github/atmos.yaml` | ||
| - `atmos.d/**/*` | ||
| - `.atmos.d/**/*` | ||
| - Home directory (`~/.atmos/atmos.yaml`) |
There was a problem hiding this comment.
What happens on windows with this?
There was a problem hiding this comment.
https://github.com/mitchellh/go-homedir (Archived, so look for maintained fork)
| - `atmos.yaml` | ||
| - `.atmos.yaml` | ||
| - `.github/atmos.yaml` | ||
| - `atmos.d/**/*` | ||
| - `.atmos.d/**/*` |
There was a problem hiding this comment.
The implied order here is highest to lowest presedence.
| From the current directory: | ||
| - `atmos.yaml` | ||
| - `.atmos.yaml` | ||
| - `.github/atmos.yaml` |
There was a problem hiding this comment.
The .github/atmos.yaml maybe should have the lowest priority.
| ## Configuration File (`atmos.yaml`) | ||
| The base path defines the root directory for components, stacks, workflows, and Atmos configurations. It can be set using the `ATMOS_BASE_PATH` environment variable or by passing the `--base-path` command-line argument. If both are set, the `--base-path` will take precedence. Both absolute and relative paths are supported. When a relative path is provided, it will be expanded to an absolute path relative to the current working directory, as Atmos executes commands that may change the working directory (e.g., Terraform). | ||
|
|
||
| When the `base_path` is explicitly defined, it will always be resolved to the absolute path relative to the current working directory, if an absolute path was not already set. |
There was a problem hiding this comment.
This will be setting ATMOS_CLI_CONFIG_PATH
| - The `--config` parameter is the relative or absolute path to a valid configuration file or directory containing strictly atmos configuration file(s). It has the highest precedence, meaning it supersedes everything. | ||
| - The `$XDG_CONFIG_HOME/atmos/atmos.yaml` file (if `$XDG_CONFIG_HOME` is set) allows the user to specify their preferences and overrides. It is only superseded by the `--config` flag. | ||
| - The environment variable `ATMOS_CLI_CONFIG_PATH` should point to a folder (or a specific file). This is mutually exclusive with the `--config` parameter. If `--config` is specified, then the `ATMOS_CLI_CONFIG_PATH` is ignored. Atmos will only search for the configuration file in the specified folder. Note that `.yaml` and `.yml` file extensions are treated interchangeably and are loaded in lexicographical order. Atmos will search for the following files, in order, relative to the `ATMOS_CLI_CONFIG_PATH` environment variable. | ||
| From the current directory: |
There was a problem hiding this comment.
| From the current directory: | |
| From the current directory: (the first one found, wins) |
| - Home directory (`~/.atmos/atmos.yaml`) | ||
| - Current directory (`./atmos.yaml`) | ||
| - Environment variable `ATMOS_CLI_CONFIG_PATH` (the ENV var should point to a folder without specifying the file name) | ||
| - The `atmos.d/` directory is loaded relative to the `base_path`. For an example, see [`examples/demo-atmos.d`](https://github.com/cloudposse/atmos/tree/main/examples/demo-atmos.d) |
There was a problem hiding this comment.
Note, we discuss atmos.d above and define it here. There's some incongruence.
There was a problem hiding this comment.
We should rewrite config handling to use https://github.com/spf13/viper
| For example, we can import from multiple locations like this: | ||
|
|
||
| ```yaml | ||
| import: |
There was a problem hiding this comment.
We do not intend to support templated imports at this time. Also, this syntax might change based on viper?
|
This PR was closed due to inactivity and merge conflicts. 😭 |
|
💥 This pull request now has conflicts. Could you fix it @haitham911? 🙏 |
what
Supported the atmos.d convention for atmos.yaml configuration, allowing automatic inclusion of configuration files from the atmos.d directory.
Made the path to atmos.d configurable within atmos.yaml, enabling users to specify custom directories for additional configurations.
Implemented deep merging of configurations in lexicographical order, recursively processing files within specified directories to ensure consistent and predictable configuration outcomes.
Added support for the import key inside atmos.yaml, allowing users to define a list of locations (local files, directories using glob patterns, and remote URLs) to import configurations from.
expose env variable ATMOS_CLI_CONFIG_PATH and ATMOS_BASE_PATH before run terraform and helm cmd
demos :
why
Simplifies configuration management by adopting the atmos.d directory convention, making it easier to include additional configurations without explicit declarations.
references
Summary by CodeRabbit
Summary by CodeRabbit
New Features
atmos.yamlfor managing imports and settings.Bug Fixes
Tests
Documentation
atmos.yamlconfiguration file, including new import paths and usage guidelines.