Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Organise SSM paramamters by path #2569

Merged
merged 19 commits into from
Nov 16, 2022
Merged

feat: Organise SSM paramamters by path #2569

merged 19 commits into from
Nov 16, 2022

Conversation

npalm
Copy link
Member

@npalm npalm commented Oct 23, 2022

Descriptions

Currently SSM parameters are not well organised,
this PR organise all paramaters by path.

SSM paths:

  • root path, default
  • prefix, added after the root
  • seperate paths for app, config and tokens

Tasks

  • Refactor module to organise imports
  • Support multi-runner module
  • Update start-template linux
  • Update start-template windows
  • Provide migration instructions

Changes in SSM

Before
Screenshot 2022-10-24 at 12 19 53

After
Screenshot 2022-10-24 at 12 31 54

Migration

Migration is only required when using pre-build AMI with your tailored start-runner script. Before all parameters were based on convetion. We have update them to be predictable with still similar conventions as default.

We set the tag ghr:ssm_config_path as EC2 instance tag to the path used for the configuration. In the config path we store the same values as before (run_as, enable_cloudwatch, agent_mode). The tokens are stored in a separate location which can be retrieved by looking up the value token_path in the config.

For a more details example check-out the following diffs:

@npalm
Copy link
Member Author

npalm commented Nov 1, 2022

Example snapshot for the multi-runner:
image

@npalm
Copy link
Member Author

npalm commented Nov 4, 2022

@ScottGuymer @GuptaNavdeep1983 kind reminder can you review the PR?

Copy link
Member

@ScottGuymer ScottGuymer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of small comments

.pre-commit-config.yaml Outdated Show resolved Hide resolved
examples/default/main.tf Show resolved Hide resolved
modules/runners/lambdas/runners/src/aws/runners.test.ts Outdated Show resolved Hide resolved
@npalm npalm force-pushed the feat/refactor-ssm-paths branch from a80ef13 to 33ff9e1 Compare November 15, 2022 10:28
npalm and others added 17 commits November 16, 2022 15:40
* chore: Remove support check_run

* format, lint
* feat: Remove support check_run (#2521)

* chore: Remove support check_run

* format, lint

* feat: Remove old scale down mechanism (< 0.19.0) (#2519)

fix: Remove old cleanup mechanism (< 0.19.0)

* feat: added changes for multi runner.

* fix: region.

* fix: more fixes.

* tuple to list.

* fixes.

* fixes.

* fixes.

* fixes.

* fixes.

* fixes.

* fix: formatting.

* fix: formatting.

* fix: formatting.

* fix: moved some blocks outside runner config.

* fix: few more updates

* fix: liniting.

* fix: updated example output

* changed runner group name.

* fix: updated the tests.

* fix: addressed review comments.

* fix: linting issues.

* fix: formatting.

* fix: updated tf version.

* fix: Remove removed prerelease option

* Add ubuntu runner to example

* refactor: use each instead of count

* fix: few small issues.

* refactor: syncer to count for multi runner

* fix: comments.

* fix: added Readme.

* fix: errors.

* move variable to runner config

* fix: updated the readme.

* Add todos

* feat: added windows runner configuration, completed todos and added the weight for runner config matchers.

* chore: Update docs

* fix: reverted tf versions.

* fix: addressed comments.

* fix: missed.

* fix: formatting.

* Update terraform versions in CI

* Update terraform versions in CI

* Update docs

* fix: coverage.

* Update docs

* improve test coverage webhook

* Apply suggestions from code review

* fix: formatting.

* fix: fixed merge issues.

* fix: syntax.

Co-authored-by: Niek Palm <npalm@users.noreply.github.com>
Co-authored-by: Niek Palm <niek.palm@philips.com>
Co-authored-by: navdeepg2021 <navdeepg2021@gmail.com>
* **ci:**

* Add multi-runner capability ([#2472](#2472)) ([c08b335](c08b335)), closes [#2521](#2521) [#2519](#2519)
* Experimental feature - Duplicate workflow job event to extra queue ([#2268](#2268)) ([ac046b8](ac046b8))
* Remove old scale down mechanism (< 0.19.0) ([#2519](#2519)) ([721d7c3](721d7c3))
* Remove support check_run ([#2521](#2521)) ([272a293](272a293))

* **ci:** Enable ci for next branch ([48769ca](48769ca)), closes [#2472](#2472) [#2517](#2517) [#2521](#2521) [#2517](#2517) [#2519](#2519) [#2517](#2517)
Currently SSM parameters are not well organised,
this PR organise all paramaters by path.

SSM paths:
- root path, default
- prefix, added after the root
- seperate paths for app, config and tokens
- Add ci for tflint
- fix errors and warning (expect a few to test CI)
- update modules
- remove dublicated download lambda to a simple module for examples
@npalm npalm force-pushed the feat/refactor-ssm-paths branch from 127fe0a to 9ad5928 Compare November 16, 2022 14:44
@npalm npalm merged this pull request into next Nov 16, 2022
@npalm npalm deleted the feat/refactor-ssm-paths branch November 16, 2022 16:25
npalm added a commit that referenced this pull request Nov 16, 2022
Migration is only required when using pre-build AMI with your tailored start-runner script. Before all parameters were based on convetion. We have update them to be predictable with still similar conventions as default.

We set the tag ghr:ssm_config_path as EC2 instance tag to the path used for the configuration. In the config path we store the same values as before (run_as, enable_cloudwatch, agent_mode). The tokens are stored in a separate location which can be retrieved by looking up the value token_path in the config.

For a more details example check-out the following diffs:

Linux start script // TODO UPDATE AFTER MERGE
Windows start script // TODO UPDATE AFTER MERGE
npalm added a commit that referenced this pull request Nov 17, 2022
Migration is only required when using pre-build AMI with your tailored start-runner script. Before all parameters were based on convetion. We have update them to be predictable with still similar conventions as default.

We set the tag ghr:ssm_config_path as EC2 instance tag to the path used for the configuration. In the config path we store the same values as before (run_as, enable_cloudwatch, agent_mode). The tokens are stored in a separate location which can be retrieved by looking up the value token_path in the config.

For a more details example check-out the following diffs:

Linux start script // TODO UPDATE AFTER MERGE
Windows start script // TODO UPDATE AFTER MERGE
npalm added a commit that referenced this pull request Dec 13, 2022
Migration is only required when using pre-build AMI with your tailored start-runner script. Before all parameters were based on convetion. We have update them to be predictable with still similar conventions as default.

We set the tag ghr:ssm_config_path as EC2 instance tag to the path used for the configuration. In the config path we store the same values as before (run_as, enable_cloudwatch, agent_mode). The tokens are stored in a separate location which can be retrieved by looking up the value token_path in the config.

For a more details example check-out the following diffs:

Linux start script // TODO UPDATE AFTER MERGE
Windows start script // TODO UPDATE AFTER MERGE
npalm added a commit that referenced this pull request Dec 13, 2022
Migration is only required when using pre-build AMI with your tailored start-runner script. Before all parameters were based on convetion. We have update them to be predictable with still similar conventions as default.

We set the tag ghr:ssm_config_path as EC2 instance tag to the path used for the configuration. In the config path we store the same values as before (run_as, enable_cloudwatch, agent_mode). The tokens are stored in a separate location which can be retrieved by looking up the value token_path in the config.

For a more details example check-out the following diffs:

Linux start script // TODO UPDATE AFTER MERGE
Windows start script // TODO UPDATE AFTER MERGE
npalm added a commit that referenced this pull request Dec 28, 2022
Migration is only required when using pre-build AMI with your tailored start-runner script. Before all parameters were based on convetion. We have update them to be predictable with still similar conventions as default.

We set the tag ghr:ssm_config_path as EC2 instance tag to the path used for the configuration. In the config path we store the same values as before (run_as, enable_cloudwatch, agent_mode). The tokens are stored in a separate location which can be retrieved by looking up the value token_path in the config.

For a more details example check-out the following diffs:

Linux start script // TODO UPDATE AFTER MERGE
Windows start script // TODO UPDATE AFTER MERGE
npalm added a commit that referenced this pull request Dec 28, 2022
Migration is only required when using pre-build AMI with your tailored start-runner script. Before all parameters were based on convetion. We have update them to be predictable with still similar conventions as default.

We set the tag ghr:ssm_config_path as EC2 instance tag to the path used for the configuration. In the config path we store the same values as before (run_as, enable_cloudwatch, agent_mode). The tokens are stored in a separate location which can be retrieved by looking up the value token_path in the config.

For a more details example check-out the following diffs:

Linux start script // TODO UPDATE AFTER MERGE
Windows start script // TODO UPDATE AFTER MERGE
npalm added a commit that referenced this pull request Dec 28, 2022
Migration is only required when using pre-build AMI with your tailored start-runner script. Before all parameters were based on convetion. We have update them to be predictable with still similar conventions as default.

We set the tag ghr:ssm_config_path as EC2 instance tag to the path used for the configuration. In the config path we store the same values as before (run_as, enable_cloudwatch, agent_mode). The tokens are stored in a separate location which can be retrieved by looking up the value token_path in the config.

For a more details example check-out the following diffs:

Linux start script // TODO UPDATE AFTER MERGE
Windows start script // TODO UPDATE AFTER MERGE
npalm added a commit that referenced this pull request Dec 28, 2022
Migration is only required when using pre-build AMI with your tailored start-runner script. Before all parameters were based on convetion. We have update them to be predictable with still similar conventions as default.

We set the tag ghr:ssm_config_path as EC2 instance tag to the path used for the configuration. In the config path we store the same values as before (run_as, enable_cloudwatch, agent_mode). The tokens are stored in a separate location which can be retrieved by looking up the value token_path in the config.

For a more details example check-out the following diffs:

Linux start script // TODO UPDATE AFTER MERGE
Windows start script // TODO UPDATE AFTER MERGE
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants