Setup environment variable collection #2963

BobanL · 2024-11-27T16:56:41Z

PULL REQUEST

Summary

Create new COLLECTION env variable to help support the different profiles
- if anyone has other suggestions for names please let me know since I don't really care for COLLECTION
Use dotenvx to load all environment variables
- One image is created and depending on the value of the environment variable COLLECTION, a different profile will be used to start the server
Update package scripts so COLLECTION and it's variables only need to be set in one place
- Subscripts were created to allow the additional variables to be injected for local development.

Cleanup 🧹

Rename APP_ENV to NBS_AUTH
Combine Azure-storage and azure init into a single dockerfile so that convert-seed-data doesn't abort when the init is complete
Remove wrapping around S3 Client creation since undefined values will not effect the creation of the client
Fix conflicting IDs in LA's example eCRs.

Related Issue

Fixes #2873

Acceptance Criteria

collections created for each
- AWS integrated
  - APP_ENV=test
  - NEXT_PUBLIC_NON_INTEGRATED_VIEWER=false
  - SOURCE=azure
  - METADATA_DATABASE_TYPE=undefined
  - METADATA_DATABASE_SCHEMA=undefined
- Azure integrated
  - APP_ENV=test
  - NEXT_PUBLIC_NON_INTEGRATED_VIEWER=false
  - SOURCE=azure
  - METADATA_DATABASE_TYPE=undefined
  - METADATA_DATABASE_SCHEMA=undefined
- AWS non-integrated
  - APP_ENV=prod
  - NEXT_PUBLIC_NON_INTEGRATED_VIEWER=true
  - SOURCE=s3
  - METADATA_DATABASE_TYPE=sqlserver
  - METADATA_DATABASE_SCHEMA=extended
- Azure non-integrated
  - APP_ENV=prod
  - NEXT_PUBLIC_NON_INTEGRATED_VIEWER=true
  - SOURCE=azure
  - METADATA_DATABASE_TYPE=sqlserver
  - METADATA_DATABASE_SCHEMA=?
Setting the new configuration name environment variable causes the correct set of environment variables to be set on startup

Additional Information

Anything else the review team should know?

Checklist

If this code affects the other scrum team, have they been notified? (In Slack, as reviewers, etc.)

docker compose --env-file .env.local --profile "*" up to test with different configs

…e cli

containers/ecr-viewer/env/.env.AWS_INTEGRATED

containers/ecr-viewer/env/.env.AZURE_INTEGRATED

containers/ecr-viewer/src/app/api/services/s3Client.ts

containers/message-parser/app/default_schemas/core.json

containers/orchestration/docker-compose.yml

containers/ecr-viewer/.env.sample

containers/ecr-viewer/docker-compose.yml

mcmcgrath13 · 2024-12-12T14:36:20Z

containers/ecr-viewer/.env.sample

-APP_ENV=test
-NEXT_PUBLIC_NON_INTEGRATED_VIEWER=true
-SOURCE=postgres
+COLLECTION=AWS_SQLSERVER_NON_INTEGRATED


spitballing:

ENV_FILE

ENV_COLLECTION

APP_CONFIG

CONFIG_NAME

CONFIG_COLLECTION

mcmcgrath13 · 2024-12-12T14:42:32Z

containers/ecr-viewer/package.json

-    "clear-local": "docker compose -f ./docker-compose.yml --profile \"*\" down -v",
-    "convert-seed-data": "npx @dotenvx/dotenvx run -f .env.local -- sh -c 'docker compose -f ./seed-scripts/docker-compose-seed.yml --profile ${SOURCE} --profile ${METADATA_DATABASE_TYPE} --profile ecr-viewer up --abort-on-container-exit'",
-    "convert-seed-data:build": "npx @dotenvx/dotenvx run -f .env.local -- sh -c 'docker compose -f ./seed-scripts/docker-compose-seed.yml --profile ${SOURCE} --profile ${METADATA_DATABASE_TYPE} --profile ecr-viewer up --abort-on-container-exit --build'",
+    "clear-local": "docker compose -f ./docker-compose.yml --profile \"*\" down -v && docker compose -f ./seed-scripts/docker-compose-seed.yml --profile \"*\" down -v",


I think we'd only need the seed-script one since it includes the outer one

mcmcgrath13 · 2024-12-12T14:52:17Z

containers/ecr-viewer/design-review/design-review.sh

@@ -89,19 +89,25 @@ npm i

 # Write env vars to .env.local
 npm run setup-local-env
-echo "NEXT_PUBLIC_NON_INTEGRATED_VIEWER=$IS_NON_INTEGRATED" >> .env.local
+echo "NBS_AUTH=false" >> .env.local


does this always hold even when the collection has it set the opposite?

BobanL added 15 commits November 25, 2024 16:15

ignore .next in jest

2b06396

mute console error from jest results

33a8246

rename app_env to nbs_auth

b262450

docker compose --env-file .env.local --profile "*" up to test with different configs

Remove app env check around s3 client

e7878dc

add collection variables for env groups

881700f

fix typos in .env files

2e24379

create single azure-storage image that contains both azurite and azur…

a1b3f0b

…e cli

add additional chaining to run specific collections

50a94be

fix local-dev command to use collection

d267607

update create-seed-data to use different configs depending on metadata

d11622e

rename schemas

aaf6922

update .env.sample to include collection

2c154e8

remove azure-storage-init from orchestration

0edacc9

update orchestration docker compose to add collection

961555e

add metadata db schema in the env variable

c2c1048

BobanL marked this pull request as ready for review December 2, 2024 15:04

BobanL requested review from lina-roth, angelathe, gordonfarrell, JNygaard-Skylight and austin-hall-skylight December 2, 2024 15:04

austin-hall-skylight requested changes Dec 2, 2024

View reviewed changes

BobanL added 4 commits December 2, 2024 14:09

remove undefined values

15bf72e

remove nbs_auth value

c53a3b9

Remove redundant variables

48c65e2

change to aws non integrated and add timeout

952e48c

austin-hall-skylight previously approved these changes Dec 2, 2024

View reviewed changes

BobanL added 3 commits December 3, 2024 14:43

Merge branch 'main' into boban/2873-env-variable-collections

8e7b765

add docker ignore

4625ccc

Merge branch 'main' into boban/2873-env-variable-collections

ac67fb2

BobanL added 8 commits December 11, 2024 14:15

remove console log from test

7486088

remove unused environment values in ecr-viewer docker-compose

141e172

reorder convert-seed-data

a6646d2

change orchestration's ecr-viewer collection

3b0e7c8

simplify build step by setting collection as profile

4817eaf

Update create-seed-data to use COLLECTION env variable

2c88219

add a local-docker command too!

299c6e3

lighthouse start app command

b4707b6

BobanL marked this pull request as ready for review December 11, 2024 21:50

Merge branch 'main' into boban/2873-env-variable-collections

710dc82

BobanL changed the title ~~env variable collections~~ Setup environment variable collection Dec 11, 2024

BobanL requested review from lina-roth, mcmcgrath13, angelathe, JNygaard-Skylight, austin-hall-skylight and gordonfarrell December 11, 2024 21:51

BobanL added 2 commits December 11, 2024 19:13

make changes to design review script

7f38062

add a comment for nbs auth in .env.sample

440db5b

BobanL removed request for lina-roth, mcmcgrath13, angelathe, JNygaard-Skylight, austin-hall-skylight and gordonfarrell December 12, 2024 13:54

mcmcgrath13 reviewed Dec 12, 2024

View reviewed changes

BobanL marked this pull request as draft December 13, 2024 16:27

BobanL closed this Dec 19, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Setup environment variable collection #2963

Setup environment variable collection #2963

BobanL commented Nov 27, 2024 •

edited

Loading

mcmcgrath13 Dec 12, 2024

mcmcgrath13 Dec 12, 2024

mcmcgrath13 Dec 12, 2024

Setup environment variable collection #2963

Setup environment variable collection #2963

Conversation

BobanL commented Nov 27, 2024 • edited Loading

PULL REQUEST

Summary

Cleanup 🧹

Related Issue

Acceptance Criteria

Additional Information

Checklist

mcmcgrath13 Dec 12, 2024

Choose a reason for hiding this comment

mcmcgrath13 Dec 12, 2024

Choose a reason for hiding this comment

mcmcgrath13 Dec 12, 2024

Choose a reason for hiding this comment

BobanL commented Nov 27, 2024 •

edited

Loading