Gen3 Monthly Release 2023.02 BDCat Preprod preprod.gen3.biodatacatalyst.nhlbi.nih.gov - 2023.02 1674838788 #5620

PlanXCyborg · 2023-01-27T16:59:50Z

Applying version 2023.02 to preprod.gen3.biodatacatalyst.nhlbi.nih.gov

PlanXCyborg · 2023-01-27T17:04:21Z

preprod.gen3.biodatacatalyst.nhlbi.nih.gov/manifest.json

Deployment changes

awshelper
- Add .global.karpenter to manifest if you'd like to use karpenter. (feat(karpenter): Added configuration to use karpenter instead of clus… cloud-automation#2128)
- To set up the metadata-delete-expired-objects job, run the gen3 kube-setup-metadata-delete-expired-objects-cronjob command. Note that the metadata-delete-expired-objects-job client must be granted access to (resource=/mds_gateway, method=access, service=mds_gateway) and (resource=/programs, method=delete, service=fence) ((PXP-9516): Add metadata-delete-expired-objects job cloud-automation#1941)
- Currently requires to manually update prisma api key in json format under this path in the adminvm (BRH-301: Extend hatchery to launch prismacloud containers cloud-automation#2061)
- ~/Gen3Secrets/prisma/apikey.json (BRH-301: Extend hatchery to launch prismacloud containers cloud-automation#2061)
- format has to be the following: (BRH-301: Extend hatchery to launch prismacloud containers cloud-automation#2061)
- { (BRH-301: Extend hatchery to launch prismacloud containers cloud-automation#2061)
- "AccessKeyID": "XXXX", (BRH-301: Extend hatchery to launch prismacloud containers cloud-automation#2061)
- "SecretKey": "XXXX" (BRH-301: Extend hatchery to launch prismacloud containers cloud-automation#2061)
- } (BRH-301: Extend hatchery to launch prismacloud containers cloud-automation#2061)
- This PR extends kube-setup-hatchery to create a kubernetes secret with for prisma cloud credentials, and extending the hatchery deployment to look for this secret and use it, if it exists. (BRH-301: Extend hatchery to launch prismacloud containers cloud-automation#2061)
dashboard
- This change will add some CSP and CORS related header to the response that returned from the dashboard services. They should not be interruptive, but each team is encouraged to double check their webpages hosted by dashboard to ensure they still works (fix: sec patch gen3-statics#27)
- Because of these CSP directives being added, starting from this version, if anyone what to load remote scripts in their dashboard-hosted webpages, they will need to update the CSP directives in this dashboard service if the current directives doesn't fit, or to ship the script files with the page (fix: sec patch gen3-statics#27)
fence
- Requires a Fence DB migration (PXP-10541 Client credentials rotation fence#1068)
portal
- New portal config options to exclude File nodes from the charts added: use "graphql.chartNodesExcludeFiles" to exclude File nodes from graphql chart, or use "components.index.homepageChartNodesExcludeFiles" to exclude File nodes from homepage chart. Note both of them are default to false so you have to explicitly set them to true to enable them (PXP-10565/PPS-177 fix: exclude file nodes from chart with options data-portal#1208)
- this change assumes WTS is already a core/central service. The dependency is hereby moved to the homepage (instead of the ./workspace page) (Centralize WTS refresh token initialization data-portal#1095)
sower
ssjdispatcher
tube

Breaking changes

arborist
- The GET /auth/mapping endpoint does not accept username as a query parameter anymore. It only supports parsing the username from the provided JWT. To specify a username, use the POST /auth/mapping endpoint. (PXP-10229 Client auth mapping arborist#153)
indexd
- Ensure you are using Fence>=2021.10. Remove deprecated GA4GH access endpoints. these have existed (and been used) from the Fence microservice since last year. See cloud automation change here (Revert "Revert "fix(migrate): improve acl->authz migration, remove de… indexd#338)

PlanXCyborg · 2023-01-27T17:17:05Z

preprod.gen3.biodatacatalyst.nhlbi.nih.gov/manifest.json

Deployment changes

awshelper
- Add .global.karpenter to manifest if you'd like to use karpenter. (feat(karpenter): Added configuration to use karpenter instead of clus… cloud-automation#2128)
- To set up the metadata-delete-expired-objects job, run the gen3 kube-setup-metadata-delete-expired-objects-cronjob command. Note that the metadata-delete-expired-objects-job client must be granted access to (resource=/mds_gateway, method=access, service=mds_gateway) and (resource=/programs, method=delete, service=fence) ((PXP-9516): Add metadata-delete-expired-objects job cloud-automation#1941)
- Currently requires to manually update prisma api key in json format under this path in the adminvm (BRH-301: Extend hatchery to launch prismacloud containers cloud-automation#2061)
- ~/Gen3Secrets/prisma/apikey.json (BRH-301: Extend hatchery to launch prismacloud containers cloud-automation#2061)
- format has to be the following: (BRH-301: Extend hatchery to launch prismacloud containers cloud-automation#2061)
- { (BRH-301: Extend hatchery to launch prismacloud containers cloud-automation#2061)
- "AccessKeyID": "XXXX", (BRH-301: Extend hatchery to launch prismacloud containers cloud-automation#2061)
- "SecretKey": "XXXX" (BRH-301: Extend hatchery to launch prismacloud containers cloud-automation#2061)
- } (BRH-301: Extend hatchery to launch prismacloud containers cloud-automation#2061)
- This PR extends kube-setup-hatchery to create a kubernetes secret with for prisma cloud credentials, and extending the hatchery deployment to look for this secret and use it, if it exists. (BRH-301: Extend hatchery to launch prismacloud containers cloud-automation#2061)
dashboard
- This change will add some CSP and CORS related header to the response that returned from the dashboard services. They should not be interruptive, but each team is encouraged to double check their webpages hosted by dashboard to ensure they still works (fix: sec patch gen3-statics#27)
- Because of these CSP directives being added, starting from this version, if anyone what to load remote scripts in their dashboard-hosted webpages, they will need to update the CSP directives in this dashboard service if the current directives doesn't fit, or to ship the script files with the page (fix: sec patch gen3-statics#27)
fence
- Requires a Fence DB migration (PXP-10541 Client credentials rotation fence#1068)
portal
- New portal config options to exclude File nodes from the charts added: use "graphql.chartNodesExcludeFiles" to exclude File nodes from graphql chart, or use "components.index.homepageChartNodesExcludeFiles" to exclude File nodes from homepage chart. Note both of them are default to false so you have to explicitly set them to true to enable them (PXP-10565/PPS-177 fix: exclude file nodes from chart with options data-portal#1208)
- this change assumes WTS is already a core/central service. The dependency is hereby moved to the homepage (instead of the ./workspace page) (Centralize WTS refresh token initialization data-portal#1095)
sower
ssjdispatcher
tube

Breaking changes

arborist
- The GET /auth/mapping endpoint does not accept username as a query parameter anymore. It only supports parsing the username from the provided JWT. To specify a username, use the POST /auth/mapping endpoint. (PXP-10229 Client auth mapping arborist#153)
indexd
- Ensure you are using Fence>=2021.10. Remove deprecated GA4GH access endpoints. these have existed (and been used) from the Fence microservice since last year. See cloud automation change here (Revert "Revert "fix(migrate): improve acl->authz migration, remove de… indexd#338)

PlanXCyborg · 2023-01-27T17:21:16Z

preprod.gen3.biodatacatalyst.nhlbi.nih.gov/manifest.json

Deployment changes

awshelper
- Add .global.karpenter to manifest if you'd like to use karpenter. (feat(karpenter): Added configuration to use karpenter instead of clus… cloud-automation#2128)
- To set up the metadata-delete-expired-objects job, run the gen3 kube-setup-metadata-delete-expired-objects-cronjob command. Note that the metadata-delete-expired-objects-job client must be granted access to (resource=/mds_gateway, method=access, service=mds_gateway) and (resource=/programs, method=delete, service=fence) ((PXP-9516): Add metadata-delete-expired-objects job cloud-automation#1941)
- Currently requires to manually update prisma api key in json format under this path in the adminvm (BRH-301: Extend hatchery to launch prismacloud containers cloud-automation#2061)
- ~/Gen3Secrets/prisma/apikey.json (BRH-301: Extend hatchery to launch prismacloud containers cloud-automation#2061)
- format has to be the following: (BRH-301: Extend hatchery to launch prismacloud containers cloud-automation#2061)
- { (BRH-301: Extend hatchery to launch prismacloud containers cloud-automation#2061)
- "AccessKeyID": "XXXX", (BRH-301: Extend hatchery to launch prismacloud containers cloud-automation#2061)
- "SecretKey": "XXXX" (BRH-301: Extend hatchery to launch prismacloud containers cloud-automation#2061)
- } (BRH-301: Extend hatchery to launch prismacloud containers cloud-automation#2061)
- This PR extends kube-setup-hatchery to create a kubernetes secret with for prisma cloud credentials, and extending the hatchery deployment to look for this secret and use it, if it exists. (BRH-301: Extend hatchery to launch prismacloud containers cloud-automation#2061)
dashboard
- This change will add some CSP and CORS related header to the response that returned from the dashboard services. They should not be interruptive, but each team is encouraged to double check their webpages hosted by dashboard to ensure they still works (fix: sec patch gen3-statics#27)
- Because of these CSP directives being added, starting from this version, if anyone what to load remote scripts in their dashboard-hosted webpages, they will need to update the CSP directives in this dashboard service if the current directives doesn't fit, or to ship the script files with the page (fix: sec patch gen3-statics#27)
fence
- Requires a Fence DB migration (PXP-10541 Client credentials rotation fence#1068)
portal
- New portal config options to exclude File nodes from the charts added: use "graphql.chartNodesExcludeFiles" to exclude File nodes from graphql chart, or use "components.index.homepageChartNodesExcludeFiles" to exclude File nodes from homepage chart. Note both of them are default to false so you have to explicitly set them to true to enable them (PXP-10565/PPS-177 fix: exclude file nodes from chart with options data-portal#1208)
- this change assumes WTS is already a core/central service. The dependency is hereby moved to the homepage (instead of the ./workspace page) (Centralize WTS refresh token initialization data-portal#1095)
sower
ssjdispatcher
tube

Breaking changes

arborist
- The GET /auth/mapping endpoint does not accept username as a query parameter anymore. It only supports parsing the username from the provided JWT. To specify a username, use the POST /auth/mapping endpoint. (PXP-10229 Client auth mapping arborist#153)
indexd
- Ensure you are using Fence>=2021.10. Remove deprecated GA4GH access endpoints. these have existed (and been used) from the Fence microservice since last year. See cloud automation change here (Revert "Revert "fix(migrate): improve acl->authz migration, remove de… indexd#338)

k-burt-uch · 2023-01-30T21:11:11Z

preprod.gen3.biodatacatalyst.nhlbi.nih.gov/manifest.json

+    "revproxy": "707767160287.dkr.ecr.us-east-1.amazonaws.com/gen3/nginx:2023.02",
+    "sheepdog": "707767160287.dkr.ecr.us-east-1.amazonaws.com/gen3/sheepdog:2023.02",
+    "metadata": "707767160287.dkr.ecr.us-east-1.amazonaws.com/gen3/metadata-service:2023.02",
+    "portal": "707767160287.dkr.ecr.us-east-1.amazonaws.com/gen3/data-portal:5.3.0",


I noticed this tag isn’t a YYYY.MM format - I take it that the data-portal fix we need isn’t in the 2023.02 tag?

k-burt-uch · 2023-01-30T21:16:57Z

preprod.gen3.biodatacatalyst.nhlbi.nih.gov/portal/gitops.json

-          },
-          "notAvailable": {
-            "enabled": true,
-            "menuText": "Not Available"


Might be missing something but I don't see any indication of PXP-10660 getting solved with this PR. Can we link that JIRA issue with this PR?

PlanXCyborg added 2 commits January 27, 2023 10:59

apply version 2023.02 to preprod.gen3.biodatacatalyst.nhlbi.nih.gov

0e366dc

apply version 2023.02 to preprod.gen3.biodatacatalyst.nhlbi.nih.gov

7546e2b

PlanXCyborg requested review from radhrreddy and a team as code owners January 27, 2023 16:59

PlanXCyborg added the gen3-release Enable additional tests and automation label Jan 27, 2023

SpencerAxelrod added 2 commits January 27, 2023 11:11

Use data-portal 5.3.0

f10b70c

Update config to Accessible and Not Accessible

c8875bd

PlanXCyborg added test-google-googleServiceAccountRemovalTest test-pidgin-getCoremetadataTest and removed gen3-release Enable additional tests and automation test-pidgin-getCoremetadataTest labels Jan 27, 2023

k-burt-uch reviewed Jan 30, 2023

View reviewed changes

k-burt-uch approved these changes Jan 30, 2023

View reviewed changes

SpencerAxelrod merged commit 5e5e0d1 into master Jan 30, 2023

SpencerAxelrod deleted the chore/apply_202302_to_preprod_gen3_biodatacatalyst_nhlbi_nih_gov_1674838788 branch January 30, 2023 21:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Gen3 Monthly Release 2023.02 BDCat Preprod preprod.gen3.biodatacatalyst.nhlbi.nih.gov - 2023.02 1674838788 #5620

Gen3 Monthly Release 2023.02 BDCat Preprod preprod.gen3.biodatacatalyst.nhlbi.nih.gov - 2023.02 1674838788 #5620

PlanXCyborg commented Jan 27, 2023

PlanXCyborg commented Jan 27, 2023

PlanXCyborg commented Jan 27, 2023

PlanXCyborg commented Jan 27, 2023

k-burt-uch Jan 30, 2023

k-burt-uch Jan 30, 2023

Gen3 Monthly Release 2023.02 BDCat Preprod preprod.gen3.biodatacatalyst.nhlbi.nih.gov - 2023.02 1674838788 #5620

Gen3 Monthly Release 2023.02 BDCat Preprod preprod.gen3.biodatacatalyst.nhlbi.nih.gov - 2023.02 1674838788 #5620

Conversation

PlanXCyborg commented Jan 27, 2023

PlanXCyborg commented Jan 27, 2023

preprod.gen3.biodatacatalyst.nhlbi.nih.gov/manifest.json

Deployment changes

Breaking changes

PlanXCyborg commented Jan 27, 2023

preprod.gen3.biodatacatalyst.nhlbi.nih.gov/manifest.json

Deployment changes

Breaking changes

PlanXCyborg commented Jan 27, 2023

preprod.gen3.biodatacatalyst.nhlbi.nih.gov/manifest.json

Deployment changes

Breaking changes

k-burt-uch Jan 30, 2023

Choose a reason for hiding this comment

k-burt-uch Jan 30, 2023

Choose a reason for hiding this comment