release 2020.03 to portal.occ-data.org #1246

haraprasadj · 2020-03-18T15:17:47Z

No description provided.

…//github.com/uc-cdis/cdis-manifest into chore/release_2020.03_to_portal.occ-data.org

PlanXCyborg · 2020-03-19T15:31:56Z

portal.occ-data.org/manifest.json

Deployment changes

arborist
- Assuming any commons running Arborist < 2.3 was using user sync only as AuthZ provider (chore(authz): set authz_provider in migration arborist#111)
- One-off: Update authz_provider to the correct values (Feat/authz provider arborist#109)
- Database deployment (feat/db: add database persistence arborist#42)
- See migrations folder for database schema and setup utilities (feat/db: add database persistence arborist#42)
fence
- Fence does NOT handle c999 the same as before. If using dbgap sync w/ consent code parsing and indexd records have acls with just phsid to represent objects someone w/ c999 should be able to see, you will need to ensure that the records also have the phsid.consent_code as well or users will lose access (Feat/dbgap options fence#702)
- Fence has a new configuration to support more dbGaP syncing options: if you require "Common Exchange Area" data access, review the new configuration and supply necessary values (Feat/dbgap options fence#702)
- Fence config ENABLED_IDENTITY_PROVIDERS section deprecated and replaced by LOGIN_OPTIONS section to allow enabling several login options for the same provider (backwards compatible) (Feat/incommon updates fence#703)
- Requires manual work in Arborist DB to fill in the empty authz_provider fields (Synapse Integration fence#696)
- new cloud automation job for database migration feat(jobs): fence db create job cloud-automation#977 (feat(fence-create): new migrate command to migrate the database fence#694)
- configuration added for requester pays Google Buckets. if this is needed, you'll need to update configuration. default config does not support requester pays bucket access (Feat/requester pays fence#668)
hatchery
- New pull_policy option is available for hatchery.containers entries, eg: (feat(pods): add pull policy to containers config hatchery#1)
- { (feat(pods): add pull policy to containers config hatchery#1)
- "target-port": 8787, (feat(pods): add pull policy to containers config hatchery#1)
- "cpu-limit": "1.0", (feat(pods): add pull policy to containers config hatchery#1)
- "memory-limit": "512Mi", (feat(pods): add pull policy to containers config hatchery#1)
- "pull_policy": "Always", (feat(pods): add pull policy to containers config hatchery#1)
- "name": "R Studio", (feat(pods): add pull policy to containers config hatchery#1)
- "image": "heliumdatastage/rstudio-server:1", (feat(pods): add pull policy to containers config hatchery#1)
- "env": { (feat(pods): add pull policy to containers config hatchery#1)
- "DISABLE_AUTH": "true" (feat(pods): add pull policy to containers config hatchery#1)
- }, (feat(pods): add pull policy to containers config hatchery#1)
- "args": [], (feat(pods): add pull policy to containers config hatchery#1)
- "path-rewrite": "/", (feat(pods): add pull policy to containers config hatchery#1)
- "use-tls": "false", (feat(pods): add pull policy to containers config hatchery#1)
- "ready-probe": "/" (feat(pods): add pull policy to containers config hatchery#1)
- } (feat(pods): add pull policy to containers config hatchery#1)
peregrine
- ARBORIST_URL environment variable replaces broken default ARBORIST config variable (fix(arborist-url) peregrine#142)
- Optional configuration variable ARBORIST (arborist base URL) (Feat/arborist peregrine#138)
- Requires Arborist to be deployed (Feat/arborist peregrine#138)
pidgin
- added required configuration variable "API_HEALTH_URL" for status endpoint (Fix/_status pidgin#31)
- use the latest version of the python 3 base image (Chore/bump image pidgin#29)
- The Dockerfile now uses an Alpine image (feat(alpine-docker): use alpine image pidgin#24)
portal
- InCommon login needs fence 4.8.0 (uc-cdis/portal#595)
- Need Guppy 0.3.7+ to handle large file size (uc-cdis/portal#603)
- if deployed with "useArboristUI": true, needs Arborist's GET /auth/mapping endpoint to be exposed via revproxy. (useArboristUI false by default) (uc-cdis/portal#592)
- New optional portal config field called adminPreAppliedFilters. Goes in data/config/*.json. (uc-cdis/portal#576)
- For a "cert" if the required document is a quiz, then hasCorrectAnswers should be true, otherwise false. (uc-cdis/portal#571)
- Windmill lists unmapped files by checking Indexd. Before centralized auth's deployment, a file was determined to be unmapped by checking if the acl field was null. After centralized auth, we use the authz field instead. The new manifest configuration setting use_indexd_authz in the global block specifies which of the two fields should be used. This field defaults to false. (uc-cdis/portal#570)
- Deployed in https://qa-canine.planx-pla.net with fence version < 3, index authz field set to false (uc-cdis/portal#570)
- (by default). (uc-cdis/portal#570)
- Deployed in https://qa-brain.planx-pla.net with fence version > 3, index authz field set to true. (uc-cdis/portal#570)
- New portal configuration and images for AnVIL (uc-cdis/portal#569)
- Requires a url to be specified in dataExplorerConfig called terraExportURL (uc-cdis/portal#568)
- This should be bvdp-saturn-${env}.appspot.com/#import-data, where ${env} is dev, staging, or prod (uc-cdis/portal#568)
- Needs a version of cloud-automation that exposes arborist's auth/proxy endpoint (uc-cdis/portal#565)
- fixed DAT requires Guppy >= 0.3.1 (uc-cdis/portal#563)
- DAT requires Guppy >= 0.3.0 (uc-cdis/portal#561)
sheepdog
- Fix Travis coverage reports (combine the results of all 3 tests into one coverage report). (PXP-3408 Update Sheepdog to Python3 sheepdog#314)
- ARBORIST_URL environment variable replaces broken default ARBORIST config variable (fix(arborist-url) sheepdog#304)
- Optional configuration variable ARBORIST (arborist base URL) (Feat/arborist2 sheepdog#298)
- Requires Arborist to be deployed (Feat/arborist2 sheepdog#298)
spark
- (uc-cdis/spark#10)
wts
- use the latest version of the python 3 base image (uc-cdis/wts#14)

PlanXCyborg · 2020-03-19T16:19:01Z

portal.occ-data.org/manifest.json

Deployment changes

arborist
- Assuming any commons running Arborist < 2.3 was using user sync only as AuthZ provider (chore(authz): set authz_provider in migration arborist#111)
- One-off: Update authz_provider to the correct values (Feat/authz provider arborist#109)
- Database deployment (feat/db: add database persistence arborist#42)
- See migrations folder for database schema and setup utilities (feat/db: add database persistence arborist#42)
fence
- Fence does NOT handle c999 the same as before. If using dbgap sync w/ consent code parsing and indexd records have acls with just phsid to represent objects someone w/ c999 should be able to see, you will need to ensure that the records also have the phsid.consent_code as well or users will lose access (Feat/dbgap options fence#702)
- Fence has a new configuration to support more dbGaP syncing options: if you require "Common Exchange Area" data access, review the new configuration and supply necessary values (Feat/dbgap options fence#702)
- Fence config ENABLED_IDENTITY_PROVIDERS section deprecated and replaced by LOGIN_OPTIONS section to allow enabling several login options for the same provider (backwards compatible) (Feat/incommon updates fence#703)
- Requires manual work in Arborist DB to fill in the empty authz_provider fields (Synapse Integration fence#696)
- new cloud automation job for database migration feat(jobs): fence db create job cloud-automation#977 (feat(fence-create): new migrate command to migrate the database fence#694)
- configuration added for requester pays Google Buckets. if this is needed, you'll need to update configuration. default config does not support requester pays bucket access (Feat/requester pays fence#668)
hatchery
- New pull_policy option is available for hatchery.containers entries, eg: (feat(pods): add pull policy to containers config hatchery#1)
- { (feat(pods): add pull policy to containers config hatchery#1)
- "target-port": 8787, (feat(pods): add pull policy to containers config hatchery#1)
- "cpu-limit": "1.0", (feat(pods): add pull policy to containers config hatchery#1)
- "memory-limit": "512Mi", (feat(pods): add pull policy to containers config hatchery#1)
- "pull_policy": "Always", (feat(pods): add pull policy to containers config hatchery#1)
- "name": "R Studio", (feat(pods): add pull policy to containers config hatchery#1)
- "image": "heliumdatastage/rstudio-server:1", (feat(pods): add pull policy to containers config hatchery#1)
- "env": { (feat(pods): add pull policy to containers config hatchery#1)
- "DISABLE_AUTH": "true" (feat(pods): add pull policy to containers config hatchery#1)
- }, (feat(pods): add pull policy to containers config hatchery#1)
- "args": [], (feat(pods): add pull policy to containers config hatchery#1)
- "path-rewrite": "/", (feat(pods): add pull policy to containers config hatchery#1)
- "use-tls": "false", (feat(pods): add pull policy to containers config hatchery#1)
- "ready-probe": "/" (feat(pods): add pull policy to containers config hatchery#1)
- } (feat(pods): add pull policy to containers config hatchery#1)
peregrine
- ARBORIST_URL environment variable replaces broken default ARBORIST config variable (fix(arborist-url) peregrine#142)
- Optional configuration variable ARBORIST (arborist base URL) (Feat/arborist peregrine#138)
- Requires Arborist to be deployed (Feat/arborist peregrine#138)
pidgin
- added required configuration variable "API_HEALTH_URL" for status endpoint (Fix/_status pidgin#31)
- use the latest version of the python 3 base image (Chore/bump image pidgin#29)
- The Dockerfile now uses an Alpine image (feat(alpine-docker): use alpine image pidgin#24)
portal
- InCommon login needs fence 4.8.0 (uc-cdis/portal#595)
- Need Guppy 0.3.7+ to handle large file size (uc-cdis/portal#603)
- if deployed with "useArboristUI": true, needs Arborist's GET /auth/mapping endpoint to be exposed via revproxy. (useArboristUI false by default) (uc-cdis/portal#592)
- New optional portal config field called adminPreAppliedFilters. Goes in data/config/*.json. (uc-cdis/portal#576)
- For a "cert" if the required document is a quiz, then hasCorrectAnswers should be true, otherwise false. (uc-cdis/portal#571)
- Windmill lists unmapped files by checking Indexd. Before centralized auth's deployment, a file was determined to be unmapped by checking if the acl field was null. After centralized auth, we use the authz field instead. The new manifest configuration setting use_indexd_authz in the global block specifies which of the two fields should be used. This field defaults to false. (uc-cdis/portal#570)
- Deployed in https://qa-canine.planx-pla.net with fence version < 3, index authz field set to false (uc-cdis/portal#570)
- (by default). (uc-cdis/portal#570)
- Deployed in https://qa-brain.planx-pla.net with fence version > 3, index authz field set to true. (uc-cdis/portal#570)
- New portal configuration and images for AnVIL (uc-cdis/portal#569)
- Requires a url to be specified in dataExplorerConfig called terraExportURL (uc-cdis/portal#568)
- This should be bvdp-saturn-${env}.appspot.com/#import-data, where ${env} is dev, staging, or prod (uc-cdis/portal#568)
- Needs a version of cloud-automation that exposes arborist's auth/proxy endpoint (uc-cdis/portal#565)
- fixed DAT requires Guppy >= 0.3.1 (uc-cdis/portal#563)
- DAT requires Guppy >= 0.3.0 (uc-cdis/portal#561)
sheepdog
- Fix Travis coverage reports (combine the results of all 3 tests into one coverage report). (PXP-3408 Update Sheepdog to Python3 sheepdog#314)
- ARBORIST_URL environment variable replaces broken default ARBORIST config variable (fix(arborist-url) sheepdog#304)
- Optional configuration variable ARBORIST (arborist base URL) (Feat/arborist2 sheepdog#298)
- Requires Arborist to be deployed (Feat/arborist2 sheepdog#298)
spark
- (uc-cdis/spark#10)
wts
- use the latest version of the python 3 base image (uc-cdis/wts#14)

release 2020.03 to portal.occ-data.org

03e6df7

uc-cdis deleted a comment from PlanXCyborg Mar 18, 2020

haraprasadj and others added 5 commits March 18, 2020 11:57

release 2020.03 to portal.occ-data.org

6226ee4

adding sheepdog version that precedes the march release

3141bf6

adding fence version that precedes the march release

370f5b5

adding arborist version that precedes the march release

83a47b5

removing indexd block and canary block

8f7c9d4

uc-cdis deleted a comment from PlanXCyborg Mar 19, 2020

haraprasadj added 2 commits March 19, 2020 10:22

testing with 2020.02 and removing nb2

e495688

Merge branch 'chore/release_2020.03_to_portal.occ-data.org' of https:…

1dd0a5d

…//github.com/uc-cdis/cdis-manifest into chore/release_2020.03_to_portal.occ-data.org

uc-cdis deleted a comment from PlanXCyborg Mar 19, 2020

haraprasadj added 2 commits March 19, 2020 10:47

revproxy changed to 1.15.5-ctds

264e386

added canary block back

5948623

haraprasadj closed this Mar 20, 2020

haraprasadj deleted the chore/release_2020.03_to_portal.occ-data.org branch March 20, 2020 14:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

release 2020.03 to portal.occ-data.org #1246

release 2020.03 to portal.occ-data.org #1246

haraprasadj commented Mar 18, 2020

PlanXCyborg commented Mar 19, 2020

PlanXCyborg commented Mar 19, 2020

release 2020.03 to portal.occ-data.org #1246

release 2020.03 to portal.occ-data.org #1246

Conversation

haraprasadj commented Mar 18, 2020

PlanXCyborg commented Mar 19, 2020

portal.occ-data.org/manifest.json

Deployment changes

PlanXCyborg commented Mar 19, 2020

portal.occ-data.org/manifest.json

Deployment changes