Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add autoRIFT jobs to ARIA deployments for Solid Earth applications #2344

Merged
merged 4 commits into from
Jul 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/deploy-enterprise-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ jobs:
default_application_status: APPROVED
cost_profile: DEFAULT
job_files: >-
job_spec/ARIA_AUTORIFT.yml
job_spec/ARIA_RAIDER.yml
job_spec/INSAR_ISCE.yml
instance_types: m6id.xlarge,m6id.2xlarge,m6id.4xlarge,m6id.8xlarge,m6idn.xlarge,m6idn.2xlarge,m6idn.4xlarge,m6idn.8xlarge
Expand Down
7 changes: 6 additions & 1 deletion .github/workflows/deploy-enterprise.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ jobs:
default_application_status: APPROVED
cost_profile: DEFAULT
job_files: >-
job_spec/ARIA_AUTORIFT.yml
job_spec/ARIA_RAIDER.yml
job_spec/INSAR_ISCE.yml
instance_types: m6id.xlarge,m6id.2xlarge,m6id.4xlarge,m6id.8xlarge,m6idn.xlarge,m6idn.2xlarge,m6idn.4xlarge,m6idn.8xlarge
Expand All @@ -61,6 +62,7 @@ jobs:
default_application_status: APPROVED
cost_profile: DEFAULT
job_files: >-
job_spec/ARIA_AUTORIFT.yml
job_spec/ARIA_RAIDER.yml
job_spec/INSAR_ISCE.yml
instance_types: m6id.xlarge,m6id.2xlarge,m6id.4xlarge,m6id.8xlarge,m6idn.xlarge,m6idn.2xlarge,m6idn.4xlarge,m6idn.8xlarge
Expand All @@ -79,7 +81,10 @@ jobs:
default_credits_per_user: 0
default_application_status: APPROVED
cost_profile: DEFAULT
job_files: job_spec/INSAR_ISCE.yml
job_files: >-
job_spec/ARIA_AUTORIFT.yml
job_spec/ARIA_RAIDER.yml
job_spec/INSAR_ISCE.yml
instance_types: m6id.xlarge,m6id.2xlarge,m6id.4xlarge,m6id.8xlarge,m6idn.xlarge,m6idn.2xlarge,m6idn.4xlarge,m6idn.8xlarge
default_max_vcpus: 1600 # Max 1652
expanded_max_vcpus: 1600 # Max 1652
Expand Down
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [7.7.0]

### Added
- `ARIA_AUTORIFT.yml` job spec for Solid Earth offset tracking in the ARIA JPL deployments

### Changed
- Increased throughput for `hyp3-a19-jpl` (0 -> 4,000 vCPUs) to support continued processing of ARIA GUNW products.
- The `hyp3-a19-jpl` and `hyp3-nisar-jpl` deployments now use the `m6id[n]` instance families to reduce the high number of spot interruptions seen with wth `c6id` instance family.
Expand Down
65 changes: 65 additions & 0 deletions job_spec/ARIA_AUTORIFT.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
AUTORIFT:
required_parameters:
- granules
parameters:
granules:
default: '""'
api_schema:
type: array
minItems: 2
maxItems: 2
example:
- S2B_MSIL1C_20200105T152259_N0208_R039_T13CES_20200105T181230
- S2B_MSIL1C_20200315T152259_N0209_R039_T13CES_20200315T181115
items:
anyOf:
- description: The name of the Sentinel-1 SLC granule to process
type: string
pattern: "^S1[AB]_IW_SLC__1S[SD][VH]"
minLength: 67
maxLength: 67
example: S1A_IW_SLC__1SSV_20150621T120220_20150621T120232_006471_008934_72D8
- description: The name of the Sentinel-2 granule to process (ESA naming convention)
type: string
pattern: "^S2[AB]_MSIL1C_"
minLength: 60
maxLength: 60
example: S2A_MSIL1C_20200627T150921_N0209_R025_T22WEB_20200627T170912
- description: The name of the Landsat 4, 5, 7, 8 or 9 Collection 2 granule to process
type: string
pattern: "^L([CO]0[89]|E07|T0[45])_L1"
minLength: 40
maxLength: 40
example: LC08_L1GT_118112_20210107_20210107_02_T2
bucket_prefix:
default: '""'
parameter_file:
api_schema:
description: Shapefile for determining the correct search parameters by geographic location. Path to shapefile must be understood by GDAL.
type: string
default: '/vsicurl/http://its-live-data.s3.amazonaws.com/autorift_parameters/v001/autorift_solidearth_0120m.shp'
cost_profiles:
DEFAULT:
cost: 1.0
validators: []
tasks:
- name: ''
image: ghcr.io/asfhyp3/hyp3-autorift
command:
- --bucket
- '!Ref Bucket'
- --bucket-prefix
- Ref::bucket_prefix
- --parameter-file
- Ref::parameter_file
- --naming-scheme
- ITS_LIVE_OD
- Ref::granules
timeout: 10800
vcpu: 1
memory: 31500
secrets:
- EARTHDATA_USERNAME
- EARTHDATA_PASSWORD
- ESA_USERNAME
- ESA_PASSWORD