Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Dependency]: Docker container for CMR search #12

Closed
LucaCinquini opened this issue Apr 16, 2023 · 5 comments
Closed

[Dependency]: Docker container for CMR search #12

LucaCinquini opened this issue Apr 16, 2023 · 5 comments
Assignees
Labels
dependency Ticket designating a dependency enhancement New feature or request U-DS
Milestone

Comments

@LucaCinquini
Copy link
Collaborator

Who: From U-DS to U-SPS
When: April 2023
What: Provide a Docker container for CMR search

@LucaCinquini LucaCinquini added the enhancement New feature or request label Apr 16, 2023
@LucaCinquini LucaCinquini added this to the 23.2 milestone Apr 16, 2023
@mike-gangl mike-gangl added U-DS dependency Ticket designating a dependency labels Apr 20, 2023
@LucaCinquini
Copy link
Collaborator Author

See PR: unity-sds/sounder-sips-chirp-workflows#3
Tested successfully by Luca.
@ngachung : could you provide an example query that will return a 16 days CHIRP dataset, which we can use to execute the e2e CHIRP workflow?

@LucaCinquini LucaCinquini moved this to In Progress in Unity Project Board Apr 22, 2023
@ngachung
Copy link

@LucaCinquini Per feedback from Alex, this range will return all 3,818 files needed for e2e CHIRP.
cmr_start_time: "2016-08-22T00:10:00Z"
cmr_stop_time: "2016-09-06T23:59:59Z"

Each day should have 240 files, 16 x 240 = 3,840 but for whatever reason 8/25/18 only has 218 files so 15 X 240 + 218 = 3,818

See https://cmr.earthdata.nasa.gov/virtual-directory/collections/C2011289787-GES_DISC/temporal/2016/08 for file count per day

@LucaCinquini
Copy link
Collaborator Author

LucaCinquini commented Apr 24, 2023

Thanks @ngachung - and which collection name should I use? C2011289787-GES_DISC?

@ngachung
Copy link

cmr_collection: "C2011289787-GES_DISC"
cmr_start_time: "2016-08-22T00:10:00Z"
cmr_stop_time: "2016-09-06T23:59:59Z"
limits: "-1"

@LucaCinquini
Copy link
Collaborator Author

Thank Nga, I tried the last example and it worked. I am a little worried though about the ability of a single job to handle 3818 files concurrently. It seems like the execution should be parallelized.

@github-project-automation github-project-automation bot moved this from In Progress to Done in Unity Project Board Apr 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependency Ticket designating a dependency enhancement New feature or request U-DS
Projects
Status: Done
Development

No branches or pull requests

3 participants