-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Deprecate stac-populator
optional component
#482
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure what you mean by it not working.
It runs each time I reboot the STAC components.
pavics-compose logs -f --tail 100 stac-populator
[...]
stac-populator | [INFO] STAC item [65e1ae2daea7231392c89b320c854562] already exists on [http://stac:8000/stac/collections/c604ffb6d610adbb9a6b4787db7b8fd7] (409), updating..
stac-populator | {
stac-populator | "id": "614d9747c2a022a5200565e6904a2ff6",
stac-populator | "body": {
stac-populator | "type": "asset",
stac-populator | "properties": {
stac-populator | "uri": "https://pavics.ouranos.ca/twitcher/ows/proxy/thredds/dodsC/birdhouse/disk2/cccs_portal/indices/Final/BCCAQv2_CMIP6/txgt_29/MS/ssp585/simulations/txgt_29_mon_BCCAQ2v2+ANUSPLIN300_FGOALS-g3_historical+ssp585_1950-2100.nc",
stac-populator | "filename": "txgt_29_mon_BCCAQ2v2+ANUSPLIN300_FGOALS-g3_historical+ssp585_1950-2100.nc",
stac-populator | "extension": ".nc",
stac-populator | "dataset_name": "BCCAQv2_CMIP6",
stac-populator | "variable_id": "txgt_29",
stac-populator | "freq": "MS",
stac-populator | "scenario": "ssp585",
stac-populator | "collection_id": "CMIP6"
stac-populator | },
stac-populator | "categories": [
stac-populator | "data"
stac-populator | ],
stac-populator | "item_id": "eda9db16602332a26d6bf6df77387a46"
stac-populator | }
stac-populator | }
stac-populator | [INFO] STAC item [eda9db16602332a26d6bf6df77387a46] already exists on [http://stac:8000/stac/collections/c604ffb6d610adbb9a6b4787db7b8fd7] (409), updating..
stac-populator | 2024-11-27 03:35:02,924 stac_generator.plugins.extraction_methods.regex INFO Found matches for regex extract
stac-populator | 2024-11-27 03:35:02,925 stac_generator.plugins.extraction_methods.header.header WARNING Header extract backend not found for https://pavics.ouranos.ca/twitcher/ows/proxy/thredds/dodsC/birdhouse/disk2/cccs_portal/indices/Final/BCCAQv2/txgt_32/YS/rcp45/simulations_spatialAvgCensus/BCCAQv2+ANUSPLIN300_MPI-ESM-MR_historical+rcp45_r1i1p1_1950-2100_txgt_32_YS_spatialAvgCensus.nc
stac-populator | 2024-11-27 03:35:02,948 stac_generator.plugins.extraction_methods.object_store_stats INFO Extracting metadata for: https://pavics.ouranos.ca/twitcher/ows/proxy/thredds/dodsC/birdhouse/disk2/cccs_portal/indices/Final/BCCAQv2_CMIP6/txgt_29/MS/ssp585/simulations/txgt_29_mon_BCCAQ2v2+ANUSPLIN300_EC-Earth3-Veg_historical+ssp585_1950-2100.nc with checksum: None
stac-populator | 2024-11-27 03:35:02,980 stac_generator.plugins.extraction_methods.object_store_stats INFO Extracting metadata for: https://pavics.ouranos.ca/twitcher/ows/proxy/thredds/dodsC/birdhouse/disk2/cccs_portal/indices/Final/BCCAQv2/txgt_32/YS/rcp45/simulations_spatialAvgCensus/BCCAQv2+ANUSPLIN300_MPI-ESM-LR_historical+rcp45_r1i1p1_1950-2100_txgt_32_YS_spatialAvgCensus.nc with checksum: None
stac-populator | [INFO] STAC collection [c604ffb6d610adbb9a6b4787db7b8fd7] already exists on [http://stac:8000/stac] (409), updating..
stac-populator | [INFO] STAC collection [0798aa197d54eb4332767a5a4077fb0f] already exists on [http://stac:8000/stac] (409), updating..
stac-populator | STAC asset generator ran for 200 seconds. Exiting.
stac-populator exited with code 0
Collections are properly validated/updated against existing/added files:
https://hirondelle.crim.ca/stac/collections/c604ffb6d610adbb9a6b4787db7b8fd7
I agree it needs some updates to make it work with latest changes in the repo (CLI refactor and all), but the component is not broken in its current state with the way it's defined.
You're using an old version of the image that is cached locally on your machine. The image is tagged with docker image rm ghcr.io/crim-ca/stac-populator:master
docker run ghcr.io/crim-ca/stac-populator:master ./populate.sh # this exits with the message "./populate.sh: no such file or directory" |
Tag |
What happens when you try this?
|
Oh wow, it is indeed a different version. |
Here it is: https://github.com/crim-ca/stac-populator/pkgs/container/stac-populator/98905655?tag=0.0.0 |
## Overview This sets the docker image for the `stac-populator` component to a version that actually contains the code that is executed when `stac-populator` is called. The previous image no longer contained the relevant code. ## Changes **Non-breaking changes** - stac-populator image changed to version 0.0.0 **Breaking changes** None ## Related Issue / Discussion - Closes #482 ## Additional Information ## CI Operations <!-- The test suite can be run using a different DACCS config with ``birdhouse_daccs_configs_branch: branch_name`` in the PR description. To globally skip the test suite regardless of the commit message use ``birdhouse_skip_ci`` set to ``true`` in the PR description. Using ``[<cmd>]`` (with the brackets) where ``<cmd> = skip ci`` in the commit message will override ``birdhouse_skip_ci`` from the PR description. Such commit command can be used to override the PR description behavior for a specific commit update. However, a commit message cannot 'force run' a PR which the description turns off the CI. To run the CI, the PR should instead be updated with a ``true`` value, and a running message can be posted in following PR comments to trigger tests once again. --> birdhouse_daccs_configs_branch: master birdhouse_skip_ci: false
Overview
This component has not been functional since it was deprecated in crim-ca/stac-populator#12
Changes
Non-breaking changes
Breaking changes
None
Related Issue / Discussion
Additional Information
Links to other issues or sources.
CI Operations
birdhouse_daccs_configs_branch: master
birdhouse_skip_ci: false