forked from venetoarpa/Arpav-PPCV-backend
-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding bootstrap configs for collection and new historical variables #219
Merged
francbartoli
merged 18 commits into
geobeyond:main
from
ricardogsilva:209-Add-configuration-for-the-new-historical-netCDF-datasets-
Sep 17, 2024
Merged
Adding bootstrap configs for collection and new historical variables #219
francbartoli
merged 18 commits into
geobeyond:main
from
ricardogsilva:209-Add-configuration-for-the-new-historical-netCDF-datasets-
Sep 17, 2024
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ricardogsilva
force-pushed
the
209-Add-configuration-for-the-new-historical-netCDF-datasets-
branch
2 times, most recently
from
September 6, 2024 17:20
4e7cd77
to
37ec59b
Compare
… the name of the netcdf main variable
ricardogsilva
force-pushed
the
209-Add-configuration-for-the-new-historical-netCDF-datasets-
branch
from
September 10, 2024 15:52
37ec59b
to
d2c1516
Compare
…d to configure them
…h dev and staging compose stacks
francbartoli
approved these changes
Sep 17, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
NOTE
For consistency, this PR includes a change in the name of the configuration parameters used by the forecast datasets to indicate their respective year period. Changes are:
DJF
->winter
MAM
->spring
JJA
->august
SON
->autumn
year
->all_year
This PR adds the relevant bootstrap configurations for the new historical datasets:
tdd
tnd
txd
tr
fd
cdds
hdds
su30
prcptot
These are included in their respective 30-year, and annual series. The 10-year series are not included here, as they are still being modified by the client.
In order to support these datasets, the following new configuration parameters are also added:
archive
, which can be eitherhistorical
orforecast
and helps distinguish the two types of coverages that are now supported by the systemhistorical_variable
, which contains entries for the new variableshistorical_year_period
, which features the year periods recognized by the new datasets (A00
,S01
,S02
, ...)climatological_standard_normal
, which contains the relvant climate normal year intervals (1991-2020
, etc.)Additionally, in order to support the new ways by which the underlying NetCDF datasets are structured, this PR adds some new features:
internal_value
property, which is used when building THREDDS and WMS identifiers. This allows decoupling a parameter's name, which is what is used in the identifiers that are visible to the outside of the system, from their corresponding values, which are used in internal components. This was notoriously needed also for referring to the pattern used in historical NetCDF datasets for identifying the climate normal, which is a string of the form"yyyy-yyyy"
- the problem here was that the system was relying on conf parameter names not being allowed to have the-
character. With the introduction of a param'sinternal_value
this is no longer a problemwms_main_layer_name
can now be a templated string, allowing for the underlying NetCDf variable names to vary according with the configuration parameters - example:{observation_year_period}_avg
thredds_url_pattern
can now be defined with an fnmatch pattern, thus allowing NetCDF names to vary - in this case the system will perform an additional request to the THREDDS server in order to scan the catalog in order to find the actual name of the file, caching the results