Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update for new CDS API #68

Merged
merged 7 commits into from
Sep 27, 2024
Merged

Conversation

pixalytics
Copy link
Contributor

@bgruening Sorry, but I've got another pull request. Yesterday, ECMWF released the new version of the API https://confluence.ecmwf.int/display/CKB/Please+read%3A+CDS+and+ADS+migrating+to+new+infrastructure%3A+Common+Data+Store+%28CDS%29+Engine so older versions of the CDS API call don't work.

I've updated the tool to comply with the new API, primarily up versioning the Python libraries, but also a slight change in how the data is called. I've updated the documentation.

The current test doesn't actually test if data is downloaded, just that the API call text is as expected. So, I'm not sure if this needs an improvement.

Copy link
Collaborator

@bgruening bgruening left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Outch, they break API? This is not good :(

Thanks for fixing it that quickly!

@@ -28,7 +28,7 @@
<configfiles>
<configfile name="cds_key_file"><![CDATA[
#set $c3s_cds_apikey = $__user__.extra_preferences.get('c3s_account|c3s_cds_apikey', "")
url: https://cds.climate.copernicus.eu/api/v2
url: https://cds.climate.copernicus.eu/api
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would assume that they keep the versioned APIs?

Is there a v3 now that we can use?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a new CDS with a backwards-compatible API. In their guidance for usage they have dropped the v2

@@ -100,7 +121,7 @@ Neither the European Commission nor ECMWF is responsible for any use
that may be made of the Copernicus information or data it contains.
]]></help>
<citations>

<citation type="text">Copernicus Climate Change Service (2024)</citation>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only have type bibtex and type doi ...

When you use bibtex you can do some freetext if you like. https://github.com/bgruening/galaxytools/blob/master/tools/whisper/whisper.xml#L349

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bgruening Thanks, I've used that approach to encode the text

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot!

@annefou
Copy link

annefou commented Sep 27, 2024

@pixalytics Thanks a lot for fixing this tool!

it was written a while ago and does not follow the recommended practices for EDAM annotations. I think that you need to move them for instance after the description of the tool.

Comment on lines 86 to 115
This tool is a wrapper to retrieve data from the Copernicus Climate Data Store.

- It allows to retrieve data from the Copernicus climate Data Store.
- Any user willing to use this tool needs to `create a new account <https://cds.climate.copernicus.eu/user/register?destination=%2F%23!%2Fhome>`_.
- Any user willing to use this tool needs to `create a new account <https://cds.climate.copernicus.eu/>`_.
- Set your CDS API Key via: User -> Preferences -> Manage Information"
- Documentation on where to get the CDS API key can be found `here <https://cds.climate.copernicus.eu/api-how-to>`_.
- Compose your request directly on C3S and copy/paste it in the input field "Request" or save it in a file.
- Be aware that for being able to download dataset from C3S, users also need to agree to their term of use (Licence to use Copernicus Products) on the C3S website.
- Documentation on where to get the CDS API key can be found `here <https://cds.climate.copernicus.eu/how-to-api>`_.
- Compose your request directly on C3S and extract the relevant information, which should be put in the input field "Request" or saved in a file. The format should be, for example:

import cdsapi

c = cdsapi.Client()

c.retrieve(
'reanalysis-era5-single-levels-monthly-means',
{
'data_format': 'netcdf',
'product_type': 'monthly_averaged_reanalysis',
'variable': '2m_temperature',
'year': '2020',
'month': '12',
'time': '00:00',
'area': [60, 10, 59.5, 10.5],
},
'download.nc')

- Be aware that for being able to download dataset from C3S, users also need to agree to licensing terms for each dataset of interest on the C3S website.

License:
~~~~~~~~

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This tool is a wrapper to retrieve data from the Copernicus Climate Data Store.
- It allows to retrieve data from the Copernicus climate Data Store.
- Any user willing to use this tool needs to `create a new account <https://cds.climate.copernicus.eu/user/register?destination=%2F%23!%2Fhome>`_.
- Any user willing to use this tool needs to `create a new account <https://cds.climate.copernicus.eu/>`_.
- Set your CDS API Key via: User -> Preferences -> Manage Information"
- Documentation on where to get the CDS API key can be found `here <https://cds.climate.copernicus.eu/api-how-to>`_.
- Compose your request directly on C3S and copy/paste it in the input field "Request" or save it in a file.
- Be aware that for being able to download dataset from C3S, users also need to agree to their term of use (Licence to use Copernicus Products) on the C3S website.
- Documentation on where to get the CDS API key can be found `here <https://cds.climate.copernicus.eu/how-to-api>`_.
- Compose your request directly on C3S and extract the relevant information, which should be put in the input field "Request" or saved in a file. The format should be, for example:
import cdsapi
c = cdsapi.Client()
c.retrieve(
'reanalysis-era5-single-levels-monthly-means',
{
'data_format': 'netcdf',
'product_type': 'monthly_averaged_reanalysis',
'variable': '2m_temperature',
'year': '2020',
'month': '12',
'time': '00:00',
'area': [60, 10, 59.5, 10.5],
},
'download.nc')
- Be aware that for being able to download dataset from C3S, users also need to agree to licensing terms for each dataset of interest on the C3S website.
License:
~~~~~~~~
This tool is a wrapper to retrieve data from the Copernicus Climate Data Store.
- It allows to retrieve data from the Copernicus climate Data Store.
- Any user willing to use this tool needs to `create a new account <https://cds.climate.copernicus.eu/>`_.
- Set your CDS API Key via: User -> Preferences -> Manage Information"
- Documentation on where to get the CDS API key can be found `here <https://cds.climate.copernicus.eu/how-to-api>`_.
- Compose your request directly on C3S and extract the relevant information, which should be put in the input field "Request" or saved in a file. The format should be, for example::
import cdsapi
c = cdsapi.Client()
c.retrieve(
'reanalysis-era5-single-levels-monthly-means',
{
'data_format': 'netcdf',
'product_type': 'monthly_averaged_reanalysis',
'variable': '2m_temperature',
'year': '2020',
'month': '12',
'time': '00:00',
'area': [60, 10, 59.5, 10.5],
},
'download.nc')
- Be aware that for being able to download dataset from C3S, users also need to agree to licensing terms for each dataset of interest on the C3S website.
License:
~~~~~~~~

This should hopefully fix the RST warning.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I was trying to work out how to get the formatting right...

@bgruening
Copy link
Collaborator

@pixalytics sorry I still can not push to your branch. Can you maybe provide commit access to your fork. I can then handle the lining issues. They are not your fault, our listing just got stricter since the tool was last modified and I would take the opportunity to fix the linting.

@bgruening bgruening merged commit e8074c9 into NordicESMhub:master Sep 27, 2024
10 checks passed
@bgruening
Copy link
Collaborator

Thanks a lot. This version will be installed over the weekend. If you need it earlier, please let me know.

@bgruening bgruening deleted the c3s-update branch September 27, 2024 19:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants