Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handling of static variables. #217

Open
AndyHoggANU opened this issue Oct 14, 2024 · 3 comments · May be fixed by #212
Open

Handling of static variables. #217

AndyHoggANU opened this issue Oct 14, 2024 · 3 comments · May be fixed by #212
Labels
enhancement New feature or request

Comments

@AndyHoggANU
Copy link

Is your feature request related to a problem? Please describe.

Many of our models write out static information (e.g. grid information) with every segment of the run. This confuses the intake catalogue. For example, if we do this:

import intake
catalog = intake.cat.access_nri
experiment='1deg_jra55_iaf_omip2_cycle1'
cat_subset = catalog.search(name = experiment)
dxt = cat_subset[experiment].search(variable = 'dxt',frequency = 'fx').to_dask()['dxt']

then we get an error -- because there are many static files which meet the criteria, but they can't be stitched together. The suggestion on the intake tutorial is to use:

dxt = cat_subset[experiment].search(variable = 'dxt',frequency = 'fx',path=".*output000.*").to_dask()['dxt']

which is fine when you know something about the path name. But we would quite like the user to be oblivious to file names. For example, if we do this:

experiment='1deg_jra55_iaf_omip2_cycle2'
cat_subset = catalog.search(name = experiment)
dxt = cat_subset[experiment].search(variable = 'dxt',frequency = 'fx',path=".*output000.*").to_dask()['dxt']

then we also get a failure ...

Describe the feature you'd like

For fx variables, maybe we could make intake read only one file in?

Describe alternatives you've considered

Sorry, no other real ideas....

@AndyHoggANU AndyHoggANU added the enhancement New feature or request label Oct 14, 2024
@marc-white
Copy link
Collaborator

I think this dovetails very nicely with what @charles-turner-1 is looking at on #212 , which will hopefully lead us to returning something sensible when you make a query like your first one @AndyHoggANU .

@charles-turner-1
Copy link
Contributor

Yup, the work in #212 should address this issue.

@AndyHoggANU
Copy link
Author

OK, great. Feel free to close this one if you want to tidy up?

@charles-turner-1 charles-turner-1 linked a pull request Oct 15, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants