Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DataArray throws error in contains_cftime_datetimes() -> _data vs data (with fix?) #7966

Closed
4 tasks done
cyschneck opened this issue Jul 9, 2023 · 5 comments
Closed
4 tasks done
Labels
bug needs triage Issue that has not been reviewed by xarray team member

Comments

@cyschneck
Copy link

cyschneck commented Jul 9, 2023

What happened?

When attempting to run a GeoCats tutorial, the <class 'xarray.core.dataarray.DataArray'> throws the error AttributeError: 'DataArray' object has no attribute '_data'

AttributeError                            Traceback (most recent call last)
Cell In[6], line 1
----> 1 monthly_temp = climatology_average(temp, freq='month')

File ~/miniconda3/envs/geocat-tutorials/lib/python3.10/site-packages/geocat/comp/climatologies.py:511, in climatology_average(dset, freq, time_dim, keep_attrs)
    508 format, frequency = freq_dict[key]
    510 # If time_dim is None, infer time dimension name. Confirm dset[time_dim] contain datetimes
--> 511 time_dim = _get_time_coordinate_info(dset, time_dim)
    513 # Check if data is uniformly spaced
    514 if xr.infer_freq(dset[time_dim]) is None:

File ~/miniconda3/envs/geocat-tutorials/lib/python3.10/site-packages/geocat/comp/climatologies.py:32, in _get_time_coordinate_info(dset, time_coord_name)
     29     time = dset.cf["time"]
     30     time_coord_name = time.name
---> 32 if not _contains_datetime_like_objects(time):
     33     raise ValueError(
     34         f"The {time_coord_name} coordinate should be either `np.datetime64` or `cftime.datetime`"
     35     )
     37 return time_coord_name

File ~/miniconda3/envs/geocat-tutorials/lib/python3.10/site-packages/geocat/comp/climatologies.py:15, in _contains_datetime_like_objects(d_arr)
     10 def _contains_datetime_like_objects(d_arr):
     11     """Check if a variable contains datetime like objects (either
     12     np.datetime64, or cftime.datetime)"""
     13     return np.issubdtype(
     14         d_arr.dtype,
---> 15         np.datetime64) or xr.core.common.contains_cftime_datetimes(d_arr)

File ~/miniconda3/envs/geocat-tutorials/lib/python3.10/site-packages/xarray/core/common.py:2013, in contains_cftime_datetimes(var)
   2011 def contains_cftime_datetimes(var: T_Variable) -> bool:
   2012     """Check if an xarray.Variable contains cftime.datetime objects"""
-> 2013     return _contains_cftime_datetimes(var._data)

File ~/miniconda3/envs/geocat-tutorials/lib/python3.10/site-packages/xarray/core/common.py:278, in AttrAccessMixin.__getattr__(self, name)
    276         with suppress(KeyError):
    277             return source[name]
--> 278 raise AttributeError(
    279     f"{type(self).__name__!r} object has no attribute {name!r}"
    280 )

AttributeError: 'DataArray' object has no attribute '_data'

Is there a reason that DataArray would not contain _data attribute?

What did you expect to happen?

DataArray appears not to have the attribute _data. The class DataArray appears to be expecting data but by passing the contains_cftime_datetimes function a DataArray, it will attempt to call the DataArray._data and thrown an error since that class does not have a _data attribute.

Does contains_cftime_datetimes not accept DataArrays?

Working on a separate branch on this function, by replacing var._data with var.data I no longer encountered any errors and was able to complete as expected (common.py at Line 2013)

def contains_cftime_datetimes(var: T_Variable) -> bool:
    """Check if an xarray.Variable contains cftime.datetime objects"""
    return _contains_cftime_datetimes(var.data)

I have included a minimal example to recreate this issue, which originally arose while using GeoCat in conda

Minimal Complete Verifiable Example

from geocat.comp import climatology_average
import xarray as xr
import geocat.datafiles as gdf

nc_file = gdf.get('netcdf_files/atm.20C.hourly6-1990-1995-TS.nc')
ds = xr.open_dataset(nc_file)
ds = ds.isel(member_id=0)  # select one model from the ensemble

temp = ds.TS
monthly_temp = climatology_average(temp, freq='month')

MVCE confirmation

  • Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
  • Complete example — the example is self-contained, including all data and the text of any traceback.
  • Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
  • New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

AttributeError                            Traceback (most recent call last)
Cell In[6], line 1
----> 1 monthly_temp = climatology_average(temp, freq='month')

File ~/miniconda3/envs/geocat-tutorials/lib/python3.10/site-packages/geocat/comp/climatologies.py:511, in climatology_average(dset, freq, time_dim, keep_attrs)
    508 format, frequency = freq_dict[key]
    510 # If time_dim is None, infer time dimension name. Confirm dset[time_dim] contain datetimes
--> 511 time_dim = _get_time_coordinate_info(dset, time_dim)
    513 # Check if data is uniformly spaced
    514 if xr.infer_freq(dset[time_dim]) is None:

File ~/miniconda3/envs/geocat-tutorials/lib/python3.10/site-packages/geocat/comp/climatologies.py:32, in _get_time_coordinate_info(dset, time_coord_name)
     29     time = dset.cf["time"]
     30     time_coord_name = time.name
---> 32 if not _contains_datetime_like_objects(time):
     33     raise ValueError(
     34         f"The {time_coord_name} coordinate should be either `np.datetime64` or `cftime.datetime`"
     35     )
     37 return time_coord_name

File ~/miniconda3/envs/geocat-tutorials/lib/python3.10/site-packages/geocat/comp/climatologies.py:15, in _contains_datetime_like_objects(d_arr)
     10 def _contains_datetime_like_objects(d_arr):
     11     """Check if a variable contains datetime like objects (either
     12     np.datetime64, or cftime.datetime)"""
     13     return np.issubdtype(
     14         d_arr.dtype,
---> 15         np.datetime64) or xr.core.common.contains_cftime_datetimes(d_arr)

File ~/miniconda3/envs/geocat-tutorials/lib/python3.10/site-packages/xarray/core/common.py:2013, in contains_cftime_datetimes(var)
   2011 def contains_cftime_datetimes(var: T_Variable) -> bool:
   2012     """Check if an xarray.Variable contains cftime.datetime objects"""
-> 2013     return _contains_cftime_datetimes(var._data)

File ~/miniconda3/envs/geocat-tutorials/lib/python3.10/site-packages/xarray/core/common.py:278, in AttrAccessMixin.__getattr__(self, name)
    276         with suppress(KeyError):
    277             return source[name]
--> 278 raise AttributeError(
    279     f"{type(self).__name__!r} object has no attribute {name!r}"
    280 )

AttributeError: 'DataArray' object has no attribute '_data'


### Anything else we need to know?

_No response_

### Environment

<details>

INSTALLED VERSIONS
------------------
commit: None
python: 3.10.12 | packaged by conda-forge | (main, Jun 23 2023, 22:40:32) [GCC 12.3.0]
python-bits: 64
OS: Linux
OS-release: 4.15.0-142-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: 1.14.1
libnetcdf: 4.9.2

xarray: 2023.6.0
pandas: 2.0.3
numpy: 1.24.4
scipy: 1.11.1
netCDF4: 1.6.4
pydap: None
h5netcdf: None
h5py: None
Nio: None
zarr: None
cftime: 1.6.2
nc_time_axis: 1.4.1
PseudoNetCDF: None
iris: None
bottleneck: 1.3.7
dask: 2023.7.0
distributed: 2023.7.0
matplotlib: 3.7.2
cartopy: 0.21.1
seaborn: None
numbagg: None
fsspec: 2023.6.0
cupy: None
pint: 0.19.2
sparse: None
flox: None
numpy_groupies: None
setuptools: 68.0.0
pip: 23.1.2
conda: None
pytest: None
mypy: None
IPython: 8.14.0
sphinx: None

/home/user/miniconda3/envs/geocat-tutorials/lib/python3.10/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
</details>
@cyschneck cyschneck added bug needs triage Issue that has not been reviewed by xarray team member labels Jul 9, 2023
@welcome
Copy link

welcome bot commented Jul 9, 2023

Thanks for opening your first issue here at xarray! Be sure to follow the issue template!
If you have an idea for a solution, we would really welcome a Pull Request with proposed changes.
See the Contributing Guide for more.
It may take us a while to respond here, but we really value your contribution. Contributors like you help make xarray better.
Thank you!

@kmuehlbauer
Copy link
Contributor

Related open issue over at geocat: NCAR/geocat-comp#381

@spencerkclark
Copy link
Member

Does contains_cftime_datetimes not accept DataArrays?

Indeed as of #7494 it no longer does. Per the type hint it only accepts xarray.Variable objects, which have a _data attribute. I'm afraid I agree with @dcherian in NCAR/geocat-comp#381 (comment) on this:

This is private API, you should vendor it though the current version isn't the most friendly for vendoring. So perhaps reimplement it.

See here for more explanation of public vs. private API in xarray.

@Illviljan
Copy link
Contributor

Related issue: #7645

If you want to continue using non-public functions, change

xr.core.common.contains_cftime_datetimes(d_arr)

to this

xr.core.common.contains_cftime_datetimes(d_arr.variable)

@cyschneck
Copy link
Author

Thanks, I'll give it a look

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug needs triage Issue that has not been reviewed by xarray team member
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants