Skip to content

Commit

Permalink
Speed up netCDF4, h5netcdf backends (#9067)
Browse files Browse the repository at this point in the history
* Speed up netCDF4 backend.

xref #9058

Accessing `.shape` on a netCDF4 variable is ~20-40ms.
This can add up for large numbers of variables, e.g.:
#9058

We already request the shape when creating NetCDF4ArrayWrapper,
so we can reuse that.

* Update h5netcdf backend too
  • Loading branch information
dcherian authored Jun 4, 2024
1 parent 1f3bc7e commit 447e5a3
Show file tree
Hide file tree
Showing 3 changed files with 8 additions and 2 deletions.
6 changes: 6 additions & 0 deletions doc/whats-new.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,12 @@ v2024.05.1 (unreleased)
New Features
~~~~~~~~~~~~

Performance
~~~~~~~~~~~

- Small optimization to the netCDF4 and h5netcdf backends (:issue:`9058`, :pull:`9067`).
By `Deepak Cherian <https://github.com/dcherian>`_.


Breaking changes
~~~~~~~~~~~~~~~~
Expand Down
2 changes: 1 addition & 1 deletion xarray/backends/h5netcdf_.py
Original file line number Diff line number Diff line change
Expand Up @@ -221,7 +221,7 @@ def open_store_variable(self, name, var):

# save source so __repr__ can detect if it's local or not
encoding["source"] = self._filename
encoding["original_shape"] = var.shape
encoding["original_shape"] = data.shape

vlen_dtype = h5py.check_dtype(vlen=var.dtype)
if vlen_dtype is str:
Expand Down
2 changes: 1 addition & 1 deletion xarray/backends/netCDF4_.py
Original file line number Diff line number Diff line change
Expand Up @@ -454,7 +454,7 @@ def open_store_variable(self, name: str, var):
pop_to(attributes, encoding, "least_significant_digit")
# save source so __repr__ can detect if it's local or not
encoding["source"] = self._filename
encoding["original_shape"] = var.shape
encoding["original_shape"] = data.shape

return Variable(dimensions, data, attributes, encoding)

Expand Down

0 comments on commit 447e5a3

Please sign in to comment.