ENH: Pass on on-disk chunk sizes as preferred chunk sizes to the xarray backend

[mraspaud]

This PR proposes a removes the (unused) chunks and cache argument from the open_dataset plugin function and sends values to open_rasterio to get a result inline with what xarray expects. Moreover, the preferred_chunks encoding attribute is populated for xarray to be able to do optimized automated chunking.

The fact that chunks and cache are ignored by xarray is documented in the example here: https://docs.xarray.dev/en/stable/internals/how-to-add-new-backend.html#backendentrypoint-subclassing
And the recommended usage of preferred_chunks is documented here: https://docs.xarray.dev/en/stable/internals/how-to-add-new-backend.html#preferred-chunk-sizes

Finally, I wasn't really sure how/where to put the tests, what do you recommend?

• Closes #xxxx
• Tests added
• Fully documented, including docs/history.rst for all changes and docs/rioxarray.rst for new API

[snowman2]

chunks is not used by xarray, however it is used by rioxarray. See:

rioxarray/rioxarray/_io.py

Lines 896 to 929 in 33c0059

	def _prepare_dask(
	result: DataArray,
	riods: RasterioReader,
	filename: Union[str, os.PathLike],
	chunks: Union[int, tuple, dict],
	) -> DataArray:
	"""
	Prepare the data for dask computations
	"""
	# pylint: disable=import-outside-toplevel
	from dask.base import tokenize
	# augment the token with the file modification time
	try:
	mtime = os.path.getmtime(filename)
	except (TypeError, OSError):
	# the filename is probably an s3 bucket rather than a regular file
	mtime = None
	if chunks in (True, "auto"):
	from dask.array.core import normalize_chunks
	if not _DASK_GTE_018:
	raise NotImplementedError("Automatic chunking requires dask >= 0.18.0")
	block_shape = (1,) + riods.block_shapes[0]
	chunks = normalize_chunks(
	chunks=(1, "auto", "auto"),
	shape=(riods.count, riods.height, riods.width),
	dtype=_rasterio_to_numpy_dtype(riods.dtypes),
	previous_chunks=tuple((c,) for c in block_shape),
	)
	token = tokenize(filename, mtime, chunks)
	name_prefix = f"open_rasterio-{token}"
	return result.chunk(chunks, name_prefix=name_prefix, token=token)

[mraspaud]

chunks is not used by xarray, however it is used by rioxarray. See:

Indeed! my objective here is to return the LazilyIndexedOuterArrays to xarray directly when the data is requested through the xarray backend to let xarray deal with the chunking and create dask arrays itself.

[mraspaud]

Anything else I can do to get this PR ready to merge?

[snowman2]

Mind adding a note in docs/history.rst?

[mraspaud]

Mind adding a note in docs/history.rst?

Done in 1808857

[snowman2]

@mraspaud, the only thing left to do in this PR is to remove the changes in the .pre-commit-config.yaml file and resolve the conflicts. The optimal chunk size can be updated separately.

[mraspaud]

Done. Sorry for the delay, I was on holidays...

[snowman2]

This looks good from my perspective, Any other changes you would like to add before merging?

[mraspaud]

Nothing more from my side, thanks for all the support!

[snowman2]

Thanks @mraspaud 👍