Add drop_isel #4819

mesejo · 2021-01-17T12:08:18Z

Closes drop_sel indices in dimension that doesn't have coordinates? #4658
Use get_index(dim) in drop_sel
Tests added
Passes pre-commit run --all-files
User visible changes (including notable bug fixes) are documented in whats-new.rst
New functions/methods are listed in api.rst

- Use get_index(dim) in drop_sel - Add drop_isel

max-sixty

This is excellent! Thanks @mesejo

I added a couple of minor comments.

It would be great to do this for DataArray too — it can be a very simple wrapper of this method.

xarray/core/dataset.py

xarray/tests/test_dataset.py

mathause · 2021-01-17T21:56:28Z

LGTM!

It would be great to do this for DataArray too — it can be a very simple wrapper of this method.

Just to add on to this - the wrapper can be written as:

class DataArray:
    ...

    def drop_isel(self, indexers=None, **indexers_kwargs):
        """Drop index positions from this dataset.
        ...
        """
        dataset = self._to_temp_dataset()
        dataset = dataset.drop_isel(indexers=indexers, **indexers_kwargs)
        return self._from_temp_dataset(dataset)

    ...

mesejo · 2021-01-18T21:11:03Z

This is excellent! Thanks @mesejo

I added a couple of minor comments.

It would be great to do this for DataArray too — it can be a very simple wrapper of this method.

Thanks for the feedback. I've addressed the issues.

max-sixty

This is great, thanks @mesejo !

mathause

LGTM. I added two suggestions but it's also fine to merge as is.

xarray/core/dataset.py

xarray/tests/test_dataset.py

keewis

I think there are a few more issues we should address in a new PR

keewis · 2021-01-18T23:41:19Z

xarray/core/dataset.py

+            A        (x, y) int64 0 2 3 5
+        """
+
+        indexers = either_dict_or_kwargs(indexers, indexers_kwargs, "drop")


Suggested change

indexers = either_dict_or_kwargs(indexers, indexers_kwargs, "drop")

indexers = either_dict_or_kwargs(indexers, indexers_kwargs, "drop_isel")

This one we should change — @mesejo would you be up for putting the one-line PR in?

Yes, I can do it. No problem, but just to be sure, drop_sel also has:
labels = either_dict_or_kwargs(labels, labels_kwargs, "drop")

Should I change it also? BTW drop_sel also uses np.random.randn in the docstrings should I change this also?

Ah, good spot, that's a mistake, it should be "drop_sel")

We're trying to move away from random values in the docstrings, but it's not urgent — if you can that would be appreciated, but correcting this & #4819 (comment) is fine to do alone.

Thank you!

keewis · 2021-01-18T23:42:12Z

xarray/tests/test_dataarray.py

@@ -2327,6 +2327,12 @@ def test_drop_index_labels(self):
        with pytest.warns(DeprecationWarning):
            arr.drop([0, 1, 3], dim="y", errors="ignore")

+    def test_drop_index_positions(self):
+        arr = DataArray(np.random.randn(2, 3), dims=["x", "y"])
+        actual = arr.drop_sel(y=[0, 1])


is it intentional that you use drop_sel here? ~~This succeeds because of the change to self.get_index(dim) in drop_sel, but it really should fail.~~ Edit: as it turns out, drop_sel is consistent with sel now, so I guess it makes sense that this does not fail. However, since the test is called test_drop_index_positions I still think this is a typo:

Suggested change

actual = arr.drop_sel(y=[0, 1])

actual = arr.drop_isel(y=[0, 1])

xarray/core/dataset.py

xarray/tests/test_dataset.py

max-sixty · 2021-01-19T06:28:36Z

Thanks for spotting these @keewis, sorry I missed them.

On the sel fallback to isel, though:

I might be missing something, but I don't think drop_sel should work for dimensions without coordinates: sel as well as drop_sel work on labels, so they need the coordinates

I tend to agree, but .sel does fall back to .isel for dimensions without coordinates, so the now-current code is more consistent. What are your thoughts?

mathause · 2021-01-19T09:24:14Z

I suggested using index = self.get_index(dim) in #4819. Mostly for consistency with sel & because there was no drop_isel. I don't have a strong opinion. I agree it's inconsistent - but it's also convenient and IMHO not surprising.

keewis · 2021-01-19T12:40:03Z

I agree it's important to be consistent with sel. This did come as a surprise to me, though: I assumed that sel would only work in coordinate space, so having it switch to index space when there's no coordinate is somewhat inconsistent with that.

Also, neither drop_sel nor sel explain how they work on dimensions without coordinates in their docstrings. If we decide to keep this I think we should make sure to update the documentation.

* upstream/master: Bugfix in list_engine (pydata#4811) Add drop_isel (pydata#4819) Fix RST. Remove the references to `_file_obj` outside low level code paths, change to `_close` (pydata#4809)

* upstream/master: speed up the repr for big MultiIndex objects (pydata#4846) dim -> coord in DataArray.integrate (pydata#3993) WIP: backend interface, now it uses subclassing (pydata#4836) weighted: small improvements (pydata#4818) Update related-projects.rst (pydata#4844) iris update doc url (pydata#4845) Faster unstacking (pydata#4746) Allow swap_dims to take kwargs (pydata#4841) Move skip ci instructions to contributing guide (pydata#4829) fix issues in drop_sel and drop_isel (pydata#4828) Bugfix in list_engine (pydata#4811) Add drop_isel (pydata#4819) Fix RST. Remove the references to `_file_obj` outside low level code paths, change to `_close` (pydata#4809)

* master: (458 commits) Add units if "unit" is in the attrs. (pydata#4850) speed up the repr for big MultiIndex objects (pydata#4846) dim -> coord in DataArray.integrate (pydata#3993) WIP: backend interface, now it uses subclassing (pydata#4836) weighted: small improvements (pydata#4818) Update related-projects.rst (pydata#4844) iris update doc url (pydata#4845) Faster unstacking (pydata#4746) Allow swap_dims to take kwargs (pydata#4841) Move skip ci instructions to contributing guide (pydata#4829) fix issues in drop_sel and drop_isel (pydata#4828) Bugfix in list_engine (pydata#4811) Add drop_isel (pydata#4819) Fix RST. Remove the references to `_file_obj` outside low level code paths, change to `_close` (pydata#4809) fix decode for scale/ offset list (pydata#4802) Expand user dir paths (~) in open_mfdataset and to_zarr. (pydata#4795) add a version info step to the upstream-dev CI (pydata#4815) fix the ci trigger action (pydata#4805) scatter plot by order of the first appearance of hue (pydata#4723) ...

Closes pydata#4658

1617485

- Use get_index(dim) in drop_sel - Add drop_isel

mesejo force-pushed the feature/add_drop_isel branch from 47cbca9 to 1617485 Compare January 17, 2021 14:54

max-sixty reviewed Jan 17, 2021

View reviewed changes

xarray/core/dataset.py Outdated Show resolved Hide resolved

xarray/core/dataset.py Show resolved Hide resolved

xarray/core/dataset.py Outdated Show resolved Hide resolved

xarray/core/dataset.py Show resolved Hide resolved

xarray/tests/test_dataset.py Show resolved Hide resolved

address issues in PR

a183f3c

max-sixty approved these changes Jan 18, 2021

View reviewed changes

mathause approved these changes Jan 18, 2021

View reviewed changes

xarray/core/dataset.py Outdated Show resolved Hide resolved

xarray/tests/test_dataset.py Outdated Show resolved Hide resolved

extract dict creation out of the loop

88e2815

max-sixty merged commit 2956067 into pydata:master Jan 18, 2021

keewis reviewed Jan 19, 2021

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add drop_isel #4819

Add drop_isel #4819

mesejo commented Jan 17, 2021 •

edited

Loading

max-sixty left a comment

mathause commented Jan 17, 2021

mesejo commented Jan 18, 2021

max-sixty left a comment

mathause left a comment

keewis left a comment

keewis Jan 18, 2021

max-sixty Jan 19, 2021

mesejo Jan 19, 2021

max-sixty Jan 19, 2021

keewis Jan 18, 2021 •

edited

Loading

max-sixty commented Jan 19, 2021

mathause commented Jan 19, 2021

keewis commented Jan 19, 2021

	indexers = either_dict_or_kwargs(indexers, indexers_kwargs, "drop")
	indexers = either_dict_or_kwargs(indexers, indexers_kwargs, "drop_isel")

	actual = arr.drop_sel(y=[0, 1])
	actual = arr.drop_isel(y=[0, 1])

Add drop_isel #4819

Add drop_isel #4819

Conversation

mesejo commented Jan 17, 2021 • edited Loading

max-sixty left a comment

Choose a reason for hiding this comment

mathause commented Jan 17, 2021

mesejo commented Jan 18, 2021

max-sixty left a comment

Choose a reason for hiding this comment

mathause left a comment

Choose a reason for hiding this comment

keewis left a comment

Choose a reason for hiding this comment

keewis Jan 18, 2021

Choose a reason for hiding this comment

max-sixty Jan 19, 2021

Choose a reason for hiding this comment

mesejo Jan 19, 2021

Choose a reason for hiding this comment

max-sixty Jan 19, 2021

Choose a reason for hiding this comment

keewis Jan 18, 2021 • edited Loading

Choose a reason for hiding this comment

max-sixty commented Jan 19, 2021

mathause commented Jan 19, 2021

keewis commented Jan 19, 2021

mesejo commented Jan 17, 2021 •

edited

Loading

keewis Jan 18, 2021 •

edited

Loading