Why how is [] different from loc[] ?

[johann-petrak]

There is no documentation for loc: https://docs.xarray.dev/en/stable/generated/xarray.DataArray.loc.html

So I tried this:

a1 = xr.DataArray(np.random.randn(10,4), dims=["rows", "cols"], 
                  coords=[list(range(10)), ["col1", "col2", "col3", "col4"]])
print(a1[1:3, 0:2].shape)
print(a1.loc[1:3, ["col1", "col2"]].shape)

This prints: (2,2) and (3,2)!
Why does the number of rows returned differ, though both use the slice 1:3 ??

Why does a1[1:3, 0:2] select 2 rows (as expected) but a1.loc[1:3, ["col1", "col2"]] selects 3 rows??
I have noticed this with a mach larger 2D DataArray and there doing something like a[100:102, [cols]] return 41 rows??????

[TomNicholas]

Hi @johann-petrak - sorry that there's nothing in the docstring for DataArray.loc, it should at the very least link to our general page on Indexing, which covers .loc as well as other indexing methods.

The table here states the difference between [] and .loc[] - the former indexes by position and the latter by label.

Your example DataArray has the integers 1, 2, 3 in the labels for the rows coordinate:

In [5]: a1
Out[5]: 
<xarray.DataArray (rows: 10, cols: 4)>
array([[ 0.63574108, -0.66717472,  0.00915737,  2.87035267],
       [ 0.14699325,  0.0393495 , -0.3325451 ,  1.72347164],
       [-1.08075426, -0.25283328, -0.22643869, -0.29025223],
       [ 0.64201311,  2.01351354,  0.64635188, -1.00025037],
       [-0.58669366, -0.06947171,  0.11525736,  0.33282736],
       [ 1.11675031, -1.26638065, -1.05421688, -0.95508593],
       [-1.94780685,  0.26467594, -1.73543175, -0.70611111],
       [-1.89087655,  0.38499429,  0.27008176, -0.56125357],
       [-1.20288908,  1.39705547, -0.54248866, -0.51891135],
       [-0.22672511, -2.07950408,  1.20850539, -1.81501284]])
Coordinates:
  * rows     (rows) int64 0 1 2 3 4 5 6 7 8 9
  * cols     (cols) <U4 'col1' 'col2' 'col3' 'col4'

So in your example a1[1:3, 0:2] is slicing the elements at index 1 through 3 of the row dimension, whereas a1.loc[1:3, ["col1", "col2"]] is selecting the elements at labels 1, 2 and 3 in the corresponding row coordinate.

---

This difference is easier to see if we alter your example to use different labels for the rows:

a1 = xr.DataArray(
    np.random.randn(10,4), dims=["rows", "cols"],
    coords=[
        np.arange(10) + 10,               # note the change here
        ["col1", "col2", "col3", "col4"],
    ]
)

<xarray.DataArray (rows: 10, cols: 4)>
array([[-1.70024691,  1.54145511, -0.18230258, -1.22038387],
       [ 0.99081958,  0.22927578, -0.01439966,  0.31420335],
       [ 0.61803064, -0.88470156, -0.68377578,  0.6172416 ],
       [ 0.51741572,  0.21337269,  0.29452151,  1.72603969],
       [ 0.28126145,  0.58515375, -0.11026639,  0.15560316],
       [-0.76637363,  0.67396002, -1.47838801, -0.1330614 ],
       [ 0.74503108, -0.30663846,  1.17255296,  0.46380143],
       [ 0.61045344, -2.10286191, -0.88960886,  1.87087362],
       [-0.18107632, -1.08329405,  0.44506773,  1.55144911],
       [ 0.15745851, -1.24268807, -0.55151284,  0.87646351]])
Coordinates:
  * rows     (rows) int64 10 11 12 13 14 15 16 17 18 19
  * cols     (cols) <U4 'col1' 'col2' 'col3' 'col4'

In [16]: a1.loc[1:3, ["col1", "col2"]]
Out[16]: 
<xarray.DataArray (rows: 0, cols: 2)>
array([], shape=(0, 2), dtype=float64)
Coordinates:
  * rows     (rows) int64 
  * cols     (cols) <U4 'col1' 'col2'

No elements were returned along the rows dimension here because there are no elements in the array which have 1, 2 or 3 as labels.

In [19]: a1.loc[11:13, ["col1", "col2"]]
Out[19]: 
<xarray.DataArray (rows: 3, cols: 2)>
array([[ 0.99081958,  0.22927578],
       [ 0.61803064, -0.88470156],
       [ 0.51741572,  0.21337269]])
Coordinates:
  * rows     (rows) int64 11 12 13
  * cols     (cols) <U4 'col1' 'col2'

The elements with labels matching 11, 12 and 13 along the rows coordinate were returned.

---

If you want to mix positional indexing and label-based indexing you will need to supply each in a different method call, though you can chain these methods together on one line, like this:

In [20]: a1[1:3].loc[:, ["col1", "col2"]]
Out[20]: 
<xarray.DataArray (rows: 2, cols: 2)>
array([[ 0.99081958,  0.22927578],
       [ 0.61803064, -0.88470156]])
Coordinates:
  * rows     (rows) int64 11 12
  * cols     (cols) <U4 'col1' 'col2'

This is probably what you expected to happen.

I personally prefer to make the dimensions I'm indexing along explicit by using .isel and .sel instead of [] and .loc[], e.g. this, which is equivalent to what we just did (but more explicit because the result doesn't depend on dimension order):

In [24]: a1.isel(rows=slice(1,3)).sel(cols=["col1", "col2"])
Out[24]: 
<xarray.DataArray (rows: 2, cols: 2)>
array([[ 0.99081958,  0.22927578],
       [ 0.61803064, -0.88470156]])
Coordinates:
  * rows     (rows) int64 11 12
  * cols     (cols) <U4 'col1' 'col2'

I recommend first reading the page on indexing for all indexing-related queries.

[johann-petrak]

@TomNicholas thanks a lot for that detailed answer and the pointer to the page about indexing, that is very helpful for a noob like me!

There is still one thing I do not understand though: if I understand correctly the semantics of slicing with indexing and without indexing are different, in that index-based slicing includes the to element while standard slicing does not.
Here is an illustration:

a1 = xr.DataArray(np.random.randn(10,4), dims=["rows", "cols"],
                   coords=dict(rows=np.arange(10), cols=["col1", "col2", "col3", "col4"]))
a2 = xr.DataArray(np.random.randn(10,4), dims=["rows", "cols"],
                   coords=dict(cols=["col1", "col2", "col3", "col4"]))
print(a1.loc[1:3, ["col1", "col2"]].shape)
# -> (3,2)
print(a2.loc[1:3, ["col1", "col2"]].shape)
# -> (2,2)

To me that is something that is totally unexpected and surprising.

[dcherian]

Yes, it's behaviour copied from pandas an d might not have been the best choice.

[johann-petrak]

Oh right, so pandas does this wrong do.
Definitely triggers my OCD :/
Thanks!

[johann-petrak]

As an afterthought it might be useful the (loudly) announce this in the docs, especially to people who are not coming from pandas and expect a slice NOT to contain the to value, as basically everywhere else in Python and Numpy.