Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Explicit indexes #5692

Merged
merged 168 commits into from
Mar 17, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
168 commits
Select commit Hold shift + click to select a range
6ccebeb
no need to wrap pandas index in lazy index adapter
benbovy Aug 11, 2021
6d5c79d
multi-index default level names
benbovy Aug 11, 2021
4742100
refactor setting Dataset/DataArray default indexes
benbovy Aug 11, 2021
55dd9d9
update multi-index (text) repr
benbovy Aug 11, 2021
752a405
remove print
benbovy Aug 11, 2021
1250ccf
minor fixes and improvements
benbovy Aug 12, 2021
1bb61d9
fix dtype of index variables created from Index
benbovy Aug 12, 2021
a551c7f
fix multi-index selection regression
benbovy Aug 12, 2021
abc3384
check conflicting multi-index level names
benbovy Aug 13, 2021
92c8ca4
update formatting (text and html)
benbovy Aug 13, 2021
b9e1cba
check level name conflicts for midx given as coord
benbovy Aug 23, 2021
60935ac
intended behavior or unwanted side effect? see #5732
benbovy Aug 23, 2021
3086d4e
get rid of multi-index virtual coordinates
benbovy Aug 23, 2021
54a04f7
add level coords in indexes & keep coord order
benbovy Aug 25, 2021
e3181d4
fix copying multi-index level variable data
benbovy Aug 25, 2021
81a60f6
collect index for multi-index level variables
benbovy Aug 25, 2021
2e4a041
wip refactor label based selection
benbovy Aug 26, 2021
f387c70
fix index query tests
benbovy Aug 27, 2021
6a9dbd6
fix multi-index adapter getitem scalar
benbovy Aug 27, 2021
64b71c9
wip refactor label based selection
benbovy Aug 27, 2021
f3116ac
wip: deeper refactoring label-based sel
benbovy Aug 30, 2021
3ca6a02
fix some tests + minor tweaks
benbovy Aug 31, 2021
05c488d
fix indexing PandasMultiIndexingAdapater
benbovy Aug 31, 2021
cc2d9c9
refactor cast label indexer to coord dtype
benbovy Aug 31, 2021
1588824
Merge branch 'main' into explicit-indexes-variables
benbovy Aug 31, 2021
a8d84c7
better handling of multi-index level labels
benbovy Aug 31, 2021
a6bce37
label-based selection tweaks and fixes
benbovy Sep 9, 2021
ad17931
sel: propagate multi-index vars attrs/encoding
benbovy Sep 9, 2021
b894f8d
wip refactor rename
benbovy Sep 9, 2021
e42e324
wip refactor rename: return new vars from Index
benbovy Sep 10, 2021
5febd7f
refactor rename: update tests
benbovy Sep 10, 2021
508cdcc
typing tweaks
benbovy Sep 10, 2021
f1535a3
fix html formatting: dims w/ or w/o index
benbovy Sep 10, 2021
d5cc3b4
Merge branch 'main' into explicit-indexes-variables
benbovy Sep 13, 2021
021090f
minor fixes and tweaks
benbovy Sep 13, 2021
0086e32
wip refactor set_index
benbovy Sep 13, 2021
a891e22
wip refactor set_index
benbovy Sep 14, 2021
e50978e
refactor reorder_levels
benbovy Sep 14, 2021
8b1e4d5
Set pd.MultiIndex name from dim name
benbovy Sep 15, 2021
e2075b7
.sel() with multi-index: return scalar coords
benbovy Sep 15, 2021
4fbdc86
fix multi-index level coordinate inline repr
benbovy Sep 15, 2021
ef6fbbd
wip refactor stack
benbovy Sep 15, 2021
8bc2590
stack: better rule for add/skip multi-index
benbovy Sep 15, 2021
64df1e9
add utility methods to class Indexes
benbovy Sep 16, 2021
da8846d
re-arrange class Indexes internals
benbovy Sep 16, 2021
28ecad6
wip refactor stack / unstack
benbovy Sep 16, 2021
e62b9cd
add PandasMultiIndex.from_product_variables
benbovy Sep 23, 2021
f7aca70
unstack: propagate index coordinate metadata
benbovy Sep 23, 2021
e44c56e
wip: fix/update tests
benbovy Sep 24, 2021
f88c2c2
fix/refactor reindex
benbovy Sep 24, 2021
1daaf8d
functools.cached_property is since py38
benbovy Sep 24, 2021
41ebc45
update reset_coords
benbovy Sep 24, 2021
e750883
fix ipython key completion test
benbovy Sep 24, 2021
6c6f09e
update test_map_index_queries
benbovy Sep 27, 2021
6437e13
do not coerce bool indexer as float coord dtype
benbovy Sep 27, 2021
a955d45
add Index.create_variables() method
benbovy Oct 13, 2021
c87586e
improve Dataset/DataArray indexes proxy class
benbovy Oct 13, 2021
db65e44
align Indexes API with Coordinates API
benbovy Oct 13, 2021
417f1f4
clean-up formatting using updated Indexes API
benbovy Oct 13, 2021
929b8f3
wip: refactor alignment
benbovy Oct 14, 2021
02cc5d2
wip refactor alignment
benbovy Oct 14, 2021
d1bbc4a
tweaks and fixes
benbovy Oct 15, 2021
0624204
wip refactor alignment and reindex
benbovy Oct 18, 2021
30fee83
wip refactor alignment / reindex: fixes and tweaks
benbovy Oct 19, 2021
981fa0d
wip refactor alignment (support join='override')
benbovy Oct 19, 2021
79b5223
wip alignment: clean-up + tests
benbovy Oct 20, 2021
63a4291
refactor alignment fix tests & types annotations
benbovy Oct 21, 2021
9f8d26d
refactor interp and interp_like
benbovy Oct 21, 2021
f291c65
wip review merge
benbovy Oct 22, 2021
8e3eef2
refactor swap_dims
benbovy Oct 25, 2021
5e5abb7
refactor isel
benbovy Oct 25, 2021
2c11dbc
add create_index option to stack
benbovy Oct 25, 2021
e65e3f3
add Index.stack and Index.unstack methods
benbovy Oct 27, 2021
d02dc17
fix PandasIndex.isel with scalar indexers
benbovy Oct 27, 2021
557e8b1
fix index adapter (variable) dtype
benbovy Oct 27, 2021
15f6b17
more indexes invariants checks
benbovy Oct 28, 2021
29035aa
fix DataArray.isel()
benbovy Oct 28, 2021
2a067bd
refactor Dataset/DataArray copy
benbovy Oct 28, 2021
c8e7a06
refactor computation (and merge)
benbovy Oct 28, 2021
9963520
minor fixes and tweaks
benbovy Oct 28, 2021
e0f498a
propagate_indexes -> filter_indexes_from_coords
benbovy Oct 28, 2021
7901404
Merge branch 'main' into explicit-indexes-variables
benbovy Oct 28, 2021
26bc7c9
Update xarray/core/coordinates.py
Illviljan Oct 29, 2021
5da11f4
refactor expand_dims
benbovy Nov 2, 2021
e0b08c1
merge: avoid compare same indexes more than once
benbovy Nov 2, 2021
fbf556c
wip refactor update coords
benbovy Nov 2, 2021
ef3feb6
refactor update/remove coords
benbovy Nov 3, 2021
52da324
Merge remote-tracking branch 'pydata/main' into explicit-indexes-vari…
benbovy Nov 3, 2021
78f4fb0
misc. fixes
benbovy Nov 3, 2021
4dcb136
fix Dataset,__delitem__
benbovy Nov 3, 2021
72aff10
fix Dataset.reset_coords (default coords)
benbovy Nov 3, 2021
c0eb3dd
refactor Dataset.from_dataframe
benbovy Nov 9, 2021
34749b2
Fix .sel with DataArray and multi-index
benbovy Nov 9, 2021
ef952b2
PandasIndex.from_variables: preserve wrapped index
benbovy Nov 9, 2021
adcec91
Merge branch 'main' of https://github.com/pydata/xarray into explicit…
benbovy Feb 2, 2022
1d6694a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Feb 2, 2022
ebb2605
two minor fixes after merging main
benbovy Feb 3, 2022
5c3ff55
filter_indexes_from_coords: preserve index order
benbovy Feb 4, 2022
8e8690a
implicit multi-index from coord: fix edge case
benbovy Feb 4, 2022
93c3f69
groupby combine: fix missing index in result
benbovy Feb 4, 2022
3316ee4
implicit multi-index coord: more robust fix
benbovy Feb 4, 2022
f9a0424
Merge branch 'main' of https://github.com/pydata/xarray into explicit…
benbovy Feb 9, 2022
f2b25f5
backward compat fix: multi-index given as data-var
benbovy Feb 9, 2022
627a001
refactor concat
benbovy Feb 9, 2022
ba1c75f
sel drop=True: remove multi-index scalar coords
benbovy Feb 10, 2022
3371623
PandasIndex.from_variables(multi-index level var)
benbovy Feb 10, 2022
8fec03c
reindex: disable invalid dimension check
benbovy Feb 10, 2022
d76be3d
add index concat tests + fix positions type
benbovy Feb 11, 2022
3b78f4b
add Indexes.get_all_dims convenient method
benbovy Feb 11, 2022
7ce000e
refactor pad
benbovy Feb 11, 2022
5fa287f
unstack: return copies of mindex.levels
benbovy Feb 11, 2022
11bf070
strip_units: prevent index->array conversion
benbovy Feb 11, 2022
35a7464
attach_units: do not preserve coord multi-indexes
benbovy Feb 11, 2022
cfb0cf1
concat: avoid multiple loads of dask arrays
benbovy Feb 14, 2022
8cc1253
groupby mindex coord: propagate level names
benbovy Feb 14, 2022
ead47a2
Dataset.copy: don't reset indexes if data is given
benbovy Feb 14, 2022
23dc4cf
reindex/align: fix coord dtype of new indexes
benbovy Feb 14, 2022
8b325ef
fix doctests
benbovy Feb 14, 2022
8528377
Merge remote-tracking branch 'pydata/main' into explicit-indexes-vari…
benbovy Feb 14, 2022
30023a4
to_stacked_array: do not coerce level dtypes
benbovy Feb 14, 2022
d7fed29
fix indent
benbovy Feb 14, 2022
76e15ef
doc: fix user-guide build errors
benbovy Feb 14, 2022
d69bb45
stack: fix new index variables not in coordinates
benbovy Feb 14, 2022
44e6893
unstack full-reindex: fix alignment errors
benbovy Feb 14, 2022
50bd989
PandasIndex coord dtype: avoid convert index->array
benbovy Feb 14, 2022
1184f50
refactor diff
benbovy Feb 14, 2022
ed7f24b
quick fix level coord dtype int32/64 on win
benbovy Feb 14, 2022
ffa06da
dask presist/compute dataarray: propagate indexes
benbovy Feb 14, 2022
7c08769
refactor roll
benbovy Feb 15, 2022
71ac97a
rename Index.query -> Index.sel
benbovy Feb 15, 2022
7600e46
remove Index.union and Index.intersection
benbovy Feb 15, 2022
1b6c97d
use future annotations in indexes.py
benbovy Feb 15, 2022
eda0e85
Index.rename: return only the new index
benbovy Feb 15, 2022
24d7719
Index.stack: return only the new index
benbovy Feb 15, 2022
2aea2e3
PandasMultiIndex class methods: return only the index
benbovy Feb 15, 2022
32d887c
wip get rid of Pandas(Multi)Index.from_pandas_index
benbovy Feb 15, 2022
851a942
remove Pandas(Multi)Index.from_pandas_index
benbovy Feb 15, 2022
3134f33
Index.from_variables: return the new index only
benbovy Feb 15, 2022
9bbf659
rename: propagate_attrs_encoding not needed
benbovy Feb 15, 2022
af34834
align exclude dims: pass through indexes
benbovy Feb 16, 2022
fe4fcc5
fix set_index append=True
benbovy Feb 16, 2022
df31fa8
refactor default_indexes and re-enable invariant check
benbovy Feb 16, 2022
631a8e2
fix formatting errors
benbovy Feb 16, 2022
21fa7a8
fix type
benbovy Feb 16, 2022
f573085
Dataset._indexes, DataArray._indexes: always dict
benbovy Feb 16, 2022
3280eeb
remove _level_coords property
benbovy Feb 16, 2022
a415188
clean-up
benbovy Feb 17, 2022
6405ffd
optimize Dataset/DataArray copy
benbovy Feb 17, 2022
2b1f90b
add default implementation for Index.copy
benbovy Feb 17, 2022
73ba9d4
add/fix indexes tests
benbovy Feb 17, 2022
3927479
tweak indexes formatting
benbovy Feb 17, 2022
a3173b5
fix doctests
benbovy Feb 17, 2022
709a58e
PandasIndex.copy: avoid too many pd.Index copies
benbovy Feb 18, 2022
ed1af4d
DataArray stack test: revert to original
benbovy Feb 21, 2022
9708eb3
test indexes/indexing: rename query -> sel
benbovy Feb 21, 2022
7f57db5
alignment: use future annotations
benbovy Feb 21, 2022
df64f0d
misc. tweaks
benbovy Feb 21, 2022
a9add15
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Feb 21, 2022
f807733
Aligner tweaks
benbovy Mar 7, 2022
e97653e
remove assert_unique_multiindex_level_names
benbovy Mar 7, 2022
72d0782
remove propagate_attrs_encoding
benbovy Mar 7, 2022
4f24faf
assert -> check + ValueError
benbovy Mar 7, 2022
601dc3a
misc. fixes and tweaks
benbovy Mar 7, 2022
5bf582b
Merge remote-tracking branch 'pydata/main' into explicit-indexes-vari…
benbovy Mar 7, 2022
bf0dbb7
update what's new
benbovy Mar 7, 2022
1baa875
Merge branch 'main' into explicit-indexes-variables
keewis Mar 7, 2022
c1b778a
concat: remove fall-backs to Variable.concat
benbovy Mar 11, 2022
77fdaf0
fix flaky test
benbovy Mar 11, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/user-guide/plotting.rst
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,7 @@ Finally, if a dataset does not have any coordinates it enumerates all data point
.. ipython:: python
:okwarning:

air1d_multi = air1d_multi.drop("date")
air1d_multi = air1d_multi.drop(["date", "time", "decimal_day"])
air1d_multi.plot()

The same applies to 2D plots below.
Expand Down
14 changes: 14 additions & 0 deletions doc/whats-new.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,18 @@ v2022.03.1 (unreleased)
New Features
~~~~~~~~~~~~

- Add a ``create_index=True`` parameter to :py:meth:`Dataset.stack` and
:py:meth:`DataArray.stack` so that the creation of multi-indexes is optional
(:pull:`5692`). By `Benoît Bovy <https://github.com/benbovy>`_.
- Multi-index levels are now accessible through their own, regular coordinates
instead of virtual coordinates (:pull:`5692`).
By `Benoît Bovy <https://github.com/benbovy>`_.

Breaking changes
~~~~~~~~~~~~~~~~

- The Dataset and DataArray ``rename*`` methods do not implicitly add or drop
indexes. (:pull:`5692`). By `Benoît Bovy <https://github.com/benbovy>`_.

Deprecations
~~~~~~~~~~~~
Expand All @@ -37,6 +45,9 @@ Bug fixes
- Set ``skipna=None`` for all ``quantile`` methods (e.g. :py:meth:`Dataset.quantile`) and
ensure it skips missing values for float dtypes (consistent with other methods). This should
not change the behavior (:pull:`6303`). By `Mathias Hauser <https://github.com/mathause>`_.
- Many bugs fixed by the explicit indexes refactor, mainly related to multi-index (virtual)
coordinates. See the corresponding pull-request on GitHub for more details. (:pull:`5692`).
By `Benoît Bovy <https://github.com/benbovy>`_.

Documentation
~~~~~~~~~~~~~
Expand All @@ -45,6 +56,9 @@ Documentation
Internal Changes
~~~~~~~~~~~~~~~~

- Many internal changes due to the explicit indexes refactor. See the
corresponding pull-request on GitHub for more details. (:pull:`5692`).
By `Benoît Bovy <https://github.com/benbovy>`_.

.. _whats-new.2022.02.0:
.. _whats-new.2022.03.0:
Expand Down
Loading