Support pandas >=1.0.0 #1197
Support pandas >=1.0.0 #1197
Conversation
|
While i'm here, i'll keep checking the pandas release note and updating this PR in order to be compatible with Pandas 1.0.0 as soon as possible after it is released. |
Codecov Report
@@ Coverage Diff @@
## master #1197 +/- ##
==========================================
+ Coverage 95.02% 95.03% +<.01%
==========================================
Files 34 34
Lines 7220 7232 +12
==========================================
+ Hits 6861 6873 +12
Misses 359 359
Continue to review full report at Codecov.
|
…pport_pandas_1.0.0
…pport_pandas_1.0.0
…pport_pandas_1.0.0
…pport_pandas_1.0.0
|
Thanks for the conclusion :D Okay, so I'll check again whole changes one last time based on that concept. |
|
Sorry, @itholic, I removed my last comments for nits but it removed your replies together. Ignore my last comments. |
|
@HyukjinKwon ah, it's okay. 👍 |
…pport_pandas_1.0.0
| @@ -777,39 +777,40 @@ def test_monotonic(self): | |||
| datas.append([(-5, 'e'), (-4, 'c'), (-3, 'b'), (-2, 'd'), (-1, 'a')]) | |||
|
|
|||
| # None type tests (None type is treated as the largets value) | |||
| datas.append([(None, 100), (2, 200), (3, 300), (4, 400), (5, 500)]) | |||
| # TODO: the commented tests below should be uncommented after fixing for pandas >= 1.0.0 | |||
There was a problem hiding this comment.
Is it difficult to patch to pandas' 1.0.0 behaviours?
There was a problem hiding this comment.
yeah its changing in 1.0.0 looks a bit complicated, let me try to fix this follow-up PR
|
Okay, let's merge this and address my comments in a followup @itholic. Can you just resolve conflicts? |
yeah, i've resolved nit comments and will address rest ones in follow-up. thanks for the comments ! |
| @@ -51,6 +54,16 @@ def _apply_as_series_or_frame(self, func): | |||
| def count(self): | |||
| def count(scol): | |||
| return F.count(scol).over(self._window) | |||
|
|
|||
| if LooseVersion(pd.__version__) >= LooseVersion('1.0.0'): | |||
There was a problem hiding this comment.
@itholic, given our discussion, we should match it to the latest behaviour. I think we don't have to check the pandas version.
|
Okay, I am going to merge this to make it rolling. @itholic, can you do all todos in the next followup? |
|
@HyukjinKwon yup. i'm working on that. |
Missing/deprecated functions/properties removed in pandas 1.0 were also removed from Koalas (#1197), but we should still show error messages at least when a user using pandas<1.0 tries to use such functions/properties.
Follow-up for #1197 Since we're following latest version of pandas, should fix several TODOs with matching pandas>=1.0.0 for now. ## For example. the behaviour of `Expanding.count()` and `ExpandingGroupby.count()` are different depending on what pandas version has been installed. - pandas < 1.0.0 ```python >>> s = pd.Series([2, 2, 3, 3, 3, 4, 4, 4, 4, 5, 5]) >>> s.groupby(s).expanding(3).count().sort_index() 2 0 1.0 1 2.0 3 2 1.0 3 2.0 4 3.0 4 5 1.0 6 2.0 7 3.0 8 4.0 5 9 1.0 10 2.0 dtype: float64 ``` - pandas >= 1.0.0 ```python >>> s = pd.Series([2, 2, 3, 3, 3, 4, 4, 4, 4, 5, 5]) >>> s.groupby(s).expanding(3).count().sort_index() 2 0 NaN 1 NaN 3 2 NaN 3 NaN 4 3.0 4 5 NaN 6 NaN 7 3.0 8 4.0 5 9 NaN 10 NaN dtype: float64 ``` Since we're following latest version of pandas, need to fix this.
Related with #1194
This PR is for updating newly added & deprecated & deleted APIs on pandas 1.0.0 to keep compatible with pandas 1.0.0 to be released soon, based on What’s new in 1.0.0