Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement any and all for pyarrow numpy strings #54591

Merged
merged 32 commits into from
Aug 28, 2023
Merged

Implement any and all for pyarrow numpy strings #54591

merged 32 commits into from
Aug 28, 2023

Conversation

phofl
Copy link
Member

@phofl phofl commented Aug 16, 2023

This should work if we want to follow NumPy semantics as close as possible. I think this should work in general, but that's a discussion for another day

phofl and others added 16 commits August 13, 2023 14:25
@phofl
Start new string array
b24afc9
@phofl
Add missing methods
b306c6f
@phofl
Implement Arrow String Array that is compatible with NumPy semantics
2dbcfb0
@phofl
Move methods
d9e61e5
@phofl
Refactor
3188c25
@phofl
Merge remote-tracking branch 'upstream/main' into string_array_numpy_…
df231f0 
…semantics

# Conflicts:
#	pandas/core/arrays/_arrow_string_mixins.py
#	pandas/tests/strings/__init__.py
#	pandas/tests/strings/test_strings.py
@phofl
Refactor
cd19bfb
@phofl
Remove
c73c6b0
@phofl
Fix
6b26309
@phofl
Update
da6d67c
@phofl
Na return value
d862eca
@phofl
Merge remote-tracking branch 'upstream/main' into string_array_numpy_…
4be0ee8 
…semantics_2

# Conflicts:
#	pandas/tests/strings/test_find_replace.py
@phofl
Fix
6cf2639
@phofl
Merge remote-tracking branch 'origin/string_array_numpy_semantics' in…
333dcae 
…to string_array_numpy_semantics
@phofl
Update
48bd626
@phofl
Implement any and all for pyarrow numpy strings
4f9387a
phofl and others added 11 commits August 21, 2023 11:15
@phofl
Fix typing
606cd71
@phofl
Merge remote-tracking branch 'upstream/main' into string_array_numpy_…
a44b042 
…semantics_na_val

# Conflicts:
#	pandas/tests/arrays/string_/test_string.py
#	pandas/tests/extension/test_string.py
@phofl
Update
6414501
@phofl
Merge remote-tracking branch 'upstream/main' into any_all
27b5057 
# Conflicts:
#	pandas/tests/arrays/string_/test_string.py
#	pandas/tests/extension/test_string.py
@phofl
Merge remote-tracking branch 'upstream/main' into string_array_numpy_…
3fec6d3 
…semantics_na_val

# Conflicts:
#	pandas/tests/strings/__init__.py
#	pandas/tests/strings/test_split_partition.py
@phofl
Fix
68acc32
@phofl
Fix
fbab6fb
@phofl
Merge branch 'string_array_numpy_semantics_na_val' into any_all
dd0f9a8 
# Conflicts:
#	pandas/tests/extension/test_string.py
@phofl
Fix
68e5f8f
@phofl
Move test
0322006
@phofl
Skip test when no pa
8bb52f4
@jbrockmendel
Copy link
Member

I think this should work in general, but that's a discussion for another day

+1

(Also I recently realized that Index(["A", "B"]).any_or_all() is different from Series(["A", "B"]).any_or_all() and we should deprecate to get matching behavior)

phofl and others added 2 commits August 22, 2023 17:23
@phofl
Fix typing
cc8e6f7
@phofl
Merge remote-tracking branch 'upstream/main' into any_all
a3ef88d 
# Conflicts:
#	pandas/core/arrays/string_arrow.py
#	pandas/tests/arrays/string_/test_string.py
#	pandas/tests/strings/test_case_justify.py
@phofl
Copy link
Member Author

phofl commented Aug 23, 2023

rebased as well

@lithomas1
Copy link
Member

Is the goal to get this in for 2.1 or is this fine for a later release?

@phofl
Copy link
Member Author

phofl commented Aug 27, 2023

Ideally we get this in, but not a deal breaker.

cc @jorisvandenbossche for a final review

@lithomas1 lithomas1 added this to the 2.1 milestone Aug 27, 2023
@lithomas1 lithomas1 added Strings String extension data type and string data Arrow pyarrow functionality labels Aug 27, 2023
@jorisvandenbossche jorisvandenbossche mentioned this pull request Aug 28, 2023
Open
41 tasks
@jorisvandenbossche jorisvandenbossche merged commit 23b546f into pandas-dev:main Aug 28, 2023
32 of 33 checks passed
@lumberbot-app
Copy link

lumberbot-app bot commented Aug 28, 2023

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

  1. Checkout backport branch and update it.
git checkout 2.1.x
git pull
  1. Cherry pick the first parent branch of the this PR on top of the older branch:
git cherry-pick -x -m1 23b546f96f49611541585c3914301ba874ced308
  1. You will likely have some merge/cherry-pick conflict here, fix them and commit:
git commit -am 'Backport PR #54591: Implement any and all for pyarrow numpy strings'
  1. Push to a named branch:
git push YOURFORK 2.1.x:auto-backport-of-pr-54591-on-2.1.x
  1. Create a PR against branch 2.1.x, I would have named this PR:

"Backport PR #54591 on branch 2.1.x (Implement any and all for pyarrow numpy strings)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the Still Needs Manual Backport label once the PR gets merged.

If these instructions are inaccurate, feel free to suggest an improvement.

@phofl phofl deleted the any_all branch August 28, 2023 11:18
@phofl
Copy link
Member Author

phofl commented Aug 28, 2023

I'll take care of the back-Ort

phofl added a commit to phofl/pandas that referenced this pull request Aug 28, 2023
@phofl
Implement any and all for pyarrow numpy strings (pandas-dev#54591)
cbf64fb 
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

(cherry picked from commit 23b546f)
@phofl phofl mentioned this pull request Aug 28, 2023
Merged
phofl added a commit that referenced this pull request Aug 28, 2023
@phofl
Backport PR #54591 on branch 2.1.x (Implement any and all for pyarrow…
72cb101 
… numpy strings) (#54796)
@jbrockmendel
Copy link
Member

I think this should work in general, but that's a discussion for another day

Another day has come!

@phofl phofl mentioned this pull request Aug 28, 2023
Open
3 tasks
mroeschke pushed a commit to mroeschke/pandas that referenced this pull request Sep 11, 2023
@phofl @jorisvandenbossche @mroeschke
Implement any and all for pyarrow numpy strings (pandas-dev#54591)
5536671 
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
@jorisvandenbossche jorisvandenbossche mentioned this pull request Jul 30, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jorisvandenbossche jorisvandenbossche jorisvandenbossche left review comments

@jbrockmendel jbrockmendel Awaiting requested review from jbrockmendel

Assignees
No one assigned
Labels
Arrow pyarrow functionality Strings String extension data type and string data
Projects
None yet
Milestone
2.1
Development

Successfully merging this pull request may close these issues.
4 participants
@phofl @jbrockmendel @lithomas1 @jorisvandenbossche