Initial Backport of string changes for 2.3 release

[WillAyd]

No description provided.

[WillAyd]

I'm unsure about the failing tests. Seems like we might have behavior in our string types that works off of what is done in #57361 but guessing we don't want to backport that PR to 2.3?

[WillAyd]

A lot of failures seem related to the handling of to_numeric with errors="ignore", which was removed for the 3.0 release.

Prior to #57361, this is what a to_numeric call would yield:

>>> pd.to_numeric('-47393996303418497800', errors="ignore")
'-47393996303418497800'

with all of the backport, it looks like the new behavior casts it to a float:

>>> pd.to_numeric('-47393996303418497800', errors="ignore")
-4.73939963e+19

[WillAyd]

@mroeschke could I get some of your help on the test failures? A lot of them look related to NumPy and have to do with how it handles overflows, casting, and some syntax things with numexpr.

Any chance you saw those on main and know what the fix is? https://github.com/pandas-dev/pandas/actions/runs/10514684927/job/29133089833?pr=59513#step:8:2013

[jbrockmendel]

@WillAyd looking at the failures in the first build, i think #58484 may need to be backported

[mroeschke]

I don't remember seeing failures related to eval but I'll dig a little bit more.

In addition to what @jbrockmendel mentioned, you'll need #59545 and the nomkl install part of #59553

[jbrockmendel]

Getting close here

___________________ TestBasic.test_write_index[fastparquet] ____________________
[gw2] linux -- Python 3.12.5 /home/runner/micromamba/envs/test/bin/python3.12
[XPASS(strict)] fastparquet write into index

[bnavigator]

I don't remember seeing failures related to eval but I'll dig a little bit more.

Already reported in May (und unfortunately not picked up by anyone) here: #58548. It tracks the failures in main and sees a fix somewhere between a3e751c and 236d89b.

Fixed by backport here: #59535

[WillAyd]

It tracks the failures in main and sees a fix somewhere between a3e751c and 236d89b.

I don't believe that issue was directly solved, but it was at least removed from the test suite as part of changes in https://github.com/pandas-dev/pandas/pull/56709/files#diff-f145e441b5820d235a78589ee9ee6c9c49fea0de6ca659a972b61b6aa29f9df8 that replaced the fixture of:

@pytest.mark.parametrize("dtype", [np.float32, np.float64])

with a fixture that instead uses:

[float, "float32", "float64"]

[lithomas1]

@WillAyd

Mind if I push some fixes for the non string dtype builds to your branch?
I finally got to taking a look at this PR and I have a couple of fixes locally.

[WillAyd]

Sure by all means. FWIW I've been trying to keep the order of cherry picked commits maintained and put any custom patches at the end

[lithomas1]

(tagging build so we get some CI signal from the wheel builders as well)

[jorisvandenbossche]

@WillAyd looking into some of the failures, and pushed a few commits:

• first commits is fixing up a backported PR's commit (feel free to merge that with the actual commit in a next rebase)
• second and fourth (coming) commit are general fixes to tests (for tests that for some reason (e.g. removed) didn't need a fix on main, but need a fix here). Feel free to combine those all in a single commit
• third commit is backporting String dtype: still return nullable NA-variant in object inference (maybe_converts_object) if requested #59487, which seemed to be missing here. The actual code change in that PR does not apply on this branch, so in practice this PR has become a test-only change. But those test changes were still needed to get a bunch of tests passing

With those changes, the number of failures went from 916 to around 140