PERF: ascii c string functions

[chris-b1]

• closes read_csv() is 3.5X Slower in Pandas 0.23.4 on Python 3.7.1 vs Pandas 0.22.0 on Python 3.5.2 #23516
• tests added / passed
• passes git diff upstream/master -u -- "*.py" | flake8 --diff
• whatsnew entry

This ends up being kind of hairy - open to suggestions on a better fix. cc @dragoljub, @cgohlke

Timings

df = pd.DataFrame(np.random.randn(1000000, 10), 
    columns=('COL{}'.format(i) for i in range(10)))
df.to_csv('~/tmp.csv')

# python 3.6 - pandas 0.23.4
In [1]: %timeit pd.read_csv('~/tmp.csv')
4.55 s ± 344 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

# python 3.7 - before
In [3]: %timeit pd.read_csv('~/tmp.csv')
21.8 s ± 769 ms per loop (mean ± std. dev. of 7 runs, 1 loop ea

# python 3.7 - after
In [1]: %timeit pd.read_csv('~/tmp.csv')
4.25 s ± 168 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

[pep8speaks]

Hello @chris-b1! Thanks for submitting the PR.

• There are no PEP8 issues in the file pandas/_version.py !

[jreback]

can you add a whatsnew note in perf

[dragoljub]

@chris-b1 great work! 👍 The simple fix always seems to add a few more layers of complexity.

Can someone build/try this on Linux to make sure there is no regression there? 😉

[jbrockmendel]

Did tracking this problem down give any insight into whether/when we can de-duplicate parse_helper.xstrtod and tokenizer.xstrtod?

[jreback]

@chris-b1 I think you have some linting errors, can you also rebase on master & add a whatsnew

andas/_libs/src/parser/tokenizer.c:26:  At least two spaces is best between code and comments  [whitespace/comments] [2]

[chris-b1]

@jbrockmendel - It will take a bit of refactoring to deal with the maybe_int out param - the parse_helper one needs its, the csv parsing one doesn't. Will have that as a follow-up, have #19361 on it.

[codecov]

Codecov Report

Merging #23981 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #23981   +/-   ##
=======================================
  Coverage   42.38%   42.38%           
=======================================
  Files         161      161           
  Lines       51701    51701           
=======================================
  Hits        21914    21914           
  Misses      29787    29787

Flag	Coverage Δ
#single	42.38% <ø> (ø)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 08395af...8af538f. Read the comment docs.

[codecov]

Codecov Report

Merging #23981 into master will decrease coverage by <.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #23981      +/-   ##
==========================================
- Coverage    92.2%    92.2%   -0.01%     
==========================================
  Files         162      162              
  Lines       51714    51714              
==========================================
- Hits        47682    47681       -1     
- Misses       4032     4033       +1

Flag	Coverage Δ
#multiple	90.6% <ø> (ø)	⬆️
#single	43.01% <ø> (-0.01%)	⬇️

Impacted Files	Coverage Δ
pandas/io/json/json.py	92.61% <0%> (-0.48%)	⬇️
pandas/util/testing.py	87.51% <0%> (+0.09%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9f2c716...5abc57d. Read the comment docs.

[jreback]

can you merge master and push again.

[jreback]

@chris-b1 this will fail on 3.6 build (fix was just merged), but ping when ready otherwise.

[chris-b1]

@jreback - other than CI issues, I think this is good to go?

[jreback]

yeah just waiting for a good ci run (don't rebase yet).

[jreback]

looks like ci is fixed up, can you rebase

[jreback]

thanks @chris-b1