Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-112361: Speed up pathlib by removing some temporary objects. #112362

Merged
merged 2 commits into from
Nov 25, 2023

Conversation

barneygale
Copy link
Contributor

@barneygale barneygale commented Nov 24, 2023

Construct only one new list object (using list.copy()) when creating a new path object with a modified tail. This slightly speeds up with_name(), with_suffix(), _make_child_relpath() (used in walking and globbing), and glob().

Construct only one new list object (using `list.copy()`) when creating a
new path object with a modified tail. This slightly speeds up
`with_name()`, `with_suffix()`, `_make_child_relpath()` (used in walking
and globbing), and `glob()`.
@barneygale
Copy link
Contributor Author

Timings:

$ ./python -m timeit -n 1000000 -s "from pathlib import Path; p = Path('foo')" "p.with_name('bar')"
1000000 loops, best of 5: 1.44 usec per loop  # before
1000000 loops, best of 5: 1.31 usec per loop  # after

$ ./python -m timeit -n 1000000 -s "from pathlib import Path; p = Path('foo.txt')" "p.with_suffix('.zip')"
1000000 loops, best of 5: 2.02 usec per loop  # before
1000000 loops, best of 5: 1.83 usec per loop  # after

$ ./python -m timeit -n 1000000 -s "from pathlib import Path; p = Path('foo.txt')" "p.with_suffix('')"
1000000 loops, best of 5: 1.88 usec per loop  # before
1000000 loops, best of 5: 1.68 usec per loop  # after

$ ./python -m timeit -n 1000000 -s "from pathlib import Path; p = Path('foo')" "p._make_child_relpath('bar')"
1000000 loops, best of 5: 1.14 usec per loop  # before
1000000 loops, best of 5: 1.11 usec per loop  # after

Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this looks really nice -- great work! I can reproduce the big speedups from the first three benchmarks in #112362 (comment).

I can't repro the reported speedup for _make_child_relpath, though (the fourth benchmark from #112362 (comment)) -- if anything, it looks like this PR makes it slightly slower for me locally? (I'm on Windows with a fresh PGO-optimised build from main.)

Lib/test/test_pathlib.py Show resolved Hide resolved
@barneygale
Copy link
Contributor Author

I can't repro the reported speedup for _make_child_relpath, though (the fourth benchmark from #112362 (comment)) -- if anything, it looks like this PR makes it slightly slower for me locally? (I'm on Windows with a fresh PGO-optimised build from main.)

Hm. With a fresh PGO rebuild on Linux I still see an improvement, but it's really marginal - only 2% faster. I'll take the change out of the PR.

Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks!

@barneygale barneygale merged commit 19a1fc1 into python:main Nov 25, 2023
29 checks passed
@barneygale
Copy link
Contributor Author

Thank you for the review Alex!

aisk pushed a commit to aisk/cpython that referenced this pull request Feb 11, 2024
…python#112362)

Construct only one new list object (using `list.copy()`) when creating a
new path object with a modified tail. This slightly speeds up
`with_name()` and `with_suffix()`
Glyphack pushed a commit to Glyphack/cpython that referenced this pull request Sep 2, 2024
…python#112362)

Construct only one new list object (using `list.copy()`) when creating a
new path object with a modified tail. This slightly speeds up
`with_name()` and `with_suffix()`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Performance or resource usage topic-pathlib
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants