Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why does the existance of an output file speed up resolution? #2983

Closed
notatallshaw opened this issue Apr 10, 2024 · 2 comments
Closed

Why does the existance of an output file speed up resolution? #2983

notatallshaw opened this issue Apr 10, 2024 · 2 comments
Labels
performance Potential performance improvement

Comments

@notatallshaw
Copy link
Contributor

notatallshaw commented Apr 10, 2024

Python 3.10 uv 0.1.31

Create a requitement.in file as such:

click >= 7.0
click-loglevel ~= 0.2
dandi >= 0.24.0
psutil ~= 5.9
pyyaml
selenium

No compile these requirements without cache to an output file, then again with the existing output file, then again without the output file:

$ time uv pip compile requirements.in --exclude-newer 2023-10-01T00:00:00Z --no-cache -o test_out.txt --quiet
real    0m9.926s
user    0m6.522s
sys     0m1.117s

$ time uv pip compile requirements.in --exclude-newer 2023-10-01T00:00:00Z --no-cache -o test_out.txt --quiet
real    0m1.334s
user    0m0.575s
sys     0m0.177s

$ rm -rf test_out.txt

$ time uv pip compile requirements.in --exclude-newer 2023-10-01T00:00:00Z --no-cache -o test_out.txt --quiet
real    0m9.899s
user    0m6.395s
sys     0m1.226s

What's going on here? Why would a file that you write to speed up resolution?

This might not be a bug but it feels wrong, it should be at least documented in the help which currently just says:

  -o, --output-file <OUTPUT_FILE>
          Write the compiled requirements to the given `requirements.txt` file
@AlexWaygood AlexWaygood added the performance Potential performance improvement label Apr 10, 2024
@charliermarsh
Copy link
Member

Because we use the existing resolution in the output file as the starting point for the subsequent resolution, unless you run with the upgrade flag.

@notatallshaw
Copy link
Contributor Author

notatallshaw commented Apr 10, 2024

Okay, I tested and this seems to be the same behavior as pip-compile:

$ time pip-compile requirements.in --pip-args "--no-cache-dir" --output-file test_out2.txt --quiet --strip-extras
real    0m30.315s
user    0m12.760s
sys     0m0.456s

$ time pip-compile requirements.in --pip-args "--no-cache-dir" --output-file test_out2.txt --quiet --strip-extras
real    0m21.981s
user    0m7.968s
sys     0m0.471s

$ rm test_out2.txt
$ time pip-compile requirements.in --pip-args "--no-cache-dir" --output-file test_out2.txt --quiet --strip-extras
real    0m29.986s
user    0m12.799s
sys     0m0.400s

I even tested that it doesn't upgrade a valid solution, I just find this very unintuitive, as it's called an output file not a sync file, but it matches the tool you are mirroring so ¯\(ツ)/¯.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Potential performance improvement
Projects
None yet
Development

No branches or pull requests

3 participants