-
Notifications
You must be signed in to change notification settings - Fork 738
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
UTF-8 Encoding Errors Prevent Package Installation #7733
Comments
I'm sort of wondering if this is just a case of Windows long-path support not being enabled. |
I think it was enabled already, double checked and made sure it was, restarted, but still the same issue:
|
Unfortunately I haven't been able to replicate it on my Windows machine. |
I found a way to reproduce it on my computer.
When Windows username contains non-ASCII characters, the default uv cache directory will trigger this error. |
Right that is most likely the reason, as my username contains the |
Great find, thank you for the PR! |
## Summary This PR fixes #7733. According to [CPython documentation on `sys.stdout`](https://docs.python.org/3.12/library/sys.html#sys.stdout), when `stdout`/`stderr` is non-character device like pipe, the encoding will be set to system locale on windows. However, on the Rust side `stdout_reader` and `stderr_reader` expect them to be encoded in UTF-8 and will fail when child process write non-ASCII character to stdout/stderr, e.g., build directory name containing non-ASCII character. Both [CPython3](https://docs.python.org/3.12/using/cmdline.html#envvar-PYTHONIOENCODING) and [PyPy](https://doc.pypy.org/en/default/man/pypy3.1.html#environment) support environment variable `PYTHONIOENCODING`. When it is set to `utf-8`, python will use UTF-8 encoding for `stdin`/`stdout`/`stderr`. Since `stdin` is not used by the spawned python process and we expect `stdout`/`stderr` to use UTF-8, this fix should work as expected. <!-- What's the purpose of the change? What does it do, and why? --> ## Test Plan I only tested it on my computer with CPython 3.12 and 3.7. With the fix applied I confirmed that [the case I described](#7733 (comment)) is fixed. I'm using Windows 11 with system locale set to code page 936.
Summary
uv is failing to install packages, reporting UTF-8 encoding errors. The root cause is likely non-ASCII characters in the Windows username, which uv may not be handling correctly.
Steps to Reproduce
Environment
Impact
This issue prevents package installation, a core functionality of uv, significantly hindering development workflows for users with non-ASCII characters in their Windows usernames.
Additional Context
The root cause appears to be non-ASCII characters in the Windows username. For example, a user with the username "María García" might experience this issue, while a user with the username "John Smith" would not.
Example:
C:\Users\María García\AppData\Local\uv\cache\builds-v0\.tmp1pEQZM\Scripts\python.exe
C:\Users\John Smith\AppData\Local\uv\cache\builds-v0\.tmp1pEQZM\Scripts\python.exe
uv may not be correctly handling these non-ASCII characters in file paths, leading to the reported UTF-8 encoding errors.
Related Issues
uv pip install
failed on Windows:stream did not contain valid UTF-8
#7549Please let me know if you need any additional information or if you'd like me to perform any specific tests to confirm this hypothesis.
The text was updated successfully, but these errors were encountered: