Add /p/73

_posts/2024/2024-09-02-python3.12-user-packages.md

@@ -0,0 +1,36 @@
+---
+title: "Make Python 3.12 install user packages without complaints"
+tags: linux python
+redirect_from: /p/73
+---
+
+I have a habit of `pip3 install --user` and then expecting these packages under `~/.local/lib/` to be available for my Python scripts whenever I need them. However, with PEP 668 landing in Python 3.12, I now have to add `--break-system-packages` even for *user* packages. This is super annoying considered that I have multiple projects sharing the same set of common packages (e.g. [`mkdocs-material`](https://squidfunk.github.io/mkdocs-material/), a nice MkDocs theme). So time to tell `pip` to jerk off on that complaint.
+
+Obviously, aliasing `pip3` (as per my personal habit, I always prefer `python3` and `pip3` over `python` and `pip`) to `pip3 --break-system-packages` could work, with all the limitations that any other shell alias bear.
+
+The key here is, by examining how virtual environments work, we can trick Python into thinking that `~/.local` is one of them. This is already documented in the [`site` package][site]:
+
+  [site]: https://docs.python.org/3/library/site.html
+
+> If a file named `pyvenv.cfg` exists one directory above `sys.executable` ...
+
+So here's the solution, assuming `~/.local/bin` is already in your `$PATH`:
+
+1. Symlink `/usr/bin/python3` to `~/.local/bin/python3`
+2. Copy `/usr/bin/pip3` to `~/.local/bin/pip3`, and change the shebang line to `#!/home/example/.local/bin/python3` (you'll have to use the absolute path here, though).
+3. Create `~/.local/pyvenv.cfg` with just one line of content:
+
+    ```ini
+    include-system-site-packages = true
+    ```
+
+    You can, of course, add other settings for `venv`, which is completely optional and up to you.
+
+Now whenever you install something with `pip3`, it'll happily install it under `~/.local/lib/python3.12/site-packages` even without the need for `--user`.
+
+If you prefer `python` or `pip` commands, you can just change the file names in the above steps accordingly.
+
+Noteworthy points are:
+
+1. This certainly will work for the system Python installation (at `/usr`), which I wouldn't recommend for obvious reasons. If you insist, you should at least do this under `/usr/local` instead.
+2. This method is ineffective against scripts already shebanged with `#!/usr/bin/python3`. Consider developing the habit of running `python3 script.py` instead of `./script.py` like I do.