] build PyCall does not work on NixOS
#952
Comments
|
Update this is reproducible even after deleting |
|
Installing Conda.jl does not actually install the distro — that only happens when a user, or a package like PyCall, first requests that something be installed. e.g. try: import Conda
Conda.add("numpy")That being said, it shouldn't need to install Conda at all — by default on Unix, PyCall tries to use the system Python, and apparently that is failing. What happens if you try python3 -c "import distutils.sysconfig; print(distutils.sysconfig.get_config_var('VERSION'))"in your terminal? |
|
I get I think the weird thing is that it's trying to run the wrong python3: |
|
FWIW part of the magic of nix-shell is that I'm freed from having a global python install across the whole system. That may be part of what's tripping it up. |
|
PyCall just searches the The workaround is simply to set ENV["PYTHON"]="/nix/store/rppr9s436950i1dlzknbmz40m2xqqnxc-python3-3.9.9/bin/python3"
Pkg.build("PyCall")to specify the particular path that you want. (You only need to do this once and PyCall will remember the path subsequently.) |
|
Here's my which includes
How does PyCall remember this path? Is it remembered per Julia session or forever or something else? Perhaps it's remembering an incorrect path? |
|
I did a little more poking around and I think I better understand the issue now. The problem is that part of PyCall's Because this file lives "forever" in |
The built file is now `Pkg.dir("PyCall","deps","deps.jl")` as discovered in the investigation of JuliaPy#952.
|
I just recently wanted to make a Julia project using a Nix shell and ran into this problem. I was able to solve it by just hacking the |
|
Thanks, @usmcamp0811! Here's my shell for https://github.com/lambdaclass/julia_koans in case it helps someone else. let
python-env = python3.buildEnv.override {
extraLibs = [
python3Packages.numpy
];
};
julia-env = julia.withPackages.override { extraLibs = [ python-env ]; } [
"PythonCall"
"Test"
];
in
mkShell {
env = {
PYTHONPATH = "${python-env}/lib/python${lib.versions.majorMinor (lib.getVersion python3)}/site-packages";
};
nativeBuildInputs = [
julia-env
python-env
];
} |
I'm running into some unexpected hiccups building PyCall.jl on NixOS. I have successfully used PyCall.jl on NixOS in the past, so I'm a bit stumped as to why it's borked now. I get the following failure:
It appears to be hung up on some aspect of the conda installation. However, building Conda.jl works without any issues:
A few things are weird to me about this situation:
cannot execute /home/skainswo/.julia/conda/3/conda.exe: NIX_LD or NIX_LD_x86_64-linux is not setsuggests to me that Conda.jl is smart enough to use nix-ld when it's available, but I haven't been able to find anything about this in the docs or in the sources for Conda.jl or PyCall.jl. Where is this warning coming from?/nix/store/nki9ywqzbvz68vr75kn2r7g1q84f5agy-python3-3.9.6/bin/python3 -c \"import distutils.sysconfig; print(distutils.sysconfig.get_config_var('VERSION'))\", but this is very odd since I don't even have that version of python installed! I have python 3.9.9 with a different hash entirely! How is it hallucinating this python install that does not exist?The main thing that has changed since the last time this worked for me is that I upgraded NixOS from 21.05 to 21.11.
Here's my shell.nix for reference:
and here's the version of python that's actually available:
The text was updated successfully, but these errors were encountered: