Weirdly hard to install a simple pure-python plugin

[jni]

In the process of testing the napari 0.5.4rc3 macOS bundle, I tried to install affinder. Conda is still going so I'm making this issue asynchronously while that's going. It's unclear whether it's going to finish. 😂 But anyway, here's the list of affinder dependencies:

https://github.com/conda-forge/affinder-feedstock/blob/c16758e7b4cde7303aba68096c216a75e4b7fca5/recipe/meta.yaml#L25-L31

These are all things that should be satisfied a priori by the bundle, I believe? But the solve takes forever, then fails:

Starting '/Users/jni/Library/napari-0.5.4rc3/bin/conda' with args ['install', '-y', '--prefix', '/Users/jni/Library/napari-0.5.4rc3/envs/napari-0.5.4rc3', '--override-channels', '-c', 'conda-forge', 'affinder==0.4.0']
Retrieving notices: ...working... 
done

Collecting package metadata (repodata.json): ...working... 
done

Solving environment: ...working... 
unsuccessful initial attempt using frozen solve. Retrying with flexible solve.

Solving environment: ...working... 

Found conflicts! Looking for incompatible packages.

I await the result of the conflicts analysis and will update here when it's completed.

Is it easy to inspect what the package specification is for the bundle?

[jni]

Full log:

https://gist.github.com/jni/99a123459b3737ffeabca8571a25a014

[Czaki]

Why we use conda, not mamba in bundle?

[jni]

(Seems like that should be a different issue @Czaki ;)

[jni]

Is it easy to inspect what the package specification is for the bundle?

👆 and I asked this so I could check whether mamba/micromamba would give different results here. btw afaik conda now uses libmamba to solve so the solve should not yield different results. I don't know whether I'm right about this though.

[Czaki]

(Seems like that should be a different issue @Czaki ;)

My solution for such problem in the past was to switch from conda to mamba (and now I will think about try pixi).

[jaimergp]

Hm, those logs are coming from the classic solver. Unclear why...

Can you paste the output of:

/Users/jni/Library/napari-0.5.4rc3/bin/conda info
/Users/jni/Library/napari-0.5.4rc3/bin/conda config --show-sources
/Users/jni/Library/napari-0.5.4rc3/bin/conda list -p /Users/jni/Library/napari-0.5.4rc3

[jni]

/Users/jni/Library/napari-0.5.4rc3/bin/conda info

     active environment : /Users/jni/micromamba/envs/all
    active env location : /Users/jni/micromamba/envs/all
            shell level : 1
       user config file : /Users/jni/.condarc
 populated config files : /Users/jni/Library/napari-0.5.4rc3/.condarc
                          /Users/jni/.condarc
          conda version : 23.9.0
    conda-build version : not installed
         python version : 3.9.20.final.0
       virtual packages : __archspec=1=m1
                          __osx=14.5=0
                          __unix=0=0
       base environment : /Users/jni/Library/napari-0.5.4rc3  (writable)
      conda av data dir : /Users/jni/Library/napari-0.5.4rc3/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/napari/osx-arm64
                          https://conda.anaconda.org/napari/noarch
                          https://conda.anaconda.org/conda-forge/osx-arm64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /Users/jni/Library/napari-0.5.4rc3/pkgs
                          /Users/jni/.conda/pkgs
       envs directories : /Users/jni/Library/napari-0.5.4rc3/envs
                          /Users/jni/.conda/envs
               platform : osx-arm64
             user-agent : conda/23.9.0 requests/2.32.3 CPython/3.9.20 Darwin/23.5.0 OSX/14.5
                UID:GID : 501:20
             netrc file : None
           offline mode : False

/Users/jni/Library/napari-0.5.4rc3/bin/conda config --show-sources

==> /Users/jni/Library/napari-0.5.4rc3/.condarc <==
auto_update_conda: False
notify_outdated_conda: False
env_prompt: [napari]({default_env})
channel_priority: strict
channels:
  - napari
  - conda-forge
repodata_fns:
  - repodata.json

==> /Users/jni/.condarc <==
channels:
  - conda-forge

/Users/jni/Library/napari-0.5.4rc3/bin/conda list -p /Users/jni/Library/napari-0.5.4rc3

# packages in environment at /Users/jni/Library/napari-0.5.4rc3:
#
# Name                    Version                   Build  Channel
archspec                  0.2.3              pyhd8ed1ab_0    conda-forge
boltons                   24.0.0             pyhd8ed1ab_0    conda-forge
brotli-python             1.1.0            py39hfa9831e_2    conda-forge
bzip2                     1.0.8                h99b78c6_7    conda-forge
c-ares                    1.33.1               hd74edd7_0    conda-forge
ca-certificates           2024.8.30            hf0a4a13_0    conda-forge
certifi                   2024.8.30          pyhd8ed1ab_0    conda-forge
cffi                      1.17.1           py39h7f933ea_0    conda-forge
charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
conda                     23.9.0           py39h2804cbe_2    conda-forge
conda-package-handling    2.3.0              pyh7900ff3_0    conda-forge
conda-package-streaming   0.10.0             pyhd8ed1ab_0    conda-forge
cpp-expected              1.1.0                hffc8910_0    conda-forge
cryptography              43.0.1           py39hc182a1d_0    conda-forge
fmt                       11.0.2               h420ef59_0    conda-forge
h2                        4.1.0              pyhd8ed1ab_0    conda-forge
hpack                     4.0.0              pyh9f0ad1d_0    conda-forge
hyperframe                6.0.1              pyhd8ed1ab_0    conda-forge
icu                       75.1                 hfee45f7_0    conda-forge
idna                      3.10               pyhd8ed1ab_0    conda-forge
jsonpatch                 1.33               pyhd8ed1ab_0    conda-forge
jsonpointer               3.0.0            py39h2804cbe_1    conda-forge
krb5                      1.21.3               h237132a_0    conda-forge
libarchive                3.7.4                h83d404f_0    conda-forge
libcurl                   8.9.1                hfd8ffcc_0    conda-forge
libcxx                    19.1.0               ha82da77_0    conda-forge
libedit                   3.1.20191231         hc8eb9b7_2    conda-forge
libev                     4.33                 h93a5062_2    conda-forge
libffi                    3.4.2                h3422bc3_5    conda-forge
libiconv                  1.17                 h0d3ecfb_2    conda-forge
libmamba                  2.0.0                h66a2e1b_1    conda-forge
libnghttp2                1.58.0               ha4dd798_1    conda-forge
libsolv                   0.7.30               h6c9b7f8_0    conda-forge
libsqlite                 3.46.1               hc14010f_0    conda-forge
libssh2                   1.11.0               h7a5bd25_0    conda-forge
libxml2                   2.12.7               h01dff8b_4    conda-forge
libzlib                   1.3.1                hfb2fe0b_1    conda-forge
lz4-c                     1.9.4                hb7217d7_0    conda-forge
lzo                       2.10              h93a5062_1001    conda-forge
mamba                     2.0.0                h6425b9e_1    conda-forge
ncurses                   6.5                  h7bae524_1    conda-forge
nlohmann_json             3.11.3               h00cdb27_1    conda-forge
openssl                   3.3.2                h8359307_0    conda-forge
packaging                 24.1               pyhd8ed1ab_0    conda-forge
pip                       24.2               pyh8b19718_1    conda-forge
pluggy                    1.5.0              pyhd8ed1ab_0    conda-forge
pycosat                   0.6.6            py39h0f82c59_0    conda-forge
pycparser                 2.22               pyhd8ed1ab_0    conda-forge
pyopenssl                 24.2.1             pyhd8ed1ab_2    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
python                    3.9.20          h9e33284_0_cpython    conda-forge
python_abi                3.9                      5_cp39    conda-forge
readline                  8.2                  h92ec313_1    conda-forge
reproc                    14.2.4.post0         h93a5062_1    conda-forge
reproc-cpp                14.2.4.post0         h965bd2d_1    conda-forge
requests                  2.32.3             pyhd8ed1ab_0    conda-forge
ruamel.yaml               0.17.40          py39h17cfd9d_0    conda-forge
ruamel.yaml.clib          0.2.8            py39h17cfd9d_0    conda-forge
setuptools                75.1.0             pyhd8ed1ab_0    conda-forge
simdjson                  3.10.1               h7b3277c_0    conda-forge
spdlog                    1.14.1               h6d8af72_1    conda-forge
tk                        8.6.13               h5083fa2_1    conda-forge
tqdm                      4.66.5             pyhd8ed1ab_0    conda-forge
tzdata                    2024a                h8827d51_1    conda-forge
urllib3                   2.2.3              pyhd8ed1ab_0    conda-forge
wheel                     0.44.0             pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h57fd34a_0    conda-forge
yaml-cpp                  0.8.0                h13dd4ca_0    conda-forge
zstandard                 0.23.0           py39hcf1bb16_1    conda-forge
zstd                      1.5.6                hb46c0d2_0    conda-forge

[jaimergp]

I think I know why: the base env in the installer is being solved with mamba 2.0, which is incompatible with conda-libmamba-solver, so we are getting an old conda (23.9), which is the latest that didn't depend on conda-libmamba-solver by default. The solver chooses to drop conda-libmamba-solver to optimize for mamba 2.0. This should be fixable in napari/packaging. Will submit a PR now.

Additionally, we are not finding mamba under condabin/ because 2.0 only puts it under bin/. Adding a PR for that in mamba-feedstock: conda-forge/mamba-feedstock#251

[jaimergp]

napari/packaging#187 is the fix, ready for review. You can see the conda list output now in the CI logs too, which confirms conda-libmamba-solver is installed and in use.

[psobolewskiPhD]

If I'm reading that correctly, affinder uses python 3.10 as minimum and bundle 3.9?
Should we bump the bundle up?

[jaimergp]

affinder uses python 3.10 as minimum and bundle 3.9?

That's a separate issue, I think. We can of course bump it up, but let's collect feedback about it first.

@jni, the packaging fix is up. What to do next:

1. Rename the installers in the releases page to something else, like -broken suffix or something.
2. Rerun the Bundle * CI jobs for the rc3 tag
3. Test the new installers
4. Delete the -broken ones

[Czaki]

I have added 0.5.4rc4 to create ne release

[jni]

@jaimergp let's definitely bump Python though, 3.9 is ancient! I'd go for 3.11 or 3.12 these days.

[jaimergp]

See napari/packaging#189 and let's discuss policy there.