Specifying llvm-openmp only in build leads to a run_exports against the latest llvm-openmp

[xhochy]

If a user does not specify llvm-openmp in build and host but only in build, it will lead to packages that will have a pin of llvm-openmp >=latest-release. The pin will be independent of the llvm-openmp version used in build.

This is due to the fact that conda-build takes the strong run_exports from build and implicitly adds them to the host environment, see https://github.com/conda/conda-build/blob/9c0ceec8d96d265e33d960c46024db78f4e3efd8/conda_build/render.py#L451 This then leads to the latest version of llvm-openmp being installed into host and its run_export taken into account. My guess is that the actual line adding them is https://github.com/conda/conda-build/blob/9c0ceec8d96d265e33d960c46024db78f4e3efd8/conda_build/render.py#L489

This is then annoying if you want to build something with the compilers pinned as in conda-forge, but outside a conda-build setting where you only have a single environment.

I'm not sure whether we should correct this on the conda-build side or in the feedstock here. For other situations like libgcc-ng this is not a problem as libgcc-ng itself doesn't have a run_export.

[jakirkham]

Yeah we had a discussion about the need for host_exports in the past

IOW an operation that adds a dependency to host when that package is in build

The original use case involved compiler libraries that need to be statically linked (so must be in build and host). However are not needed in run

This sounds like another case where we need this. Namely we want a tighter coupling between build and host. Though we want a laxer constraint in run

[isuruf]

This is then annoying if you want to build something with the compilers pinned as in conda-forge, but outside a conda-build setting where you only have a single environment.

What do you mean by this?

[isuruf]

or other situations like libgcc-ng this is not a problem as libgcc-ng itself doesn't have a run_export.

IMO, libgcc-ng run_export is wrong. It should have a run_export. If so, we won't have to have the hack at
https://github.com/conda-forge/ctng-compiler-activation-feedstock/blob/main/recipe/meta.yaml#L3-L14

[xhochy]

This is then annoying if you want to build something with the compilers pinned as in conda-forge, but outside a conda-build setting where you only have a single environment.

What do you mean by this?

We have an environment where I would like to use the same pinnings as conda-forge, i.e., clang and llvm-openmp are pinned to 16. As this is a simple development environment, all dependencies (i.e. compilers and native deps) are installed in the same environment. This only works if the packages don't require a newer llvm-openmp, sadly with the documented approach (i.e. only having llvm-openmp in build) all packages that were recently built have a llvm-openmp>=18.x pin.

[mbargull]

We've hit this now in the R 4.4 migration:
r-base has llvm-openmp in requirements/build, pinned to 16 via conda-forge-pinning.
llvm-openmp added to host through strong run_exports is llvm-openmp>=16.0.6 which now is llvm-openmp=18.1.8.
Which in turn means r-base=4.4.1 has dependencies

- llvm-openmp>=16.0.6
- llvm-openmp>=18.1.8

(i.e., llvm-openmp>=18.1.8 effectively).
This does not seem ideal since it results in an overly restrictive dependency (at this point in time).
If, e.g., we'd have to mark llvm-openmp=18.1.8 (or whatever current version) as broken, we'd break all newly built packages that had llvm-openmp in requirements/build.

(I only got aware of this issue since https://github.com/conda-forge/r-data.table-feedstock/blob/807a816c953af4ddabaa591366f79f38b88d7eb5/recipe/meta.yaml#L36 has it in requirements/host instead which in turn made it unresolvanble for r-base=4.4 since it requests llvm-openmp=16 as per conda-forge-pinning.)

[mbargull]

Another case is conda-forge/r-rcpparmadillo-feedstock#72 :
There, we have a cross-compilation build for osx-arm64 and as such we have cross-r-base=4.4 in requirements/build which pulls in r-base=4.4.1 which in turn pulls in the llvm-openmp>=18.1.8 constraint which clashes with the llvm-openmp=16 requested due to pinning.

Meaning, our osx-arm64 builds for recipes that use llvm-openmp and require packages built against llvm-openmp in recent times are broken.

[isuruf]

I think we should have llvm-openmp and libgomp only in host. i.e. re build r-base with those in host only.

[isuruf]

Or in both build and host. Not just in build.

[mbargull]

Yes, we have to have them in host.

(Apart from cross-compilations) what were the reasons to have them in build?
Was it that Clang needed the header in lib/clang/${clang_version}/include/ or what?
Or do the compilers prefer link path in their prefix?
I honestly can't remember.

[mbargull]

Would it suffice to add strong_constrains run_exports to the compilers and move the OMP deps to host in general?

[xhochy]

That should help.