ENH: Improve stdlib-handling

[h-vetinari]

Checklist

• I added a descriptive title
• I searched open requests and couldn't find a duplicate

What is the idea?

#4999 added the {{ stdlib("c") }} function, and I'm working on rolling this out across conda-forge (chiefly in regro/cf-scripts#2135). However, I hit a road-block when I realized that smithy cannot rerender recipes using that new jinja function, in the sense that the configs for the respective CI jobs don't get the right c_stdlib{,_version} keys populated.

This is apparently all coming from conda-build, and so we need to fix it here first.

Why is this needed?

Necessary to satisfy the goals of #4981, and be able to reflect the fact that many packages now have newer requirements on the C-stdlib than the long-running defaults currently still active in conda-forge (both on macos and soon linux too).

What should happen?

• Determine what needs to change in conda-build
• Do the change
• Write tests

Additional Context

No response

[h-vetinari]

Determine what needs to change in conda-build

One question in particular is if conda-build wants to set defaults for the stdlib (as it does for the compilers already), and whether those match the naming of conda-forge.

OS	stdlib metapackage in conda-forge	stdlib metapackage in default
linux	sysroot_{{ platform }}	Same setup as conda-forge
osx	macosx_deployment_target_{{ platform }}	feedstock not (yet?) mirrored
win	vs_{{ platform }}	Anaconda's vc has not yet integrated a version-less vs (see this commit)

Anaconda will similarly need to push the c-stdlib lower bounds on several key packages (e.g. macos 10.13 for anything related to abseil, protobuf, etc.), so presumably similar tools will be desirable. For example, Ray created an issue broadly related to this a long time ago, though here we're not talking about distributing the SDK, only having a metapackage that has a run-export on __osx >=x.y (and similarly for linux; for windows see here).

CC @jezdez @kenodegard @chenghlee @beeankha @mbargull

PS. AFAIU, having a default is not necessary for conda-forge, so this decision could be deferred - we only need to detect the use of the stdlib jinja (in conjunction with the right CBC keys). My question arises mainly from looking at variant.py and the prominent position compilers have in find_used_variables_in_text, resp. whether the stdlib setup should follow that.

[mbargull]

• Determine what needs to change in conda-build
• Do the change
• Write tests

N.B.: "Write tests" should be the first or second, not last point here (assuming order)...

One question in particular is if conda-build wants to set defaults for the stdlib (as it does for the compilers already), and whether those match the naming of conda-forge.

We should not add more channel-/package-specific defaults.
I'd say the already present defaults are just tech-debt and rather than thinking about adding more defaults, we should think about adding an option like no-default-config to uncover where these defaults may introduce unwanted behavior.

---

I'll look into fixing the tests for that feature and a fix; should not take too long..

[h-vetinari]

We should not add more channel-/package-specific defaults. [...]

That's a completely fair assessment, but not something that was easy to tell just from looking at the current setup. 😅

Really appreciate you taking the time to write up a fix!