Rename IFD (import from derivation) to something more appropriate

[fricklerhandwerk]

Problem

It's not always import and not always from derivation

Proposal

We may want to do it rather sooner than later, as there will be more to change the longer we wait.

@roberth: evaluation-time build dependency
@fricklerhandwerk: the mechanism is "reading build outputs during evaluation"

Priorities

Add 👍 to issues you find important.

[roberth]

The new term should be easy to understand both in an evaluator context and in a scheduling context.

• scheduling context: E.g. in logs and in CI the context is "this had to be built before we could continue", whereas in an
• evaluator context: it's all about the actual data and where it came from (from a build / from an output).

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-06-05-nix-team-meeting-minutes-60/28933/1

[fricklerhandwerk]

@NobbZ had a proposal that seems to satisfy requirements and sounds concise enough: read from realisation (RFR)

[infinisil]

I proposed this in a previous docs team meeting: "Nested evaluation", and I think this is much easier to read and understand than "read from realisation"

[Ericson2314]

Isn't it nested building not evaluation?

Realization for evaluation, building for evaluation. I'm starting to like "for evaluation" a lot.

[NobbZ]

"Nested evaluation"

Isn't that quite easy to confuse with the experimental feature "recursive-nix"?

We are already in a situation where some poeple are using "recursive nix" as an umbrella term that includes IFD and the experimental feature recursive-nix. Adding yet another term to the mix, that sounds more like an umbrella/category to refer to different techniques that achieve a similar goal is probably counter productive.

It is not that I want to push my own suggestion, I just threw it in for brainstorming in the discourse and got far further with it than expected, it is much more that I'd like to see terms that are distinguishable enough to make clear we are talking about different things!

They do not need to be technically correct, they need to be understood. Thats why I still consider "IFD"/"Import from derivation" as the ideal thing. We use it like that for many years, and it is well understood, there are a lot of third party ressources out there that already refer to it by that term. Renaming it just creates confusion.

[infinisil]

"Nested evaluation"

Isn't that quite easy to confuse with the experimental feature "recursive-nix"?

Good point. These features are actually similar, I wonder if it could make sense to think of them as the same core idea and use terms for that convey that. (Recursive Nix is a good name, but we shouldn't hold back from renaming it if necessary, since it's still an experimental feature).

Thats why I still consider "IFD"/"Import from derivation" as the ideal thing.

Renaming creates confusion for pre-existing knowledge and resources, but makes things easier to understand for everybody in the future. And with Nix continuing its growth, the fraction of the former becomes less and less, which makes me think it could be worth the trouble.

[infinisil]

Thinking about this again, we can call both "Recursive Nix", but to distinguish we can use:

• Eval-time recursive Nix (current IFD): Does a nested build/eval cycle during evaluation
• Build-time recursive Nix (current recursive-nix): Does an extra eval/build cycle during building

@NixOS/nix-team Can you confirm whether it would make sense to have this naming?

[thufschmitt]

https://github.com/orgs/NixOS/teams/nix-team Can you confirm whether it would make sense to have this naming?

There's indeed a symmetry between IFD and recursive-Nix, but calling them nearly the same feels quite confusing. I'd rather have some frankly orthogonal namings

[roberth]

• Eval-time recursive Nix (current IFD): Does a nested build/eval cycle during evaluation

Recursion within what? How is the nesting defined?

In recursive-nix, we have instantiation and realisation happening from within the sandbox. If one of these realisations involves recursive-nix we have at least two distinct sandbox processes that are conceptually contained within one another.

In IFD, if we want nesting, how do we achieve this? Not by performing IFD during the realisation, because that would lead us to recursive-nix instead. So it has to be before or after the realisation - the same as any other composition of operations in the language. Nix is a recursive language: you can invoke functions that invoke functions produced by functions, and any recursion around IFD is just the recursion we already have, even without IFD.

So the recursion in "eval-time recursive Nix" has to refer to the recursion in the language, which is also present without IFD. This is bad.

When I imagine a conversation with a beginner, they ask "doesn't Nix support recursion anyway, during evaluation"? Then I'd have to say: but it also says "Nix", which is really not useful at all because the language is also called Nix.
Then I'd have to stop explaining the term but instead explain the need to do a realisation during the evaluation, after which they'll ask "Nix produces builds all the time. Why is this recursive? How is it nested?" at which point I'd have to say you have to use the recursion (functions calling other functions) that's already in the language to make it behave in a recursive manner.

[fricklerhandwerk]

• @tomberek: since with RFC 92 IFD won't be as important, maybe renaming is not worth the while
  • @Ericson2314: disagree, the features are orthogonal
• @edolstra: RFR is not bad, but people usually won't know what realisation is either.
  • @Ericson2314: we can link to the definition in the glossary, this works now
• agreement to proceed with renaming, assigned to @fricklerhandwerk

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-08-28-nix-team-meeting-minutes-83/32290/1

[infinisil]

Very related comment by @Ericson2314: #8094 (review)

[asymmetric]

@fricklerhandwerk I see the Nix team agreed to proceed with the renaming, but renaming to what?

Also, is this blocked on something, or can the renaming go ahead?

[fricklerhandwerk]

AFAICT there are no blockers and there is at least silent (that is, unenthusiastic) agreement to proceed with "read from realisation".

Just a few words of caution in case you want to start with opening PRs: We should be extra careful with adding redirects and release notes, and try to orchestrate the change across the various resources as far as possible. Still there will almost inevitably be a transition period with potential for confusion.

[roberth]

For what it's worth, RFR is ambiguous with Request For Review.
Might "read from derivation output" be an improvement?
It's also a bit more accurate. The acronym should refer to the process of realisation, not as a noun, because otherwise it could be misunderstood to refer to a realisation object (e.g. a CA .doi), which does not contain the store path contents. For a fixed output derivation, you could conceivably construct a realisation object without having the store path contents.

[roberth]

EBEB was coined in an RFC discussion at some point.

Source NixOS/nixpkgs#273110 (comment)

I don't like it, because the last B is often irrelevant. It creates a situation where we have two distinct terms for the same phenomenon, which is EBE. Case in point: nix eval --expr 'readFile (writeText "foo" "bar")' is IFD, but not EBEB.
Having two terms makes the acronym harder to search for and slightly more work to learn and recognize as mostly the same concept. I don't think the extra descriptiveness compensates this cost.

[roberth]

Brainstorming:

• Ad hoc realisation
• Unanticipated realisation (an epiphany?... 🙈 )

[Qyriad]

What about "realization-dependent evaluation" (RDE), or "realization-blocked evaluation" (RBE)?

[zimbatm]

Unless something really good stands out, we could also accept "IFD" as a historical artefact, and spend the time documenting it instead.

This would avoid having to rename the allow-import-from-derivation config option. And general churn where we will still have to explain to users what IFD stands for when they read old references.

From my experience, the issue is not the naming but all the surprising places where it can pop up. It's surprisingly easy to trigger it without thinking about it too much.

[roberth]

We have a dedicated page about IFD now, so we're already defaulting to this.
I agree it's an acceptable state of affairs and has those advantages.

[roberth]

Have to admit that RDE sounds cool though.

What's that awful high pitched noise? Just evaluating....