packages-config: refactor and simplification

[Atemu]

cc @adisbladis @davidak

Motivation for this change

Fixes #110348

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

[davidak]

I don't understand why this is needed at all.

Cant we just say: import any package from pkgs/

There is the thing that Nix don't know about packages. Would it help to add such a concept?

And even when there are good reasons why that don't work, we have all packages collected in pkgs/top-level/all-packages.nix, why do we have to list them here a second time?

It all feels like bad design. More complicated than needed, which causes errors like this over and over again.

Do you mind explaining it shortly for my understanding?

[Atemu]

Nixpkgs is a tree, not a list; "import any package from pkgs/" isn't so simple. It's similar to listing all files in a FS; you have recurse into subfolders and aggregate a list of all the files you find there.
The problematic bit are these package subsets, not packages.

There is currently no way of knowing whether an attr of pkgs is a package or a subset of packages. (..or a function, broken or even evaluable, see #107539)

We need to tell Nix which attrs of pkgs are subsets and should be recursed into by setting set.recurseIntoAttrs on all of them which is precisely what this file does.
The problem we face is that we now have sub-subsets too which need to be declared somehow.

[Atemu]

/marvin opt-in
/status needs_reviewer

[marvin-mk2]

Hi! I'm an experimental bot. My goal is to guide this PR through its stages, hopefully ending with a merge. You can read up on the usage here.

[marvin-mk2]

Reminder: Please review!

This Pull Request is awaiting review. If you are the assigned reviewer, please have a look. Try to find another reviewer if necessary. If you can't, please say so. If the status is not accurate, please change it. If nothing happens, this PR will be should be put back in the needs_reviewer queue in one day.

---

Note: This feature is currently broken. The bot will not actually change the status. If you see this message multiple times, please request a status change manually.

[davidak]

Result of nixpkgs-review pr 113713 run on x86_64-linux 1

[davidak]

I built packages.json using

[nix-shell:~/.cache/nixpkgs-review/pr-113713-1/nixpkgs]$ nix-build pkgs/top-level/release.nix -A tarball

and compared it to latest master https://hydra.nixos.org/build/149921732#tabs-summary

number of packages

PR 113713:

[davidak@gaming:~/Downloads]$ jq '.packages | length' packages.json
76881

master:

[davidak@gaming:~/Downloads]$ jq '.packages | length' packages.json
71674

diff

[davidak@gaming:~/Downloads]$ jq '.packages | keys' packages.json > packages.txt

[davidak@gaming:~/Downloads]$ jq '.packages | keys' packages-master.json > packages-master.txt

[davidak@gaming:~/Downloads]$ diff packages-master.txt packages.txt

https://gist.github.com/davidak/daca412ac5557e5fafa75726f241bbc9/edit

So this PR does not break the previous behavior and also adds emacs27Packages back. But that syntax is deprecated in #107152.

The previous versioned names emacsPackages have been moved to
aliases.nix and are now considered deprecated in favour of emacs.pkgs.

cc @adisbladis

[Atemu]

emacsPackages still exists as an alias.

[davidak]

emacsPackages have been moved to aliases.nix and are now considered deprecated in favour of emacs.pkgs.

so i think the packages should show up as emacs.pkgs.x in the search

see the documentation changes in https://github.com/NixOS/nixpkgs/pull/107152/files

@adisbladis maybe you can clarify?

[Atemu]

I think we should merge this as-is until we find a better solution.

I tried recursing into emacs.pkgs but nix-env doesn't seem to want to recurse into a drv which seems sensible.

[davidak]

I agree. It looks cleaner (less code) and fixes an issue.

I can't evaluate if the code is good, but i tested it and it does what it should.