watchman: huge closure with boost.dev dependence

[bryango]

Describe the bug

The watchman closure is huge:

$ nix path-info -Sh nixpkgs#watchman
/nix/store/6i3xggwm49wban90kz6w7xc4jvqqln1y-watchman-2023.01.30.00	 600.3M

due to the gcc & boost dependence:

$ nix path-info -rsh nixpkgs#watchman | sort -hk2 | tail -3
/nix/store/6i4vwgsiqp8jd8pb8cgf99sx8081rmsj-icu4c-73.2              	  37.7M
/nix/store/jhflvwr40xbb0xr6jx4311icp9cym1fp-python3-3.10.12         	  83.0M
/nix/store/b9574608lqarxg01fsz15kcxyrawzwzh-boost-1.81.0-dev        	 145.1M
/nix/store/1x1q5sqa0ilbi8fz7aayk02pjy5g7jhh-gcc-12.3.0              	 208.3M

It seems that the gcc dependency here is due to the python binding:

$ nix why-depends --all --precise nixpkgs#watchman nixpkgs#gccForLibs.outPath
/nix/store/6i3xggwm49wban90kz6w7xc4jvqqln1y-watchman-2023.01.30.00
    → /nix/store/1x1q5sqa0ilbi8fz7aayk02pjy5g7jhh-gcc-12.3.0
└───nix/store/jhflvwr40xbb0xr6jx4311icp9cym1fp-python3-3.10.12/lib/python3.10/site-packages/pywatchman/bser.cpython-310-x86_64-linux-gnu.so: …7-8-dev/include/bits./nix/store/1x1q5sqa0ilbi8fz7aayk02pjy5g7jhh-gcc-12.3.0/lib/gcc/x86_64-unkno…

Steps To Reproduce

Installing watchman adds another 406.06 MiB to my system, as reported by nix-tree.

Expected behavior

Note: updated to respond to some comments by aaronjheng in the PR linked below.

• Many runtime dependencies can be removed, as indicated by the AUR source & binary packages. However, I don't know how to do that...
• The upstream binary is tiny, with a size < 10 M. It is statically-linked though. However, when there is such a storage overhead, should we consider static linking as well? Or is there a nixpkgs policy against it? Sorry in advance if it's a noob question...

Notify maintainers

@cstrahan @kylesferrazza

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

$ nix-shell -p nix-info --run "nix-info -m" | sed -E "s|$HOME|\$HOME|g"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.41-1-MANJARO, Manjaro Linux, noversion, rolling`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.17.0`
 - nixpkgs: `$HOME/.nix-defexpr/channels/nixpkgs`

$ nix registry list | grep nixpkgs | grep user
user   flake:nixpkgs github:NixOS/nixpkgs/0bab81e6319eb418e1509fe4b88401a022f3bf99

[dcecile]

Checking with 2023.08.14.00, the GCC dependency is gone, bringing the package size down to 277.6M:

$ nix path-info -Sh nixpkgs#watchman 
evaluating derivation 'flake:nixpkgs#watchman'/nix/store/jsci9hsfbxiyvanr09y2jv1xmplxa6jd-watchman-2023.08.14.00	 277.6M

$ nix path-info -rsh nixpkgs#watchman | sort -hk2 | tail -3
/nix/store/9y8pmvk8gdwwznmkzxa6pwyah52xy3nk-glibc-2.38-27          	  28.8M
/nix/store/l77dphhwcw7j5k5kqxw1rqxbvhszgaw7-icu4c-73.2             	  37.7M
/nix/store/h1r04r2cay50i5k96b3miw27v8r9zwkr-boost-1.81.0-dev       	 145.1M

[bryango]

Awesome! I think the remaining issue is due to the boost.dev dependency (>100MiB), but this seems to be a long-standing problem: #45462. For watchman, the dependency chain is: watchman -> folly -> boost.dev:

$ nix why-depends --all --precise nixpkgs\#watchman nixpkgs\#boost.dev
/nix/store/jsci9hsfbxiyvanr09y2jv1xmplxa6jd-watchman-2023.08.14.00
└───bin/watchman: …z63zs4-fmt-8.1.1/lib:/nix/store/bjmm3lshlbv99rx8mldnbb1yzhl2kya2-folly-2023.02.27.00/lib:/nix/st…
    → /nix/store/bjmm3lshlbv99rx8mldnbb1yzhl2kya2-folly-2023.02.27.00
    └───lib/cmake/folly/folly-targets.cmake: …INCLUDE_DIRECTORIES "/nix/store/h1r04r2cay50i5k96b3miw27v8r9zwkr-boost-1.81.0-dev/include;/nix/s…
        lib/pkgconfig/libfolly.pc: …DGFLAGS_IS_A_DLL=0 -I/nix/store/h1r04r2cay50i5k96b3miw27v8r9zwkr-boost-1.81.0-dev/include -I/nix…
        → /nix/store/h1r04r2cay50i5k96b3miw27v8r9zwkr-boost-1.81.0-dev

Update: ok, I think I figured it out: the dep folly should be a split package just like boost. We should probably add:

nixpkgs/pkgs/development/libraries/boost/generic.nix

Lines 266 to 267 in 006b631

	outputs = [ "out" "dev" ];
	setOutputFlags = false;

I will try to do a PR when I am free.

Update: #278643 merged into staging, will be auto closed when it flows into master.