stdenv fails to build on darwin

[gilligan]

Issue description

stdenv fails to build on darwin

Steps to reproduce

nix-build -A stdenv

Technical details

% nix-build $NIXPKGS -A stdenv                                                                                                                                                1 ↵
these derivations will be built:
  /nix/store/4hqi93a7cf33xh90ykncl0pjaxpsigvp-stdenv-darwin-boot-2.drv
  /nix/store/16f6zmzvqlj3l4r8lc65lzl1pq0gmyzd-libc++abi-3.7.1.drv
  /nix/store/211cqsj45qhhhg3cnijdklxyy4qaz1b5-CF-osx-10.9.5.drv
  /nix/store/p4k6hxkc0i52xq3d906hncw37zm37a1b-hook.drv
  /nix/store/nn3sizfpm3y3dlkb2rabb6k3gbc92drw-bsdmake-dev-tools-3.2.6.drv
  /nix/store/3vv2nbnq30rbv9vsl1h0kdkmdclxpcss-adv_cmds-osx-10.5.8.drv
  /nix/store/s2v50hragjc803axdb23m3h9bp3l1llx-hook.drv
  /nix/store/834wlmj214lqcw5dax5q2gfkd25c0a3p-libc++-3.7.1.drv
  /nix/store/2r31fil21wc92q0afmanzf7ygh5z0z3c-objc4-osx-10.9.5.drv
  /nix/store/qwf94wpglifwiwqlqh0ky6gvm8hpfvwb-help2man-1.47.4.drv
  /nix/store/6bqm6vd5abhl13bbf63nf3lc2fi4mqpb-libtool-2.4.6.drv
  /nix/store/92rm6ppizhnaanr66wafjp8smvcghd6g-cctools-port-877.5.drv
  /nix/store/rnhgg077j3n3vm7a67dy9sgy6f0n73dg-binutils-2.26.drv
  /nix/store/iml80d92q577rlbdkv491dp0iwssl3d4-cctools-binutils-darwin.drv
  /nix/store/wxcxkxx3wa629hwh3g3nk0na28lz82z1-bash-4.3-p42.drv
  /nix/store/vklj00xs1pgmakh5dygdq0y6pl71vdhi-clang-wrapper-9.9.9.drv
  /nix/store/nd21fvlvq9m8i2bhcirp855i5j8pgfa4-stdenv-darwin-boot-3.drv
  /nix/store/00yv4q3f69d5cq3pwaj34psh5wk3343m-zlib-1.2.8.drv
  /nix/store/1lyhjilwii1psr04w6pjzgph0spbid86-bzip2-1.0.6.0.1.drv
  /nix/store/2vzfa123layxgcslvn0an99iha11wcxx-ed-1.13.drv
  /nix/store/46r64f457dgrh0lh1ksx9kyk0driqwrd-ncurses-5.9.drv
  /nix/store/4mgk0hfhhh9yyqzbc6j6fczm7fdmbmhp-pcre-8.38.drv
  /nix/store/g8yfsc3yyxszwg8dhdqgkk1l17jwqr6q-hook.drv
  /nix/store/b1qrvmrfl0bj9bjn5r24fyjrjnlqgxpl-icu4c-56.1.drv
  /nix/store/b9kzhinxdg389n0d0ikvmw02v22ngfc9-CF-osx-10.9.5.drv
  /nix/store/fjww9kwkp0y6bfdk762ilkxq43drlhqa-xz-5.2.2.drv
  /nix/store/fyba1fl2pyixsgdv3qcfi8v771cam8zi-clang-wrapper-9.9.9.drv
  /nix/store/jbphlra83s84iys63vw5i8jx47ck2xnr-stdenv-darwin-boot-4.drv
  /nix/store/09jhypsy7cippnjcxifp7ayx51430w7h-perl-5.22.2.drv
  /nix/store/k0czm345fkhrgri0h53hdai9vs5iccgr-gnum4-1.4.17.drv
  /nix/store/2kfxmfkksgq5awdlwh217vbcy49lp5z5-bison-3.0.4.drv
  /nix/store/5finfknnbffv07l3djdx2cxf7v3fjyi9-binutils-2.26.drv
  /nix/store/7srj5g507cn46yyv5wq5j2mpdiir7s62-gnused-4.2.2.drv
  /nix/store/rk5jp0m9i71nps9y5ggyy82pw9qbvasb-libclosure-osx-10.10.5.drv
  /nix/store/gz91pzr7z65232bq1ib70xqw1qbzdm8f-hook.drv
  /nix/store/x04127qa5f9kna6zqgyrhj4krck2z0p4-icu4c-56.1.drv
  /nix/store/x63jpg0fgigb3n86rla39fwhicf0gjqn-libdispatch-osx-10.9.5.drv
  /nix/store/y69drwjv2vd86gyaf4w04n98gzrj0z6i-launchd-osx-10.9.5.drv
  /nix/store/81gbxpakpbqix9q8wsk3928z345qsxk4-CF-osx-10.9.5.drv
  /nix/store/ahqdjqrdsafqpn285wjs9vzvjpvvidxf-libiconv-osx-10.9.5.drv
  /nix/store/2kmgrlb30lk75w730czb0080bwils45h-objc4-osx-10.9.5.drv
  /nix/store/jqmfwx42s9ggswzss83xl2lgxafyw7w8-autoconf-2.69.drv
  /nix/store/aajr8mnzpzn92ggsgmarkq3pkdjzh6y4-automake-1.15.drv
  /nix/store/j2wxg4mxy15i9061kf74w27ryvgj2f5c-openssl-1.0.2h.drv
  /nix/store/356cziamjp610yc6b8hqisij5q7hv004-hook.drv
  /nix/store/z8lsi88nkb1y1zk3gpm505n1sv2x7lhf-gettext-0.19.7.drv
  /nix/store/c00cgml0w2kql3hq5rnjxz0dg5nm3ay0-perl-LocaleGettext-1.05.drv
  /nix/store/lwmcz69vrq8y8b7rh50hq528p5zjw8y9-help2man-1.47.4.drv
  /nix/store/p4fgp3savhz17f9bzqz8ah4876wf1wjx-libtool-2.4.6.drv
  /nix/store/al7jzdi9zy20iq963vafh7qgqh61x38l-cctools-port-877.5.drv
  /nix/store/jsbhsk2zhk8gwy4qg77vql1w8rc43ydk-gmp-6.1.0.drv
  /nix/store/bywznriqyr6ww33m1pq2bkqxfza4944n-coreutils-8.25.drv
  /nix/store/h8ya7bydjcdcq22h0jxm55j2rh5p8s1q-gnumake-4.1.drv
  /nix/store/a6lid7j9m6jaalpd1c5kis86gffya8s1-hook.drv
  /nix/store/kh71aqwj8p11wy3fllydpqqvvn3bzxn4-gnutar-1.29.drv
  /nix/store/mgkrh5q3y4bzhvg7xch6ssjflc46wxsc-diffutils-3.3.drv
  /nix/store/0cxs348y02zgazmmhx2479zq3d8vqxpf-hook.drv
  /nix/store/gqi3mjavvydxy97i8b1r9y73jbjbnkp5-libxml2-2.9.4.drv
  /nix/store/lv64g6wvw5q6fybl3ypc7a75dml9wc0s-libedit-20150325-3.1.drv
  /nix/store/pshsj2k4b9kn2a90q9nbiqxrlrmng3nz-libffi-3.2.1.drv
  /nix/store/z5mgk9n826819x994szn95mrkz0fa5wk-llvm-3.7.1.drv
  /nix/store/p695j1qgib2y50pikxg96lz6jify6n3y-clang-3.7.1.drv
  /nix/store/ps8mvx1jgqv88q7wyym683waphynbj4z-gnugrep-2.25.drv
  /nix/store/xbxzz2mwghsnpr8xiiw9ndw05gd54sf2-cctools-binutils-darwin.drv
  /nix/store/nak83rb1ky5gppl5bz3ss45brrg1zr59-clang-wrapper-3.7.1.drv
  /nix/store/sz01p9adrlckvkcwlzxis52aypjdczss-patch-2.7.5.drv
  /nix/store/2bbg46vr2qwb0dz1z03yzisb3j2628yw-libsigsegv-2.10.drv
  /nix/store/wk83947wavsf7igy93hsm9cp2jcd28pm-gawk-4.1.3.drv
  /nix/store/y81zgy0zagyqhxb1cmri44lrakzddk0m-findutils-4.6.0.drv
  /nix/store/yfcgzbh3cvk53481za29cj831vzijkrm-gzip-1.8.drv
  /nix/store/032ifhw22sjn5qpjnxalic6vapi8ynw5-stdenv-darwin.drv
building path(s) ‘/nix/store/4icsw4b9a7dncg5hysqxz00n9q00rlvv-stdenv-darwin-boot-2’
output ‘/nix/store/4icsw4b9a7dncg5hysqxz00n9q00rlvv-stdenv-darwin-boot-2’ is not allowed to refer to path ‘/nix/store/28127wbxdv63hh31r3z4a5jnhcr8ivh7-bootstrap-libcxx’
cannot build derivation ‘/nix/store/211cqsj45qhhhg3cnijdklxyy4qaz1b5-CF-osx-10.9.5.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/3vv2nbnq30rbv9vsl1h0kdkmdclxpcss-adv_cmds-osx-10.5.8.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/wxcxkxx3wa629hwh3g3nk0na28lz82z1-bash-4.3-p42.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/vklj00xs1pgmakh5dygdq0y6pl71vdhi-clang-wrapper-9.9.9.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/834wlmj214lqcw5dax5q2gfkd25c0a3p-libc++-3.7.1.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/16f6zmzvqlj3l4r8lc65lzl1pq0gmyzd-libc++abi-3.7.1.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/ahqdjqrdsafqpn285wjs9vzvjpvvidxf-libiconv-osx-10.9.5.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/032ifhw22sjn5qpjnxalic6vapi8ynw5-stdenv-darwin.drv’: 1 dependencies couldn't be built
error: build of ‘/nix/store/032ifhw22sjn5qpjnxalic6vapi8ynw5-stdenv-darwin.drv’ failed

• System: OSX
• Nix version: 1.11.2
• Nixpkgs version: 16.09pre-git

---

\cc @zimbatm @LnL7 @copumpkin any ideas?

[copumpkin]

I'd bisect and try to understand what broke it

[gilligan]

@copumpkin alright. I'll try tomorrow

[gilligan]

Actually that was quite fast:

% git bisect bad                                                                                                                                                              1 ↵
8ec5adc953d6bbabbe62d7f7229f146202704e53 is the first bad commit
commit 8ec5adc953d6bbabbe62d7f7229f146202704e53
Author: Thomas Tuegel <ttuegel@gmail.com>
Date:   Tue May 31 06:26:22 2016 -0500

    cmake: use supplied RPATH instead of guessing

:040000 040000 36e81d49e69346b85d93fd6a52ef6360d867799a d1e951abc34cc48685109cd656d9d183909a6b1a M  pkgs

[copumpkin]

Cool, thanks! @ttuegel, have any ideas why that would've changed something? The issue appears to be that your rpath change caused one of our builds to hold onto more of the bootstrap tools than we wanted it to.

[gilligan]

@copumpkin @ttuegel stdenv not building is quite a blocker - any chance we could revert this for now until we have a real solution? I don't understand anything about the details but it appears to be "only" an optimization to avoid some guess work so losing that for a bit might not be that bad?

[ttuegel]

I'll go ahead and revert it, but I'll need a Darwin user to fix it for me (since I can't test).

Probably you just need to filter CMAKE_INSTALL_RPATH. I realize this is inconvenient, but by default CMake uses some strange heuristics to guess the RPATH and that is Not Good.

I'm also a little startled that this causes more paths to be retained. CMAKE_INSTALL_RPATH is built by looking at -L flags in NIX_CFLAGS, so every path it picks up should already be picked up by the compiler. I actually introduced this change so that fewer paths would be retained (necessary for Qt/KDE).

[ttuegel]

I'll go ahead and revert it, but I'll need a Darwin user to fix it for me (since I can't test).

Actually, nobody should try to resurrect that patch. It goes against the CMake philosophy. The CMake philosophy is probably wrong, but it's our job to conform.

[copumpkin]

@ttuegel that seems worth a discussion on the issue tracker perhaps? A lot of things that might make "philosophical" sense in a broader unix community probably make less sense in Nix, so I wouldn't take such stances too dogmatically.

[copumpkin]

I definitely know that in the past I've been annoyed at CMake for trying to be too clever, so patches to it that make it less clever are great in my book.

[ttuegel]

I definitely know that in the past I've been annoyed at CMake for trying to be too clever, so patches to it that make it less clever are great in my book.

The patch overrides any attempt by any CMake module to do anything whatsoever to the RPATH. I did this for Qt because Qt's CMake modules are fundamentally incompatible with multiple outputs because of things they do with RPATH. However, it also stops packages written by people who know what they're doing with CMake from doing smart things with RPATH. Like LLVM.

[copumpkin]

Perhaps I'll just go back to hoping CMake magically disappears 😄

[ttuegel]

At least CMake is documented. 😃 Like CMake, autoconf and friends will let developers do any crazy-silly thing they want and we as packagers are doomed to never find the right setting to override it. Although, I think nobody who uses autoconf even knows how it works, so most developers are unlikely to do crazy stuff.

I think in this case I can just make the change I need to cmakeFlags locally in Qt builds. I got ahead of myself thinking that most builds would benefit from this. (They probably would.) It did not occur to me that somebody might actually know what they're doing!