Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

glibcCross: use a libgcc built separately from gcc #247900

Merged
15 commits merged into from Aug 15, 2023
Merged

glibcCross: use a libgcc built separately from gcc #247900

15 commits merged into from Aug 15, 2023

Commits on Aug 14, 2023

  1. libgcc: (re)init at 12.3.0

    This commit restores the pkgs/development/libraries/gcc/libgcc
    package, which was deleted by commit 9818d12.
    
    We need to be able to build libgcc separately from gcc in order to
    avoid a circular dependency.  Nixpkgs is unusual -- unlike any other
    distribution, it cannot tolerate circular dependencies between
    dynamically linked libraries.  Because of this, upstream is
    extremely unsympathetic to the trouble that the glibc<->gcc circular
    dependency causes for us; if we don't solve it ourselves it will not
    be solved.
    Adam Joseph committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    f445f64 View commit details
    Browse the repository at this point in the history
  2. libgcc: take from gcc unless explicitly overridden

    Adam Joseph committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    fcaa5a7 View commit details
    Browse the repository at this point in the history
  3. libgcc: minor formatting adjustments

    Adam Joseph committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    72fa597 View commit details
    Browse the repository at this point in the history
  4. libgcc: use finalAttrs instead of rec

    Adam Joseph committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    2ecf2d9 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    b5893e7 View commit details
    Browse the repository at this point in the history
  6. libgcc: add glibc as a buildInput

    This is necessary in order to prevent gcc from switching on
    `inhibit_libc`, which cripples the `libgcc_s.so` unwinder.
    Adam Joseph committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    383d62d View commit details
    Browse the repository at this point in the history
  7. gcc: factor out forceLibgccToBuildCrtStuff

    Adam Joseph committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    92186a4 View commit details
    Browse the repository at this point in the history
  8. libgcc: use forceLibgccToBuildCrtStuff

    This duplicates (by reference) the two-line adjustment to libgcc's
    Makefile needed in order to get crtstuff to build without a full
    build of gcc.
    Adam Joseph committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    da371c7 View commit details
    Browse the repository at this point in the history
  9. libgcc: configureFlags: minimize

    A lot of these flags were unnecessary.
    Adam Joseph committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    fa0ebe8 View commit details
    Browse the repository at this point in the history
  10. libgcc: gccConfigureFlags: minimize, fix

    This commit minimizes libgcc's gccConfigureFlags, and -- importantly
    -- includes the three flags needed in order to prevent
    `inhibit_libc` from becoming active.
    Adam Joseph committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    8c37dae View commit details
    Browse the repository at this point in the history
  11. libgcc: let-rename stdenvNoLibs to stdenv

    Adam Joseph committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    17ce868 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    c0e4121 View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    18c52d0 View commit details
    Browse the repository at this point in the history
  14. glibcCross: use a libgcc built separately from gcc

     ### Summary
    
    This PR completely and finally solves the gcc<->glibc circular
    `buildInputs` problem, for cross compilation.  The same technique
    can be applied to native builds in the future.
    
    Closes #213453
    
     ### Motivation
    
    Prior to this PR, we had the following circular `buildInputs` problem:
    
    1. gcc has glibc in its `buildInputs`
    
       - a compiled copy of glibc must be present before building gcc;
         if it isn't, gcc cripples itself (`inhibit_libc`) and refuses
         to build libgcc_s.so
    
    2. glibc has libgcc_s.so in its `buildInputs`
    
       - glibc `dlopen()`s libgcc_s.so in order to implement POSIX
         thread cancellation.  For security reasons `glibc` requires
         that the path to `libgcc_s.so` is [hardwired] into `glibc` at
         compile time, so it's technically not a true dynamic link -- it
         just pretends to be one.
    
    3. libgcc_s.so is built in the same derivation as gcc
    
       - libgcc_s.so is built as part of the gcc build process
    
    We must cut one of these three links in the loop.
    
     ### Previous Attempts
    
    Previously #238154 had
    attempted to cut link (1) by building `gcc` without `glibc`, and
    using the `libgcc_s` which emerges from that build.  Unfortunately
    this just doesn't work.  GCC's configure script extracts quite a lot
    of information from the glibc headers (which are a build artifact --
    you can't just copy them out of the source tarball) and various
    `./configure`-driven linking attempts.  If `glibc` isn't around at
    build time you wind up with a `libgcc_s.so` that is missing various
    unwinder features (see #213453
    for the most problematic one).
    
    Musl "cuts" link (2), or rather never creates it in the first place.
    ["Cancellation cleanup handling in musl has no relationship to C++
    exceptions and unwinding... glibc implements cancellation as an
    exception"](https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-cancellation).
    IMHO Musl made the smarter decision here.  It is incredibly rare to
    find a codebase that uses both POSIX thread cancellation *and* C++
    exceptions.  I have never seen a codebase that uses both *and*
    expects them to be aware of each other, and I would be astonished if
    one existed.  Glibc paid an immense cost in complexity for something
    nobody has ever used.
    
     ### Changes Made
    
    This PR cuts link (3): instead of building libgcc_s.so as part of
    gcc, we build it separately from gcc.  Now there is a strict acyclic
    graph of `buildInputs`:
    
    ```
     gccWithoutTargetLibc
     |
     +--->glibc-nolibgcc
     |    |
     |    v
     +--->libgcc
     |    |
     |    v
     +--->glibc
     |    |
     |    v
     +--->gcc
    ```
    
    In other words, there's a simple linear `buildInputs` chain
    `glibc-nolibgcc` `->` `libgcc` `->` `glibc` `->` `gcc` where all
    four packages are compiled by (and therefore have as a
    `(native)BuildInput`) `gccWithoutTargetLibc`.
    
    `gccWithoutTargetLibc` and `glibc-nolibgcc` are strictly
    bootstrapping artifacts; nothing else has them as a `buildInput` and
    they shouldn't appear in the closure of any final deployment
    packages.  `glibc-nolibgcc` lacks `libgcc_s.so`, so it will segfault
    if you try to use it with POSIX thread cancellation.  Fortunately
    all we need from it is (a) its headers (`lib.getDev`) and (b) to use
    it in the `./configure` script for `libgcc`.
    
    When translated over to the native bootstrap, `xgcc` takes the place
    of `gccWithoutTargetLibc`, and the "first `glibc`" (we build two of
    them) takes the place of `glibc-nolibgcc`.  At that point our native
    and cross bootstrap have the same overall architecture, and it
    becomes possible to merge them (at last!)
    
    [213453]: #213453
    [238154]: #238154
    [hardwired]: https://github.com/NixOS/nixpkgs/blob/7553d0fe29801938bcb280bb324b579ef9016aea/pkgs/development/libraries/glibc/default.nix#L69-L88
    Adam Joseph committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    64046f0 View commit details
    Browse the repository at this point in the history
  15. tests.cross.sanity: enable mbuffer test

    This test passes now.  Also fixes a minor oversight in the bug --
    the test case needs to `touch $out` on success.
    Adam Joseph committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    1912681 View commit details
    Browse the repository at this point in the history