Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

gfortran binaries installed by meson have rpath problem #10711

Open
langevin-usgs opened this issue Aug 18, 2022 · 6 comments
Open

gfortran binaries installed by meson have rpath problem #10711

langevin-usgs opened this issue Aug 18, 2022 · 6 comments
Labels
language:fortran OS:macos Issues specific to Apple Operating Systems like MacOS and iOS

Comments

@langevin-usgs
Copy link

Describe the bug
gfortran seems to have changed the way the way RPATH is handled on Mac as described here. When we use the latest versions of gfortran installed with Homebrew and Macports, the binaries "installed" by Meson fail to run and return

dyld[64021]: Library not loaded: '@rpath/libgfortran.5.dylib'
  Referenced from: '/Users/langevin/langevin/dev/modflow6-fork.git/bin/mf6'
  Reason: tried: '/usr/local/lib/libgfortran.5.dylib' (no such file), '/usr/lib/libgfortran.5.dylib' (no such file)

The original binaries created by meson prior to being "installed" work just fine, and so it appears there is an RPATH issue that arises during the meson "install" process.

To Reproduce
This occurs when building MODFLOW here. We haven't seen the issue yet on Github actions, presumably because we are using an older version of gfortran. It appears that the RPATH change is being back ported to earlier version of gfortran; we've seen issues with both gfortran 11.3 and 12.1.

system parameters

  • This problem only shows up on Mac
  • Problem occurs on macOS Monterey. Not sure about other MacOS versions
  • Using python 3.9.7
  • meson --version is 0.63.1
  • ninja --version is 1.10.2.git.kitware.jobserver-1

Other MODFLOW team members affected include @Hofer-Julian and @w-bonelli
@dcbaker dcbaker added OS:macos Issues specific to Apple Operating Systems like MacOS and iOS language:fortran labels Aug 19, 2022
@Hofer-Julian
Copy link
Contributor

@rgommers did people experience similar problems with scipy?

@rgommers
Copy link
Contributor

rgommers commented Sep 4, 2022

Not that I know of (yet). But both in CI and in most people's dev setups, the gfortran from conda-forge is used (now at version 11.0.1).

@Hofer-Julian
Copy link
Contributor

the gfortran from conda-forge is used.

That might be the reason why you don't hit that.
We were able to reproduce it with homebrew as well as MacPorts.

@rgommers
Copy link
Contributor

One related issue I just had is that meson-python currently adds a spurious runpath, which caused this issue: scipy/scipy#17070. Once that's gone, things will work again when mixing wheels and conda packages. In the meantime, using meson install is still fine.

For the purposes of this issue, it would be useful if you could show the rpath/runpath entries both in the build and install directories, to make visible what is changing between them.

@langevin-usgs
Copy link
Author

This is what I'm seeing on my Mac with "otool -l ./mf6" after a meson install:

% otool -l ./mf6
./mf6:
Load command 0
      cmd LC_SEGMENT_64
  cmdsize 72
  segname __PAGEZERO
   vmaddr 0x0000000000000000
   vmsize 0x0000000100000000
  fileoff 0
 filesize 0
  maxprot 0x00000000
 initprot 0x00000000
   nsects 0
    flags 0x0
Load command 1
      cmd LC_SEGMENT_64
  cmdsize 712
  segname __TEXT
   vmaddr 0x0000000100000000
   vmsize 0x0000000000330000
  fileoff 0
 filesize 3342336
  maxprot 0x00000005
 initprot 0x00000005
   nsects 8
    flags 0x0
Section
  sectname __text
   segname __TEXT
      addr 0x0000000100003000
      size 0x00000000002069f8
    offset 12288
     align 2^4 (16)
    reloff 0
    nreloc 0
     flags 0x80000400
 reserved1 0
 reserved2 0
Section
  sectname __text_startup
   segname __TEXT
      addr 0x0000000100209a00
      size 0x0000000000000030
    offset 2136576
     align 2^4 (16)
    reloff 0
    nreloc 0
     flags 0x80000400
 reserved1 0
 reserved2 0
Section
  sectname __text_cold
   segname __TEXT
      addr 0x0000000100209a30
      size 0x0000000000000009
    offset 2136624
     align 2^0 (1)
    reloff 0
    nreloc 0
     flags 0x80000400
 reserved1 0
 reserved2 0
Section
  sectname __stubs
   segname __TEXT
      addr 0x0000000100209a3a
      size 0x000000000000019e
    offset 2136634
     align 2^1 (2)
    reloff 0
    nreloc 0
     flags 0x80000408
 reserved1 0 (index into indirect symbol table)
 reserved2 6 (size of stubs)
Section
  sectname __stub_helper
   segname __TEXT
      addr 0x0000000100209bd8
      size 0x00000000000002c2
    offset 2137048
     align 2^2 (4)
    reloff 0
    nreloc 0
     flags 0x80000400
 reserved1 0
 reserved2 0
Section
  sectname __const
   segname __TEXT
      addr 0x0000000100209ea0
      size 0x00000000000befe0
    offset 2137760
     align 2^5 (32)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Section
  sectname __cstring
   segname __TEXT
      addr 0x00000001002c8e80
      size 0x00000000000118d7
    offset 2920064
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000002
 reserved1 0
 reserved2 0
Section
  sectname __eh_frame
   segname __TEXT
      addr 0x00000001002da758
      size 0x00000000000558a8
    offset 2991960
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Load command 2
      cmd LC_SEGMENT_64
  cmdsize 232
  segname __DATA_CONST
   vmaddr 0x0000000100330000
   vmsize 0x000000000026c000
  fileoff 3342336
 filesize 2539520
  maxprot 0x00000003
 initprot 0x00000003
   nsects 2
    flags 0x10
Section
  sectname __got
   segname __DATA_CONST
      addr 0x0000000100330000
      size 0x00000000000000f8
    offset 3342336
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000006
 reserved1 69 (index into indirect symbol table)
 reserved2 0
Section
  sectname __const
   segname __DATA_CONST
      addr 0x0000000100330100
      size 0x0000000000268248
    offset 3342592
     align 2^5 (32)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Load command 3
      cmd LC_SEGMENT_64
  cmdsize 392
  segname __DATA
   vmaddr 0x000000010059c000
   vmsize 0x0000000000040000
  fileoff 5881856
 filesize 49152
  maxprot 0x00000003
 initprot 0x00000003
   nsects 4
    flags 0x0
Section
  sectname __la_symbol_ptr
   segname __DATA
      addr 0x000000010059c000
      size 0x0000000000000228
    offset 5881856
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000007
 reserved1 100 (index into indirect symbol table)
 reserved2 0
Section
  sectname __data
   segname __DATA
      addr 0x000000010059c240
      size 0x000000000000b028
    offset 5882432
     align 2^5 (32)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Section
  sectname __bss
   segname __DATA
      addr 0x00000001005a7280
      size 0x0000000000000844
    offset 0
     align 2^5 (32)
    reloff 0
    nreloc 0
     flags 0x00000001
 reserved1 0
 reserved2 0
Section
  sectname __common
   segname __DATA
      addr 0x00000001005a7ae0
      size 0x0000000000034484
    offset 0
     align 2^5 (32)
    reloff 0
    nreloc 0
     flags 0x00000001
 reserved1 0
 reserved2 0
Load command 4
      cmd LC_SEGMENT_64
  cmdsize 72
  segname __LINKEDIT
   vmaddr 0x00000001005dc000
   vmsize 0x0000000000094000
  fileoff 5931008
 filesize 594256
  maxprot 0x00000001
 initprot 0x00000001
   nsects 0
    flags 0x0
Load command 5
            cmd LC_DYLD_INFO_ONLY
        cmdsize 48
     rebase_off 5931008
    rebase_size 4264
       bind_off 5935272
      bind_size 24
  weak_bind_off 0
 weak_bind_size 0
  lazy_bind_off 5935296
 lazy_bind_size 1824
     export_off 5937120
    export_size 71800
Load command 6
     cmd LC_SYMTAB
 cmdsize 24
  symoff 6013400
   nsyms 15141
  stroff 6256336
 strsize 268928
Load command 7
            cmd LC_DYSYMTAB
        cmdsize 80
      ilocalsym 0
      nlocalsym 12288
     iextdefsym 12288
     nextdefsym 2783
      iundefsym 15071
      nundefsym 70
         tocoff 0
           ntoc 0
      modtaboff 0
        nmodtab 0
   extrefsymoff 0
    nextrefsyms 0
 indirectsymoff 6255656
  nindirectsyms 169
      extreloff 0
        nextrel 0
      locreloff 0
        nlocrel 0
Load command 8
          cmd LC_LOAD_DYLINKER
      cmdsize 32
         name /usr/lib/dyld (offset 12)
Load command 9
     cmd LC_UUID
 cmdsize 24
    uuid F0DEADF2-4620-3C95-83D6-D9EDEBB82CC7
Load command 10
      cmd LC_BUILD_VERSION
  cmdsize 32
 platform 1
    minos 12.0
      sdk 12.0
   ntools 1
     tool 3
  version 819.6
Load command 11
      cmd LC_SOURCE_VERSION
  cmdsize 16
  version 0.0
Load command 12
       cmd LC_MAIN
   cmdsize 24
  entryoff 2136576
 stacksize 0
Load command 13
          cmd LC_LOAD_DYLIB
      cmdsize 56
         name @rpath/libgfortran.5.dylib (offset 24)
   time stamp 2 Wed Dec 31 18:00:02 1969
      current version 6.0.0
compatibility version 6.0.0
Load command 14
          cmd LC_LOAD_DYLIB
      cmdsize 56
         name @rpath/libgcc_s.1.1.dylib (offset 24)
   time stamp 2 Wed Dec 31 18:00:02 1969
      current version 1.1.0
compatibility version 1.0.0
Load command 15
          cmd LC_LOAD_DYLIB
      cmdsize 56
         name /usr/lib/libSystem.B.dylib (offset 24)
   time stamp 2 Wed Dec 31 18:00:02 1969
      current version 1311.100.3
compatibility version 1.0.0
Load command 16
      cmd LC_FUNCTION_STARTS
  cmdsize 16
  dataoff 6008920
 datasize 4480
Load command 17
      cmd LC_DATA_IN_CODE
  cmdsize 16
  dataoff 6013400
 datasize 0

This is what is shown with "otool -l ./mf6" when I run it in the builddir

% otool -l ./mf6
./mf6:
Load command 0
      cmd LC_SEGMENT_64
  cmdsize 72
  segname __PAGEZERO
   vmaddr 0x0000000000000000
   vmsize 0x0000000100000000
  fileoff 0
 filesize 0
  maxprot 0x00000000
 initprot 0x00000000
   nsects 0
    flags 0x0
Load command 1
      cmd LC_SEGMENT_64
  cmdsize 712
  segname __TEXT
   vmaddr 0x0000000100000000
   vmsize 0x0000000000330000
  fileoff 0
 filesize 3342336
  maxprot 0x00000005
 initprot 0x00000005
   nsects 8
    flags 0x0
Section
  sectname __text
   segname __TEXT
      addr 0x0000000100003000
      size 0x00000000002069f8
    offset 12288
     align 2^4 (16)
    reloff 0
    nreloc 0
     flags 0x80000400
 reserved1 0
 reserved2 0
Section
  sectname __text_startup
   segname __TEXT
      addr 0x0000000100209a00
      size 0x0000000000000030
    offset 2136576
     align 2^4 (16)
    reloff 0
    nreloc 0
     flags 0x80000400
 reserved1 0
 reserved2 0
Section
  sectname __text_cold
   segname __TEXT
      addr 0x0000000100209a30
      size 0x0000000000000009
    offset 2136624
     align 2^0 (1)
    reloff 0
    nreloc 0
     flags 0x80000400
 reserved1 0
 reserved2 0
Section
  sectname __stubs
   segname __TEXT
      addr 0x0000000100209a3a
      size 0x000000000000019e
    offset 2136634
     align 2^1 (2)
    reloff 0
    nreloc 0
     flags 0x80000408
 reserved1 0 (index into indirect symbol table)
 reserved2 6 (size of stubs)
Section
  sectname __stub_helper
   segname __TEXT
      addr 0x0000000100209bd8
      size 0x00000000000002c2
    offset 2137048
     align 2^2 (4)
    reloff 0
    nreloc 0
     flags 0x80000400
 reserved1 0
 reserved2 0
Section
  sectname __const
   segname __TEXT
      addr 0x0000000100209ea0
      size 0x00000000000befe0
    offset 2137760
     align 2^5 (32)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Section
  sectname __cstring
   segname __TEXT
      addr 0x00000001002c8e80
      size 0x00000000000118d7
    offset 2920064
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000002
 reserved1 0
 reserved2 0
Section
  sectname __eh_frame
   segname __TEXT
      addr 0x00000001002da758
      size 0x00000000000558a8
    offset 2991960
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Load command 2
      cmd LC_SEGMENT_64
  cmdsize 232
  segname __DATA_CONST
   vmaddr 0x0000000100330000
   vmsize 0x000000000026c000
  fileoff 3342336
 filesize 2539520
  maxprot 0x00000003
 initprot 0x00000003
   nsects 2
    flags 0x10
Section
  sectname __got
   segname __DATA_CONST
      addr 0x0000000100330000
      size 0x00000000000000f8
    offset 3342336
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000006
 reserved1 69 (index into indirect symbol table)
 reserved2 0
Section
  sectname __const
   segname __DATA_CONST
      addr 0x0000000100330100
      size 0x0000000000268248
    offset 3342592
     align 2^5 (32)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Load command 3
      cmd LC_SEGMENT_64
  cmdsize 392
  segname __DATA
   vmaddr 0x000000010059c000
   vmsize 0x0000000000040000
  fileoff 5881856
 filesize 49152
  maxprot 0x00000003
 initprot 0x00000003
   nsects 4
    flags 0x0
Section
  sectname __la_symbol_ptr
   segname __DATA
      addr 0x000000010059c000
      size 0x0000000000000228
    offset 5881856
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000007
 reserved1 100 (index into indirect symbol table)
 reserved2 0
Section
  sectname __data
   segname __DATA
      addr 0x000000010059c240
      size 0x000000000000b028
    offset 5882432
     align 2^5 (32)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Section
  sectname __bss
   segname __DATA
      addr 0x00000001005a7280
      size 0x0000000000000844
    offset 0
     align 2^5 (32)
    reloff 0
    nreloc 0
     flags 0x00000001
 reserved1 0
 reserved2 0
Section
  sectname __common
   segname __DATA
      addr 0x00000001005a7ae0
      size 0x0000000000034484
    offset 0
     align 2^5 (32)
    reloff 0
    nreloc 0
     flags 0x00000001
 reserved1 0
 reserved2 0
Load command 4
      cmd LC_SEGMENT_64
  cmdsize 72
  segname __LINKEDIT
   vmaddr 0x00000001005dc000
   vmsize 0x0000000000094000
  fileoff 5931008
 filesize 594256
  maxprot 0x00000001
 initprot 0x00000001
   nsects 0
    flags 0x0
Load command 5
            cmd LC_DYLD_INFO_ONLY
        cmdsize 48
     rebase_off 5931008
    rebase_size 4264
       bind_off 5935272
      bind_size 24
  weak_bind_off 0
 weak_bind_size 0
  lazy_bind_off 5935296
 lazy_bind_size 1824
     export_off 5937120
    export_size 71800
Load command 6
     cmd LC_SYMTAB
 cmdsize 24
  symoff 6013400
   nsyms 15141
  stroff 6256336
 strsize 268928
Load command 7
            cmd LC_DYSYMTAB
        cmdsize 80
      ilocalsym 0
      nlocalsym 12288
     iextdefsym 12288
     nextdefsym 2783
      iundefsym 15071
      nundefsym 70
         tocoff 0
           ntoc 0
      modtaboff 0
        nmodtab 0
   extrefsymoff 0
    nextrefsyms 0
 indirectsymoff 6255656
  nindirectsyms 169
      extreloff 0
        nextrel 0
      locreloff 0
        nlocrel 0
Load command 8
          cmd LC_LOAD_DYLINKER
      cmdsize 32
         name /usr/lib/dyld (offset 12)
Load command 9
     cmd LC_UUID
 cmdsize 24
    uuid F0DEADF2-4620-3C95-83D6-D9EDEBB82CC7
Load command 10
      cmd LC_BUILD_VERSION
  cmdsize 32
 platform 1
    minos 12.0
      sdk 12.0
   ntools 1
     tool 3
  version 819.6
Load command 11
      cmd LC_SOURCE_VERSION
  cmdsize 16
  version 0.0
Load command 12
       cmd LC_MAIN
   cmdsize 24
  entryoff 2136576
 stacksize 0
Load command 13
          cmd LC_LOAD_DYLIB
      cmdsize 56
         name @rpath/libgfortran.5.dylib (offset 24)
   time stamp 2 Wed Dec 31 18:00:02 1969
      current version 6.0.0
compatibility version 6.0.0
Load command 14
          cmd LC_LOAD_DYLIB
      cmdsize 56
         name @rpath/libgcc_s.1.1.dylib (offset 24)
   time stamp 2 Wed Dec 31 18:00:02 1969
      current version 1.1.0
compatibility version 1.0.0
Load command 15
          cmd LC_LOAD_DYLIB
      cmdsize 56
         name /usr/lib/libSystem.B.dylib (offset 24)
   time stamp 2 Wed Dec 31 18:00:02 1969
      current version 1311.100.3
compatibility version 1.0.0
Load command 16
          cmd LC_RPATH
      cmdsize 40
         path /opt/local/lib/libgcc (offset 12)
Load command 17
          cmd LC_RPATH
      cmdsize 32
         path @loader_path (offset 12)
Load command 18
          cmd LC_RPATH
      cmdsize 72
         path /opt/local/lib/gcc12/gcc/x86_64-apple-darwin21/12.2.0 (offset 12)
Load command 19
          cmd LC_RPATH
      cmdsize 40
         path /opt/local/lib/gcc12 (offset 12)
Load command 20
      cmd LC_FUNCTION_STARTS
  cmdsize 16
  dataoff 6008920
 datasize 4480
Load command 21
      cmd LC_DATA_IN_CODE
  cmdsize 16
  dataoff 6013400
 datasize 0

Compiler is:

% gfortran --version
GNU Fortran (MacPorts gcc12 12.2.0_0+stdlib_flag) 12.2.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

The meson installed version of the binary gives:

% ./mf6
dyld[52446]: Library not loaded: '@rpath/libgfortran.5.dylib'
  Referenced from: '/Users/langevin/langevin/dev/modflow6-fork.git/bin/mf6'
  Reason: tried: '/usr/local/lib/libgfortran.5.dylib' (no such file), '/usr/lib/libgfortran.5.dylib' (no such file)
zsh: abort      ./mf6

The binary created in the build directory works just fine, so it is in the "install" that is stripping off some load commands. My apologies if I'm doing something wrong here, which could very well be the case. I'm running on macOS Monterey version 12.6.

@rgommers rgommers mentioned this issue Oct 5, 2022
Merged
10 tasks
@rgommers
Copy link
Contributor

rgommers commented Oct 6, 2022

I've tried this with apexpy and Homebrew's gcc/gfortran 12.2.0 - works fine there. While devs report that the MacPorts gcc 12.2.0 is broken.

I also tried building SciPy with Homebrew gcc 12.2, but that doesn't work - for some reason the Homebrew compilers are unable to find the C++ standard library. Building with conda-forge Clang + Homebrew gfortran works fine.

This is what I'm seeing on my Mac with "otool -l ./mf6" after a meson install:
...
@rpath/libgfortran.5.dylib

That looks wrong indeed. I cannot reproduce that with Meson 0.63.3 - I'm getting an absolute path:

otool -l build-install/lib/python3.9/site-packages/apexpy/fortranapex.cpython-39-darwin.so | rg libgfortran
         name /opt/homebrew/opt/gcc/lib/gcc/current/libgfortran.5.dylib (offset 24)

@jfowkes jfowkes mentioned this issue Dec 21, 2022
Closed
@jfowkes jfowkes mentioned this issue Jan 3, 2023
Closed
10 tasks
@Akemi Akemi mentioned this issue Mar 4, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
language:fortran OS:macos Issues specific to Apple Operating Systems like MacOS and iOS
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@rgommers @dcbaker @langevin-usgs @Hofer-Julian