Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[native-library] If a dllimport is specified with an absolute path, look for it first #85255

Merged
merged 1 commit into from
Apr 25, 2023

Conversation

lambdageek
Copy link
Member

The unmanaged native library probing documentation says to try absolute paths without variations

https://learn.microsoft.com/en-us/dotnet/core/dependency-loading/default-probing#unmanaged-native-library-probing

@lambdageek
Copy link
Member Author

/cc @rolfbjarne @elinor-fung

@ghost
Copy link

ghost commented Apr 24, 2023

Tagging subscribers to this area:
See info in area-owners.md if you want to be subscribed.

Issue Details

The unmanaged native library probing documentation says to try absolute paths without variations

https://learn.microsoft.com/en-us/dotnet/core/dependency-loading/default-probing#unmanaged-native-library-probing

Author: lambdageek
Assignees: lambdageek
Labels:

area-AssemblyLoader-mono

Milestone: -

src/mono/mono/metadata/native-library.c Outdated Show resolved Hide resolved
@lambdageek lambdageek force-pushed the fix-native-loader-abspath branch from e9f5505 to 130794d Compare April 24, 2023 15:38
@lambdageek lambdageek marked this pull request as ready for review April 24, 2023 15:56
@lambdageek
Copy link
Member Author

/azp run runtime-extra-platforms

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@lambdageek
Copy link
Member Author

/backport to release/7.0-staging

@github-actions
Copy link
Contributor

Started backporting to release/7.0-staging: https://github.com/dotnet/runtime/actions/runs/4789974115

@lambdageek
Copy link
Member Author

Verified that without this, doing

        [DllImport ("/System/Library/Frameworks/SceneKit.framework/SceneKit")]
        private extern static void Something();

will probe like this:

dyld[93207]: find path "/Users/alklig/Projects/catahi/bin/Debug/net8.0-maccatalyst/maccatalyst-arm64/catahi.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit"
dyld[93207]:   possible path(Catalyst prefix on disk): "/System/iOSSupport/Users/alklig/Projects/catahi/bin/Debug/net8.0-maccatalyst/maccatalyst-arm64/catahi.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit"
dyld[93207]:   possible path(cryptex Catalyst prefix): "/System/Volumes/Preboot/Cryptexes/OS/System/iOSSupport/Users/alklig/Projects/catahi/bin/Debug/net8.0-maccatalyst/maccatalyst-arm64/catahi.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit"
dyld[93207]:   possible path(Catalyst prefix): "/System/iOSSupport/Users/alklig/Projects/catahi/bin/Debug/net8.0-maccatalyst/maccatalyst-arm64/catahi.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit"
dyld[93207]:   possible path(original path on disk): "/Users/alklig/Projects/catahi/bin/Debug/net8.0-maccatalyst/maccatalyst-arm64/catahi.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit"
dyld[93207]:   possible path(cryptex prefix): "/System/Volumes/Preboot/Cryptexes/OS/Users/alklig/Projects/catahi/bin/Debug/net8.0-maccatalyst/maccatalyst-arm64/catahi.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit"
dyld[93207]:   possible path(original path): "/Users/alklig/Projects/catahi/bin/Debug/net8.0-maccatalyst/maccatalyst-arm64/catahi.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit"
dyld[93207]:   possible path(default fallback): "/System/Library/Frameworks/SceneKit.framework/SceneKit"
dyld[93207]:   found: dylib-from-cache: (0x028A) "/System/Library/Frameworks/SceneKit.framework/SceneKit"

with this PR it probes like this:


dyld[82894]: find path "/System/Library/Frameworks/SceneKit.framework/SceneKit"
dyld[82894]:   switch to canonical cache path: /System/Library/Frameworks/SceneKit.framework/Versions/A/SceneKit
dyld[82894]:   possible path(Catalyst prefix on disk): "/System/iOSSupport/System/Library/Frameworks/SceneKit.framework/Versions/A/SceneKit"
dyld[82894]:   found: already-loaded-by-path: "/System/iOSSupport/System/Library/Frameworks/SceneKit.framework/Versions/A/SceneKit"

lambdageek added a commit to lambdageek/runtime that referenced this pull request Apr 24, 2023
…ook for it first

The unmanaged native library probing documentation says to try
absolute paths without variations

https://learn.microsoft.com/en-us/dotnet/core/dependency-loading/default-probing#unmanaged-native-library-probing

Manual backport of  dotnet#85255 to net6
@lambdageek
Copy link
Member Author

net6 backport is #85271

@lambdageek lambdageek merged commit 41f6e79 into dotnet:main Apr 25, 2023
lambdageek added a commit that referenced this pull request Apr 26, 2023
…ook for it first (#85271)

The unmanaged native library probing documentation says to try
absolute paths without variations

https://learn.microsoft.com/en-us/dotnet/core/dependency-loading/default-probing#unmanaged-native-library-probing

Manual backport of  #85255 to net6
rolfbjarne added a commit to rolfbjarne/xamarin-macios that referenced this pull request Apr 27, 2023
1. Mono changed dyld lookup to start looking in directories in
   NATIVE_DLL_SEARCH_DIRECTORIES before the actual given path, even when the
   given path is absolute [1].
2. This turned out to break Mac Catalyst, because when a DllImport says a
   P/Invoke is in "/System/Library/Frameworks/SceneKit.framework/SceneKit",
   Mono would try loading by prefixing the directories in
   NATIVE_DLL_SEARCH_DIRECTORIES. We add the Contents/MonoBundle directory to
   NATIVE_DLL_SEARCH_DIRECTORIES, so Mono would try to load
   "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
   and things would go wrong.
3. We found a workaround: add "/" to NATIVE_DLL_SEARCH_DIRECTORIES. This works
   on Ventura, but apparently not on older macOS version, because the actual
   path we pass to dlopen ends up being "///System/Library/Frameworks/SceneKit.framework/SceneKit"
   (note the three initial slashes instead of a single slash).
4. Add a second workaround, where we add a dll import resolver to load exactly
   the path we want to load.

[1]: dotnet/runtime@5a1baeb
[2]: dotnet/runtime#85255

Technical sidenote:

Why trying to load "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit"
turned out so bad on Mac Catalyst is not obvious. What happens is this:

* The app calls 'dlopen ("/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit")'
* dlopen checks if this is a Mac Catalyst override of a macOS system
  framework, by prefixing "/System/iOSSupport" and trying to load that. So
  dlopen would try to load "/System/iOSSupport/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
  which would obviously fail.
* Then dlopen would try a few more fallbacks, eventually trying
  "/System/Library/Frameworks/SceneKit.framework/SceneKit", and successfully
  loading that library.
* Unfortunately "/System/Library/Frameworks/SceneKit.framework/SceneKit" is
  the wrong library to load for Mac Catalyst ("/System/iOSSupport/System/Library/Frameworks/SceneKit.framework/SceneKit"
  is the correct version). These two libraries are incompatible, and calling
  one when you mean to call the other will do nasty things like corrupting the
  stack.
rolfbjarne added a commit to xamarin/xamarin-macios that referenced this pull request Apr 27, 2023
1. Mono changed dyld lookup to start looking in directories in
   NATIVE_DLL_SEARCH_DIRECTORIES before the actual given path, even when the
   given path is absolute [1].
2. This turned out to break Mac Catalyst, because when a DllImport says a
   P/Invoke is in "/System/Library/Frameworks/SceneKit.framework/SceneKit",
   Mono would try loading by prefixing the directories in
   NATIVE_DLL_SEARCH_DIRECTORIES. We add the Contents/MonoBundle directory to
   NATIVE_DLL_SEARCH_DIRECTORIES, so Mono would try to load
   "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
   and things would go wrong.
3. We found a workaround: add "/" to NATIVE_DLL_SEARCH_DIRECTORIES. This works
   on Ventura, but apparently not on older macOS version, because the actual
   path we pass to dlopen ends up being "///System/Library/Frameworks/SceneKit.framework/SceneKit"
   (note the three initial slashes instead of a single slash).
4. Add a second workaround, where we add a dll import resolver to load exactly
   the path we want to load.

[1]: dotnet/runtime@5a1baeb
[2]: dotnet/runtime#85255

Technical sidenote:

Why trying to load "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit"
turned out so bad on Mac Catalyst is not obvious. What happens is this:

* The app calls 'dlopen ("/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit")'
* dlopen checks if this is a Mac Catalyst override of a macOS system
  framework, by prefixing "/System/iOSSupport" and trying to load that. So
  dlopen would try to load "/System/iOSSupport/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
  which would obviously fail.
* Then dlopen would try a few more fallbacks, eventually trying
  "/System/Library/Frameworks/SceneKit.framework/SceneKit", and successfully
  loading that library.
* Unfortunately "/System/Library/Frameworks/SceneKit.framework/SceneKit" is
  the wrong library to load for Mac Catalyst ("/System/iOSSupport/System/Library/Frameworks/SceneKit.framework/SceneKit"
  is the correct version). These two libraries are incompatible, and calling
  one when you mean to call the other will do nasty things like corrupting the
  stack.
rolfbjarne added a commit to rolfbjarne/xamarin-macios that referenced this pull request Apr 27, 2023
…ccore@2668.

1. Mono changed dyld lookup to start looking in directories in
   NATIVE_DLL_SEARCH_DIRECTORIES before the actual given path, even when the
   given path is absolute [1].
2. This turned out to break Mac Catalyst, because when a DllImport says a
   P/Invoke is in "/System/Library/Frameworks/SceneKit.framework/SceneKit",
   Mono would try loading by prefixing the directories in
   NATIVE_DLL_SEARCH_DIRECTORIES. We add the Contents/MonoBundle directory to
   NATIVE_DLL_SEARCH_DIRECTORIES, so Mono would try to load
   "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
   and things would go wrong.
3. We found a workaround: add "/" to NATIVE_DLL_SEARCH_DIRECTORIES. This works
   on Ventura, but apparently not on older macOS version, because the actual
   path we pass to dlopen ends up being "///System/Library/Frameworks/SceneKit.framework/SceneKit"
   (note the three initial slashes instead of a single slash).
4. Add a second workaround, where we add a dll import resolver to load exactly
   the path we want to load.

[1]: dotnet/runtime@5a1baeb
[2]: dotnet/runtime#85255

Technical sidenote:

Why trying to load "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit"
turned out so bad on Mac Catalyst is not obvious. What happens is this:

* The app calls 'dlopen ("/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit")'
* dlopen checks if this is a Mac Catalyst override of a macOS system
  framework, by prefixing "/System/iOSSupport" and trying to load that. So
  dlopen would try to load "/System/iOSSupport/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
  which would obviously fail.
* Then dlopen would try a few more fallbacks, eventually trying
  "/System/Library/Frameworks/SceneKit.framework/SceneKit", and successfully
  loading that library.
* Unfortunately "/System/Library/Frameworks/SceneKit.framework/SceneKit" is
  the wrong library to load for Mac Catalyst ("/System/iOSSupport/System/Library/Frameworks/SceneKit.framework/SceneKit"
  is the correct version). These two libraries are incompatible, and calling
  one when you mean to call the other will do nasty things like corrupting the
  stack.

Backport of xamarin#18159.
vs-mobiletools-engineering-service2 pushed a commit to vs-mobiletools-engineering-service2/xamarin-macios that referenced this pull request Apr 27, 2023
1. Mono changed dyld lookup to start looking in directories in
   NATIVE_DLL_SEARCH_DIRECTORIES before the actual given path, even when the
   given path is absolute [1].
2. This turned out to break Mac Catalyst, because when a DllImport says a
   P/Invoke is in "/System/Library/Frameworks/SceneKit.framework/SceneKit",
   Mono would try loading by prefixing the directories in
   NATIVE_DLL_SEARCH_DIRECTORIES. We add the Contents/MonoBundle directory to
   NATIVE_DLL_SEARCH_DIRECTORIES, so Mono would try to load
   "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
   and things would go wrong.
3. We found a workaround: add "/" to NATIVE_DLL_SEARCH_DIRECTORIES. This works
   on Ventura, but apparently not on older macOS version, because the actual
   path we pass to dlopen ends up being "///System/Library/Frameworks/SceneKit.framework/SceneKit"
   (note the three initial slashes instead of a single slash).
4. Add a second workaround, where we add a dll import resolver to load exactly
   the path we want to load.

[1]: dotnet/runtime@5a1baeb
[2]: dotnet/runtime#85255

Technical sidenote:

Why trying to load "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit"
turned out so bad on Mac Catalyst is not obvious. What happens is this:

* The app calls 'dlopen ("/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit")'
* dlopen checks if this is a Mac Catalyst override of a macOS system
  framework, by prefixing "/System/iOSSupport" and trying to load that. So
  dlopen would try to load "/System/iOSSupport/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
  which would obviously fail.
* Then dlopen would try a few more fallbacks, eventually trying
  "/System/Library/Frameworks/SceneKit.framework/SceneKit", and successfully
  loading that library.
* Unfortunately "/System/Library/Frameworks/SceneKit.framework/SceneKit" is
  the wrong library to load for Mac Catalyst ("/System/iOSSupport/System/Library/Frameworks/SceneKit.framework/SceneKit"
  is the correct version). These two libraries are incompatible, and calling
  one when you mean to call the other will do nasty things like corrupting the
  stack.
vs-mobiletools-engineering-service2 pushed a commit to vs-mobiletools-engineering-service2/xamarin-macios that referenced this pull request Apr 27, 2023
1. Mono changed dyld lookup to start looking in directories in
   NATIVE_DLL_SEARCH_DIRECTORIES before the actual given path, even when the
   given path is absolute [1].
2. This turned out to break Mac Catalyst, because when a DllImport says a
   P/Invoke is in "/System/Library/Frameworks/SceneKit.framework/SceneKit",
   Mono would try loading by prefixing the directories in
   NATIVE_DLL_SEARCH_DIRECTORIES. We add the Contents/MonoBundle directory to
   NATIVE_DLL_SEARCH_DIRECTORIES, so Mono would try to load
   "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
   and things would go wrong.
3. We found a workaround: add "/" to NATIVE_DLL_SEARCH_DIRECTORIES. This works
   on Ventura, but apparently not on older macOS version, because the actual
   path we pass to dlopen ends up being "///System/Library/Frameworks/SceneKit.framework/SceneKit"
   (note the three initial slashes instead of a single slash).
4. Add a second workaround, where we add a dll import resolver to load exactly
   the path we want to load.

[1]: dotnet/runtime@5a1baeb
[2]: dotnet/runtime#85255

Technical sidenote:

Why trying to load "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit"
turned out so bad on Mac Catalyst is not obvious. What happens is this:

* The app calls 'dlopen ("/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit")'
* dlopen checks if this is a Mac Catalyst override of a macOS system
  framework, by prefixing "/System/iOSSupport" and trying to load that. So
  dlopen would try to load "/System/iOSSupport/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
  which would obviously fail.
* Then dlopen would try a few more fallbacks, eventually trying
  "/System/Library/Frameworks/SceneKit.framework/SceneKit", and successfully
  loading that library.
* Unfortunately "/System/Library/Frameworks/SceneKit.framework/SceneKit" is
  the wrong library to load for Mac Catalyst ("/System/iOSSupport/System/Library/Frameworks/SceneKit.framework/SceneKit"
  is the correct version). These two libraries are incompatible, and calling
  one when you mean to call the other will do nasty things like corrupting the
  stack.
rolfbjarne added a commit to rolfbjarne/xamarin-macios that referenced this pull request Apr 27, 2023
1. Mono changed dyld lookup to start looking in directories in
   NATIVE_DLL_SEARCH_DIRECTORIES before the actual given path, even when the
   given path is absolute [1].
2. This turned out to break Mac Catalyst, because when a DllImport says a
   P/Invoke is in "/System/Library/Frameworks/SceneKit.framework/SceneKit",
   Mono would try loading by prefixing the directories in
   NATIVE_DLL_SEARCH_DIRECTORIES. We add the Contents/MonoBundle directory to
   NATIVE_DLL_SEARCH_DIRECTORIES, so Mono would try to load
   "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
   and things would go wrong.
3. We found a workaround: add "/" to NATIVE_DLL_SEARCH_DIRECTORIES. This works
   on Ventura, but apparently not on older macOS version, because the actual
   path we pass to dlopen ends up being "///System/Library/Frameworks/SceneKit.framework/SceneKit"
   (note the three initial slashes instead of a single slash).
4. Add a second workaround, where we add a dll import resolver to load exactly
   the path we want to load.

[1]: dotnet/runtime@5a1baeb
[2]: dotnet/runtime#85255

Technical sidenote:

Why trying to load "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit"
turned out so bad on Mac Catalyst is not obvious. What happens is this:

* The app calls 'dlopen ("/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit")'
* dlopen checks if this is a Mac Catalyst override of a macOS system
  framework, by prefixing "/System/iOSSupport" and trying to load that. So
  dlopen would try to load "/System/iOSSupport/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
  which would obviously fail.
* Then dlopen would try a few more fallbacks, eventually trying
  "/System/Library/Frameworks/SceneKit.framework/SceneKit", and successfully
  loading that library.
* Unfortunately "/System/Library/Frameworks/SceneKit.framework/SceneKit" is
  the wrong library to load for Mac Catalyst ("/System/iOSSupport/System/Library/Frameworks/SceneKit.framework/SceneKit"
  is the correct version). These two libraries are incompatible, and calling
  one when you mean to call the other will do nasty things like corrupting the
  stack.

This is a backport of #xamarin#18159.
rolfbjarne added a commit to rolfbjarne/xamarin-macios that referenced this pull request Apr 27, 2023
…ccore@2668.

1. Mono changed dyld lookup to start looking in directories in
   NATIVE_DLL_SEARCH_DIRECTORIES before the actual given path, even when the
   given path is absolute [1].
2. This turned out to break Mac Catalyst, because when a DllImport says a
   P/Invoke is in "/System/Library/Frameworks/SceneKit.framework/SceneKit",
   Mono would try loading by prefixing the directories in
   NATIVE_DLL_SEARCH_DIRECTORIES. We add the Contents/MonoBundle directory to
   NATIVE_DLL_SEARCH_DIRECTORIES, so Mono would try to load
   "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
   and things would go wrong.
3. We found a workaround: add "/" to NATIVE_DLL_SEARCH_DIRECTORIES. This works
   on Ventura, but apparently not on older macOS version, because the actual
   path we pass to dlopen ends up being "///System/Library/Frameworks/SceneKit.framework/SceneKit"
   (note the three initial slashes instead of a single slash).
4. Add a second workaround, where we add a dll import resolver to load exactly
   the path we want to load.

[1]: dotnet/runtime@5a1baeb
[2]: dotnet/runtime#85255

Technical sidenote:

Why trying to load "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit"
turned out so bad on Mac Catalyst is not obvious. What happens is this:

* The app calls 'dlopen ("/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit")'
* dlopen checks if this is a Mac Catalyst override of a macOS system
  framework, by prefixing "/System/iOSSupport" and trying to load that. So
  dlopen would try to load "/System/iOSSupport/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
  which would obviously fail.
* Then dlopen would try a few more fallbacks, eventually trying
  "/System/Library/Frameworks/SceneKit.framework/SceneKit", and successfully
  loading that library.
* Unfortunately "/System/Library/Frameworks/SceneKit.framework/SceneKit" is
  the wrong library to load for Mac Catalyst ("/System/iOSSupport/System/Library/Frameworks/SceneKit.framework/SceneKit"
  is the correct version). These two libraries are incompatible, and calling
  one when you mean to call the other will do nasty things like corrupting the
  stack.

This is a backport of xamarin#18159.
rolfbjarne pushed a commit to xamarin/xamarin-macios that referenced this pull request Apr 28, 2023
…ccore@2668. (#18167)

1. Mono changed dyld lookup to start looking in directories in
   NATIVE_DLL_SEARCH_DIRECTORIES before the actual given path, even when the
   given path is absolute [1].
2. This turned out to break Mac Catalyst, because when a DllImport says a
   P/Invoke is in "/System/Library/Frameworks/SceneKit.framework/SceneKit",
   Mono would try loading by prefixing the directories in
   NATIVE_DLL_SEARCH_DIRECTORIES. We add the Contents/MonoBundle directory to
   NATIVE_DLL_SEARCH_DIRECTORIES, so Mono would try to load
   "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
   and things would go wrong.
3. We found a workaround: add "/" to NATIVE_DLL_SEARCH_DIRECTORIES. This works
   on Ventura, but apparently not on older macOS version, because the actual
   path we pass to dlopen ends up being "///System/Library/Frameworks/SceneKit.framework/SceneKit"
   (note the three initial slashes instead of a single slash).
4. Add a second workaround, where we add a dll import resolver to load exactly
   the path we want to load.

[1]: dotnet/runtime@5a1baeb
[2]: dotnet/runtime#85255

Technical sidenote:

Why trying to load "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit"
turned out so bad on Mac Catalyst is not obvious. What happens is this:

* The app calls 'dlopen ("/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit")'
* dlopen checks if this is a Mac Catalyst override of a macOS system
  framework, by prefixing "/System/iOSSupport" and trying to load that. So
  dlopen would try to load "/System/iOSSupport/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
  which would obviously fail.
* Then dlopen would try a few more fallbacks, eventually trying
  "/System/Library/Frameworks/SceneKit.framework/SceneKit", and successfully
  loading that library.
* Unfortunately "/System/Library/Frameworks/SceneKit.framework/SceneKit" is
  the wrong library to load for Mac Catalyst ("/System/iOSSupport/System/Library/Frameworks/SceneKit.framework/SceneKit"
  is the correct version). These two libraries are incompatible, and calling
  one when you mean to call the other will do nasty things like corrupting the
  stack.


Backport of #18159
rolfbjarne added a commit to xamarin/xamarin-macios that referenced this pull request Apr 28, 2023
…ccore@2668. (#18172)

1. Mono changed dyld lookup to start looking in directories in
   NATIVE_DLL_SEARCH_DIRECTORIES before the actual given path, even when the
   given path is absolute [1].
2. This turned out to break Mac Catalyst, because when a DllImport says a
   P/Invoke is in "/System/Library/Frameworks/SceneKit.framework/SceneKit",
   Mono would try loading by prefixing the directories in
   NATIVE_DLL_SEARCH_DIRECTORIES. We add the Contents/MonoBundle directory to
   NATIVE_DLL_SEARCH_DIRECTORIES, so Mono would try to load
   "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
   and things would go wrong.
3. We found a workaround: add "/" to NATIVE_DLL_SEARCH_DIRECTORIES. This works
   on Ventura, but apparently not on older macOS version, because the actual
   path we pass to dlopen ends up being "///System/Library/Frameworks/SceneKit.framework/SceneKit"
   (note the three initial slashes instead of a single slash).
4. Add a second workaround, where we add a dll import resolver to load exactly
   the path we want to load.

[1]: dotnet/runtime@5a1baeb
[2]: dotnet/runtime#85255

Technical sidenote:

Why trying to load "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit"
turned out so bad on Mac Catalyst is not obvious. What happens is this:

* The app calls 'dlopen ("/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit")'
* dlopen checks if this is a Mac Catalyst override of a macOS system
  framework, by prefixing "/System/iOSSupport" and trying to load that. So
  dlopen would try to load "/System/iOSSupport/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
  which would obviously fail.
* Then dlopen would try a few more fallbacks, eventually trying
  "/System/Library/Frameworks/SceneKit.framework/SceneKit", and successfully
  loading that library.
* Unfortunately "/System/Library/Frameworks/SceneKit.framework/SceneKit" is
  the wrong library to load for Mac Catalyst ("/System/iOSSupport/System/Library/Frameworks/SceneKit.framework/SceneKit"
  is the correct version). These two libraries are incompatible, and calling
  one when you mean to call the other will do nasty things like corrupting the
  stack.


Backport of #18159
rolfbjarne added a commit to xamarin/xamarin-macios that referenced this pull request Apr 28, 2023
…ccore@2668. (#18171)

1. Mono changed dyld lookup to start looking in directories in
   NATIVE_DLL_SEARCH_DIRECTORIES before the actual given path, even when the
   given path is absolute [1].
2. This turned out to break Mac Catalyst, because when a DllImport says a
   P/Invoke is in "/System/Library/Frameworks/SceneKit.framework/SceneKit",
   Mono would try loading by prefixing the directories in
   NATIVE_DLL_SEARCH_DIRECTORIES. We add the Contents/MonoBundle directory to
   NATIVE_DLL_SEARCH_DIRECTORIES, so Mono would try to load
   "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
   and things would go wrong.
3. We found a workaround: add "/" to NATIVE_DLL_SEARCH_DIRECTORIES. This works
   on Ventura, but apparently not on older macOS version, because the actual
   path we pass to dlopen ends up being "///System/Library/Frameworks/SceneKit.framework/SceneKit"
   (note the three initial slashes instead of a single slash).
4. Add a second workaround, where we add a dll import resolver to load exactly
   the path we want to load.

[1]: dotnet/runtime@5a1baeb
[2]: dotnet/runtime#85255

Technical sidenote:

Why trying to load "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit"
turned out so bad on Mac Catalyst is not obvious. What happens is this:

* The app calls 'dlopen ("/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit")'
* dlopen checks if this is a Mac Catalyst override of a macOS system
  framework, by prefixing "/System/iOSSupport" and trying to load that. So
  dlopen would try to load "/System/iOSSupport/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
  which would obviously fail.
* Then dlopen would try a few more fallbacks, eventually trying
  "/System/Library/Frameworks/SceneKit.framework/SceneKit", and successfully
  loading that library.
* Unfortunately "/System/Library/Frameworks/SceneKit.framework/SceneKit" is
  the wrong library to load for Mac Catalyst ("/System/iOSSupport/System/Library/Frameworks/SceneKit.framework/SceneKit"
  is the correct version). These two libraries are incompatible, and calling
  one when you mean to call the other will do nasty things like corrupting the
  stack.

This is a backport of #18159.
rolfbjarne pushed a commit to xamarin/xamarin-macios that referenced this pull request Apr 28, 2023
#18166)

1. Mono changed dyld lookup to start looking in directories in
   NATIVE_DLL_SEARCH_DIRECTORIES before the actual given path, even when the
   given path is absolute [1].
2. This turned out to break Mac Catalyst, because when a DllImport says a
   P/Invoke is in "/System/Library/Frameworks/SceneKit.framework/SceneKit",
   Mono would try loading by prefixing the directories in
   NATIVE_DLL_SEARCH_DIRECTORIES. We add the Contents/MonoBundle directory to
   NATIVE_DLL_SEARCH_DIRECTORIES, so Mono would try to load
   "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
   and things would go wrong.
3. We found a workaround: add "/" to NATIVE_DLL_SEARCH_DIRECTORIES. This works
   on Ventura, but apparently not on older macOS version, because the actual
   path we pass to dlopen ends up being "///System/Library/Frameworks/SceneKit.framework/SceneKit"
   (note the three initial slashes instead of a single slash).
4. Add a second workaround, where we add a dll import resolver to load exactly
   the path we want to load.

[1]: dotnet/runtime@5a1baeb
[2]: dotnet/runtime#85255

Technical sidenote:

Why trying to load "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit"
turned out so bad on Mac Catalyst is not obvious. What happens is this:

* The app calls 'dlopen ("/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit")'
* dlopen checks if this is a Mac Catalyst override of a macOS system
  framework, by prefixing "/System/iOSSupport" and trying to load that. So
  dlopen would try to load "/System/iOSSupport/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
  which would obviously fail.
* Then dlopen would try a few more fallbacks, eventually trying
  "/System/Library/Frameworks/SceneKit.framework/SceneKit", and successfully
  loading that library.
* Unfortunately "/System/Library/Frameworks/SceneKit.framework/SceneKit" is
  the wrong library to load for Mac Catalyst ("/System/iOSSupport/System/Library/Frameworks/SceneKit.framework/SceneKit"
  is the correct version). These two libraries are incompatible, and calling
  one when you mean to call the other will do nasty things like corrupting the
  stack.

This is a backport of #18159.
rolfbjarne added a commit to rolfbjarne/xamarin-macios that referenced this pull request Apr 28, 2023
1. Mono changed dyld lookup to start looking in directories in
   NATIVE_DLL_SEARCH_DIRECTORIES before the actual given path, even when the
   given path is absolute [1].
2. This turned out to break Mac Catalyst, because when a DllImport says a
   P/Invoke is in "/System/Library/Frameworks/SceneKit.framework/SceneKit",
   Mono would try loading by prefixing the directories in
   NATIVE_DLL_SEARCH_DIRECTORIES. We add the Contents/MonoBundle directory to
   NATIVE_DLL_SEARCH_DIRECTORIES, so Mono would try to load
   "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
   and things would go wrong.
3. We found a workaround: add "/" to NATIVE_DLL_SEARCH_DIRECTORIES. This works
   on Ventura, but apparently not on older macOS version, because the actual
   path we pass to dlopen ends up being "///System/Library/Frameworks/SceneKit.framework/SceneKit"
   (note the three initial slashes instead of a single slash).
4. Add a second workaround, where we add a dll import resolver to load exactly
   the path we want to load.

[1]: dotnet/runtime@5a1baeb
[2]: dotnet/runtime#85255

Technical sidenote:

Why trying to load "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit"
turned out so bad on Mac Catalyst is not obvious. What happens is this:

* The app calls 'dlopen ("/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit")'
* dlopen checks if this is a Mac Catalyst override of a macOS system
  framework, by prefixing "/System/iOSSupport" and trying to load that. So
  dlopen would try to load "/System/iOSSupport/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
  which would obviously fail.
* Then dlopen would try a few more fallbacks, eventually trying
  "/System/Library/Frameworks/SceneKit.framework/SceneKit", and successfully
  loading that library.
* Unfortunately "/System/Library/Frameworks/SceneKit.framework/SceneKit" is
  the wrong library to load for Mac Catalyst ("/System/iOSSupport/System/Library/Frameworks/SceneKit.framework/SceneKit"
  is the correct version). These two libraries are incompatible, and calling
  one when you mean to call the other will do nasty things like corrupting the
  stack.

This is a backport of xamarin#18159.
vs-mobiletools-engineering-service2 pushed a commit to vs-mobiletools-engineering-service2/xamarin-macios that referenced this pull request Apr 28, 2023
1. Mono changed dyld lookup to start looking in directories in
   NATIVE_DLL_SEARCH_DIRECTORIES before the actual given path, even when the
   given path is absolute [1].
2. This turned out to break Mac Catalyst, because when a DllImport says a
   P/Invoke is in "/System/Library/Frameworks/SceneKit.framework/SceneKit",
   Mono would try loading by prefixing the directories in
   NATIVE_DLL_SEARCH_DIRECTORIES. We add the Contents/MonoBundle directory to
   NATIVE_DLL_SEARCH_DIRECTORIES, so Mono would try to load
   "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
   and things would go wrong.
3. We found a workaround: add "/" to NATIVE_DLL_SEARCH_DIRECTORIES. This works
   on Ventura, but apparently not on older macOS version, because the actual
   path we pass to dlopen ends up being "///System/Library/Frameworks/SceneKit.framework/SceneKit"
   (note the three initial slashes instead of a single slash).
4. Add a second workaround, where we add a dll import resolver to load exactly
   the path we want to load.

[1]: dotnet/runtime@5a1baeb
[2]: dotnet/runtime#85255

Technical sidenote:

Why trying to load "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit"
turned out so bad on Mac Catalyst is not obvious. What happens is this:

* The app calls 'dlopen ("/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit")'
* dlopen checks if this is a Mac Catalyst override of a macOS system
  framework, by prefixing "/System/iOSSupport" and trying to load that. So
  dlopen would try to load "/System/iOSSupport/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
  which would obviously fail.
* Then dlopen would try a few more fallbacks, eventually trying
  "/System/Library/Frameworks/SceneKit.framework/SceneKit", and successfully
  loading that library.
* Unfortunately "/System/Library/Frameworks/SceneKit.framework/SceneKit" is
  the wrong library to load for Mac Catalyst ("/System/iOSSupport/System/Library/Frameworks/SceneKit.framework/SceneKit"
  is the correct version). These two libraries are incompatible, and calling
  one when you mean to call the other will do nasty things like corrupting the
  stack.
@elinor-fung
Copy link
Member

Thank you for handling this!

rolfbjarne pushed a commit to xamarin/xamarin-macios that referenced this pull request May 3, 2023
…core@2668. (#18181)

1. Mono changed dyld lookup to start looking in directories in
   NATIVE_DLL_SEARCH_DIRECTORIES before the actual given path, even when the
   given path is absolute [1].
2. This turned out to break Mac Catalyst, because when a DllImport says a
   P/Invoke is in "/System/Library/Frameworks/SceneKit.framework/SceneKit",
   Mono would try loading by prefixing the directories in
   NATIVE_DLL_SEARCH_DIRECTORIES. We add the Contents/MonoBundle directory to
   NATIVE_DLL_SEARCH_DIRECTORIES, so Mono would try to load
   "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
   and things would go wrong.
3. We found a workaround: add "/" to NATIVE_DLL_SEARCH_DIRECTORIES. This works
   on Ventura, but apparently not on older macOS version, because the actual
   path we pass to dlopen ends up being "///System/Library/Frameworks/SceneKit.framework/SceneKit"
   (note the three initial slashes instead of a single slash).
4. Add a second workaround, where we add a dll import resolver to load exactly
   the path we want to load.

[1]: dotnet/runtime@5a1baeb
[2]: dotnet/runtime#85255

Technical sidenote:

Why trying to load "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit"
turned out so bad on Mac Catalyst is not obvious. What happens is this:

* The app calls 'dlopen ("/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit")'
* dlopen checks if this is a Mac Catalyst override of a macOS system
  framework, by prefixing "/System/iOSSupport" and trying to load that. So
  dlopen would try to load "/System/iOSSupport/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
  which would obviously fail.
* Then dlopen would try a few more fallbacks, eventually trying
  "/System/Library/Frameworks/SceneKit.framework/SceneKit", and successfully
  loading that library.
* Unfortunately "/System/Library/Frameworks/SceneKit.framework/SceneKit" is
  the wrong library to load for Mac Catalyst ("/System/iOSSupport/System/Library/Frameworks/SceneKit.framework/SceneKit"
  is the correct version). These two libraries are incompatible, and calling
  one when you mean to call the other will do nasty things like corrupting the
  stack.


Backport of #18159
rolfbjarne added a commit to xamarin/xamarin-macios that referenced this pull request May 3, 2023
#18178)

1. Mono changed dyld lookup to start looking in directories in
   NATIVE_DLL_SEARCH_DIRECTORIES before the actual given path, even when the
   given path is absolute [1].
2. This turned out to break Mac Catalyst, because when a DllImport says a
   P/Invoke is in "/System/Library/Frameworks/SceneKit.framework/SceneKit",
   Mono would try loading by prefixing the directories in
   NATIVE_DLL_SEARCH_DIRECTORIES. We add the Contents/MonoBundle directory to
   NATIVE_DLL_SEARCH_DIRECTORIES, so Mono would try to load
   "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
   and things would go wrong.
3. We found a workaround: add "/" to NATIVE_DLL_SEARCH_DIRECTORIES. This works
   on Ventura, but apparently not on older macOS version, because the actual
   path we pass to dlopen ends up being "///System/Library/Frameworks/SceneKit.framework/SceneKit"
   (note the three initial slashes instead of a single slash).
4. Add a second workaround, where we add a dll import resolver to load exactly
   the path we want to load.

[1]: dotnet/runtime@5a1baeb
[2]: dotnet/runtime#85255

Technical sidenote:

Why trying to load "/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit"
turned out so bad on Mac Catalyst is not obvious. What happens is this:

* The app calls 'dlopen ("/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit")'
* dlopen checks if this is a Mac Catalyst override of a macOS system
  framework, by prefixing "/System/iOSSupport" and trying to load that. So
  dlopen would try to load "/System/iOSSupport/path/to/my.app/Contents/MonoBundle//System/Library/Frameworks/SceneKit.framework/SceneKit",
  which would obviously fail.
* Then dlopen would try a few more fallbacks, eventually trying
  "/System/Library/Frameworks/SceneKit.framework/SceneKit", and successfully
  loading that library.
* Unfortunately "/System/Library/Frameworks/SceneKit.framework/SceneKit" is
  the wrong library to load for Mac Catalyst ("/System/iOSSupport/System/Library/Frameworks/SceneKit.framework/SceneKit"
  is the correct version). These two libraries are incompatible, and calling
  one when you mean to call the other will do nasty things like corrupting the
  stack.

This is a backport of #18159.
@ghost ghost locked as resolved and limited conversation to collaborators May 31, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants