(macOS BS) There are no Frameworks in /System/Library/Frameworks to be loaded with Pyobjc

[LcTrKiD]

Using pyobjc from system python and PyObjc 6.2.1 (Python 3.8.3) when I try to import CoreServices framework it won't load because /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (and no other Framework) exist.

>>> import CoreServices
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/munki/Python.framework/Versions/3.7/lib/python3.7/site-packages/CoreServices/__init__.py", line 15, in <module>
    from CoreServices import LaunchServices
  File "/usr/local/munki/Python.framework/Versions/3.7/lib/python3.7/site-packages/CoreServices/LaunchServices/__init__.py", line 17, in <module>
    objc.pathForFramework('/System/Library/Frameworks/CoreServices.framework/CoreServices'),
  File "/usr/local/munki/Python.framework/Versions/3.7/lib/python3.7/site-packages/objc/_dyld.py", line 122, in pathForFramework
    fpath, name, version = infoForFramework(dyld_find(path))
  File "/usr/local/munki/Python.framework/Versions/3.7/lib/python3.7/site-packages/objc/_dyld.py", line 117, in dyld_find
    return dyld_framework(filename, framework_name, version)
  File "/usr/local/munki/Python.framework/Versions/3.7/lib/python3.7/site-packages/objc/_dyld.py", line 88, in dyld_framework
    raise ImportError("Framework %s could not be found" % (framework_name,))
ImportError: Framework CoreServices could not be found

[macmule]

With Python3 installed from command-line-tools:

sh-3.2# python3
Python 3.8.2 (default, Jun  5 2020, 03:04:24) 
[Clang 12.0.0 (clang-1200.0.22.7)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import CoreServices
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'CoreServices'

[LcTrKiD]

I found this in https://mrmacintosh.com/whats-new-in-macos-big-sur-11-0-beta-1-20a4299v/

New in macOS Big Sur 11 beta, the system ships with a built-in dynamic linker cache of all system-provided libraries. As part of this change, copies of dynamic libraries are no longer present on the filesystem. Code that attempts to check for dynamic library presence by looking for a file at a path or enumerating a directory will fail. Instead, check for library presence by attempting to dlopen() the path, which will correctly check for the library in the cache. (62986286)

[knightsc]

@LcTrKiD yup that is exactly what's going on. All the shared libraries are smashed into a dylib shared cache like on iOS. On macOS it can be found here: /System/Library/dyld/dyld_shared_cache_x86_64.

The code in _dyld.py needs to be updated so it's not calling os.path exists. This nasty hack will make things go

for f in inject_suffixes(_search()):
        # if os.path.exists(f):
        return f

But the ideal solution should be to not check paths at all and instead try to load the dylib. If the dylib loading works then it's valid otherwise it must not exist.

The entire dylib_shared_cache is loaded at startup and all dylibs just hang around in memory

[LcTrKiD]

I’m going to diff _dyld.py with older versions since this works with system python and an old version of pyobjc

[maxbelanger]

We've fixed this temporarily in Dropbox via (roughly) the following swizzling:

    import objc
    import objc._dyld

    def __path_for_framework_safe(path: str) -> str:
        return path

    objc._dyld.pathForFramework = __path_for_framework_safe
    objc.pathForFramework = __path_for_framework_safe

This means pyobjc always dlopens (via NSBundle) based on the canonical and absolute path of the framework, which works with the cache.

[ronaldoussoren]

The head of the trunk should work on BS with the commit I did a just now. I did a minimal fix based on Max's patch: Return the input path when it starts with "/System/" in "objc.pathForFramework". With this patch "import CoreServices" works for me (well, the same change in the installed version on the beta worked; there's a non-zero change that I botched copying this trivial change to the system I did the commit from).

[ronaldoussoren]

Time to close the issue.

Note that the branch for pyobjc-7 has a more correct fix, but the difference in semantics shouldn't affect normal use of PyObjC and that fix will have to be changed again (it uses a private API that should be made public in the next beta).