incompatible_objc_framework_cleanup

[googlewalt]

We are making some internal cleanups that affect how bazel supports user provided apple frameworks.

Expected timeline:

• released with Bazel 0.26 (May 2019), off by default
• enabled by default with Bazel 0.27 (June 2019)

We expect that this change will not affect most users:

1. If you are not compiling for Apple platform using Objective-C/C++ or Swift, this change does not affect you.

2. If you are compiling for Apple platform using Objective-C/C++ or Swift, but are using the apple_static_framework_import and apple_dynamic_framework_import rules provided in the apple rules package to import your frameworks, most of this change will be transparent to you. The non-transparent change are as follows:

   1. @import statements for user-provided framework may no longer work. Such uses had been deprecated for objc_library, but happened to continue to work for user-provided frameworks. After this change it will no longer work in sandboxed mode.

      The details are as follows. Previously, if a compilation action depends on a framework, it gets all the files in the framework in its set of inputs. This change makes only the framework headers available to compilation action; notably, the framework module map is no longer available. The change makes the apple framework rules consistent with that of objc_library (which also does not make its module map available to dependent compilation actions), but it does mean that some behavior that depends on framework module maps will no longer work. For example, if you are running in sandboxed mode (--spawn_strategy=sandboxed),

      • @import statements for such frameworks will no longer work, and their use will result in compile-time errors. You can convert @import statements to equivalent #import statements as follows:

        @import Foo;
        

        to:

        #import <Foo/Foo.h>
        

      • The compiler autolink feature (-fautolink) is not supported in bazel, but if you are explicitly enabling it through copt and using @import statements to pull in link-time dependencies, this feature will no longer work and will lead to link-time errors. Instead, a framework's SDK dylib and framework dependencies should be passed in explicitly through the corresponding attributes to the apple_static_framework_import rule.

      If you are not running in sandboxed mode, these features will continue to work, but we encourage people to migrate away from them.

   2. Formerly, for swift compilation, if a swift target depends on a user-provided framework, the compile actions for the swift target will implicitly use the module map file of the framework only when that framework contains some headers needed by the sources.

      After the objc framework cleanup, the module map file for any dependent framework will be explicitly specified in the compile action via the -fmodule-map-file flag.

      It is possible that this change in behavior will expose bugs in previously unused module map files. In particular, Xcode generates an improper module map file in the degenerate case where a framework exports no headers. If you use such a module map, you will encounter the following clang error:

      error: inferred submodules require a module with an umbrella
      

      You can work around this problem deleting the "module * { export * }" line from the module map, or by excluding the module map file from the framework.

3. In the unlikely scenario where you are maintaining your own Starlark rules to handle apple frameworks, this change will affect you. We don't expect many (if any) to be in this category, but here is what you need to know to migrate your rules.

   The changes pertain to how to construct the proper objc provider to represent the frameworks. What changed is how the core files and search paths are represented.

   Formerly, to construct an objc provider for a static framework, you add all the framework files to static_framework_file. To construct an objc provider for a dynamic framework, you:

   • add all the framework files to dynamic_framework_file.
   • add all the framework directories (directory ending in .framework) to dynamic_framework_dir.

   The objc framework cleanup aims to break up the framework into its logical components: headers, module maps, libraries, and resources, so that it requires less custom support. After the objc framework cleanup, the objc providers for frameworks are constructed as follows (static and dynamic frameworks follow similar procedures):

   • Add any headers to the header field.
   • Add any header framework search paths to the framework_search_paths field.
   • Add any module map to the module_map field.
   • Add any static archive/dylib into the static_framework_file/dynamic_framework_file field, respectively.

   The following fields in the objc provider remains the same as before: always_link, force_load_library, sdk_dylib, sdk_framework, weak_sdk_framework.

   Here are the changes pertaining to the objc provider APIs:

   • The following objc provider APIs are being removed:

     • dynamic_framework_dir
     • framework_dir

   • The following objc provider APIs can continue to be used to examine the framework files, but recall that those fields now have slightly different semantics, as they are intended to store only dylibs and archives:

     • dynamic_framework_file
     • static_framework_file

   • The following objc provider API can be used to get the list of framework header search paths, i.e. to construct a dependent compilation command:

     • framework_search_path_only

   • The following objc provider APIs are new, and can be used to query the framework names and search paths of the framework static archive / dylib, i.e. to construct a dependent linking command.

     • dynamic_framework_names
     • dynamic_framework_paths
     • static_framework_names
     • static_framework_paths

   Please reference the following for how migration was done for apple/swift rules:

   • rules_apple:apple/internal/apple_framework_import.bzl
   • rules_swift:swift/internal/compiling.bzl
   • rules_swift:swift/internal/linking.bzl

[steeve]

Hello,
What was the rationale behind removing @import support from obc_library? That seems like a step backward from a user standpoint ?

[keith]

@googlewalt do you mind providing more context on removing @import support and if it's something that could come back in the future? I've heard that google doesn't use this type of import but it's very popular in the iOS community in general

[segiddins]

Not being able to use @import is likely to hamper our ability to test out adopting Bazel -- in on main repo, we have over 30,000 instances of @import, and not all of them are just importing a straightforward module -- we make use of explicit submodules in multiple places. Some more context on why support for the feature was removed, and any potential roadmap to reinstate support would be super helpful for the purposes of our evaluation.

[steeve]

If it wasn't obvious, we too use @import exclusively, so I'm adding my voice :)

[googlewalt]

Sorry for the delay in response.

Internally, use of @import is problematic because it requires modules, and the current method of caching of modules does not scale in a parallel remote execution environment. There are interoperability issues with internal tools as well.

As for support for @import in bazel, you can use @import statements in some cases, but it depends on how the corresponding module is defined:

1. If @import refers to a system framework, it works. I imagine this is the most common use case in existing code.

2. If @import refers to a user provided framework (i.e. defined via apple_static_framework_import or apple_dynamic_framework_import): it does not work, but it can be worked around for local execution if you disable sandboxing.

   Note that this is the only change in behavior related to @import caused by the objc framework cleanup.

3. If @import refers to an objc_library, it does not work. This use case has not worked for at least 2 years; I suspect it has never worked but I could be wrong.

If use case (2) is important to you, we can continue discussion here. But for use case (3) please file a new issue.

[segiddins]

Both (2) and (3) are very important to us -- currently we build all of our (350+) libraries as static frameworks, and make use of <> imports and @imports exclusively, in many cases using custom module maps.

[segiddins]

Now that I've started digging into bazel more, I'm fairly certain (2) could be addressed by treating modulemaps as headers ?

[mpalczew]

Just noticed another issue.

Let's say you have objective-C library (objc_library) A and Two swift libraries (swift_library) B and C.
And the following dependency graph
A->B->C

This is now fails, because B-Swift.h has @import "C" in it.

[ob]

Talking with @segiddins (2) could be addressed if objc_library had a cc_inputs field. I filed #10107 to discuss this.