Next Steps for rules_spm - Please Vote!

[cgrindel]

TL;DR: Please let us know how you would like to see rules_spm move forward.

How rules_spm Works Today

The current implementation of rules_spm uses Swift Package Manager (SPM) to fetch and build packages. The ruleset then generates Bazel targets to expose the built artifacts to the Bazel workspace. The original reasoning for this approach was that SPM would handle the heavy lifting obviating the need for rules_spm to constantly mirror SPM features and syntax changes.

Problems with this Approach

Generally, this approach has worked well. However, there have been a few gotchas.

First, one needs to understand quite a bit about clang artifacts to expose them as Bazel targets (e.g. identify public headers, parse module map files). To ensure that rules_spm could do this properly, its implementation includes a good deal of logic that inspects clang source files. In fact, a Starlark module.modulemap parser was created to ensure that rules_spm retrieved clang module information properly. In short, there is no free lunch when it comes to handling clang targets.

Second, the current implementation does not handle target compilation gracefully (e.g., build for iOS on macOS). In theory, this could be rectified, but it would need to do a better job getting its configuration information such that Bazel transitions are well supported.

Third, there are issues with duplicate symbol errors when the client project and a client project dependency both load the same dependency.

• Depend on package but not its dependencies #148
• Handle project that uses rules_spm and one of its dependencies also uses rules_spm #86

Lastly, there are some clunky aspects to how dependent packages are declared in spm_repositories. Namely, the client must specify the products that they want to use.

Alternative: Build with Bazel

Another option is to automate what most folks do today. They create Bazel build files for their dependencies and let Bazel build and link everything. The biggest benefits are:

• Bazel builds and configuration propagation just work.
• Smarter builds that can selectively build only what the client project needs.

A couple of weeks ago, I embarked on a POC to understand what it would take to implement build with Bazel. This comment from the issue summarizes my findings.

Next Steps

Below is a poll listing the options for how to move forward with this project. Please vote for your preferred path forward. Also, don't hesitate to add a reply to this discussion with any additional feedback.

Option 1: Fetch with SPM, Build with Bazel in Pure Starlark (no external tool download)

Add build with Bazel mode to spm_repositories. The ruleset will fetch with SPM and build with Bazel. We will implement a Swift package parser in Starlark. It will work in a similar fashion as the existing module map parser. Changes to SPM over time may require more updates to rules_spm.

Option 2: Fetch with SPM, Build with Bazel using Swift Binary (external tool download)

Add build with Bazel mode to spm_repositories. The ruleset will fetch with SPM and build with Bazel. We will implement a Swift package parser in Swift. This external Swift parser will be a binary download, hosted in a separate GitHub repository that provides binary releases for a common set of platforms (OS: Linux/macOS, CPU: x86_64/arm64). You may need to provide your own version of the binary if your system is not supported.

Option 3: Fetch and Build with Bazel, Executable that Generates Bazel repository and target declarations

Deprecate rules_spm and transition to a new project that provides an executable that merely generates Bazel declarations to fetch and build packages. Bazel would fetch and build the dependencies.

Option 4: Live with the Current Implementation

Live with the existing warts of the system and try to address any major shortcomings.

For more information about the options, please review this summary from the POC.

[pswaminathan]

Option 3: Fetch and Build with Bazel, Executable that Generates Bazel repository and target declarations

Deprecate rules_spm and transition to a new project that provides an executable that merely generates Bazel declarations to fetch and build packages. Bazel would fetch and build the dependencies.

I'm unsure of what this means. Would this mean something like: the new executable generates declarations to drop in an e.g. WORKSPACE file? That is—something akin to Gazelle's update-repos?

[cgrindel]

Basically, yes.

[cgrindel]

Thanks to everyone for their feedback. I will put together a design document that attempts to address everyone's feedback and lays out the next steps.