SwiftPM requires librarian to be installed before it can build anything

[finagolfin]

Description

This was introduced by #5720, which moved detecting the archiver executable from inside of llbuild to SPM. This has always been kind of a mess with SPM, with llbuild always grabbing whatever archiver was in the path. This didn't crop up much unless you were building the rare Swift package that had a static library and happened not to have a binutils package installed in your system.

However, trunk SPM now won't build anything without an archiver included in the Swift toolchain or system path, which is too strict. This is particularly an issue because the Swift toolchain doesn't ship with an archiver: I just checked when debugging this issue and was surprised to find that the official builds don't include llvm-ar.

The Swift compiler itself is much better on this issue, as you can supply an alternate -tools-directory flag where it should look for an archiver, as done for Android. I just ran a variant of that compiler command and confirmed it uses the NDK llvm-ar:

> ./swift-5.6.3-RELEASE-ubuntu20.04/usr/bin/swiftc -tools-directory /home/butta/android-ndk-r25b/toolchains/llvm/prebuilt/linux-x86_64/bin
-target aarch64-unknown-linux-android24 -sdk android-ndk-r25b/toolchains/llvm/prebuilt/linux-x86_64/sysroot
-resource-dir swift-5.6.3-android-aarch64-24-sdk/usr/lib/swift swift/test/Interpreter/Inputs/lto/module1.swift
-emit-library -static -v
...
/home/butta/android-ndk-r25b/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar crs libmodule1.a /tmp/TemporaryDirectory.yREHfG/module1-1.o

However, I see no flag to do the same with SPM, whereas I could always pass the -tools-directory I want to SPM with the extra swiftc flags before. With SPM itself checking for an archiver now, it errors before that compiler flag is ever used. I worked around this issue on my Android CI by adding the NDK llvm-ar to my system path.

There are various ways to fix this: my preference would be to start shipping llvm-ar with the Swift toolchain and make that the default archiver we use.

Expected behavior

Don't require ar or llvm-ar to be installed before building any Swift package, particularly since most won't use it.

Actual behavior

error: toolchain is invalid: could not find llvm-ar

Steps to reproduce

1. Make sure binutils is not installed.
2. Try to build any Swift package.

I realize that binutils is listed as a dependency of the Swift toolchain on linux, but when cross-compiling for other platforms and supplying your own archiver, as I'm doing for Android, it shouldn't be required that a system archiver be installed.

Swift Package Manager version/commit hash

Tested with the September 5 trunk snapshot

Swift & OS version (output of swift --version && uname -a)

September 5 trunk snapshot running on Ubuntu 20.04 x86_64

[tomerd]

cc @compnerd @abertelrud

[tomerd]

we can roll back #5720 if necessary, but I'd like to see what @compnerd thinks as he wanted this change to fix things on Windows.

[finagolfin]

This is a bit of a corner case, so no need to revert #5720. We should put some thought into how we handle this though, and do something to at least soften this new requirement before the 5.8 branch.

[compnerd]

We already have the dependency on ar for things like swift-syntax which is now being built as part of the toolchain itself. I think that adjusting the path is reasonable, though an alternative approach would be to set the AR environment variable. Packaging llvm-ar is completely reasonable and something that we should do though.

[finagolfin]

Starting with the latest Dec. 19 trunk snapshot tag, the Swift toolchain now bundles in llvm-ar, which fixes my original Android problem.

However, I'd like to use this bundled llvm-ar for all Unix platforms from now on, not just for Android, by changing the default archiver the Swift compiler looks for, which I tried to do initially in swiftlang/swift#62510 but which will require installing llvm-ar also in the linux CI (the corelibs aren't built with the forked LLVM tools, so it's easier to just install another llvm-ar in the host, which is actually how the current corelibs build works).

I'll get on that next month once the CI opens up again, leaving this issue open till then.