Improve the use of both_libraries #16452

Job	Run time
Arch Linux	20m 8s
Arch / PyPy	21m 7s
CUDA (on Arch)	7m 44s
Ubuntu Rolling (gcc, g++)	19m 45s
Fedora	20m 6s
Ubuntu Rolling (--unity=on -Ddefault_library=static, --no-unittests, gcc, g++)	13m 53s
Gentoo	21m 37s
OpenSUSE	16m 22s
Ubuntu Rolling (-Ddefault_library=both, --no-unittests, gcc, g++)	14m 5s
Ubuntu Bionic	15m 21s
Ubuntu Rolling (clang, clang++)	22m 31s
Ubuntu Rolling (--unity=on, --no-unittests, clang, clang++)	15m 30s
Ubuntu Rolling (--cross ubuntu-armhf.json --cross linux-mingw-w64-64bit.json, 0, gcc, g++)	13m 57s
Ubuntu Rolling (--cross ubuntu-armhf.json --cross linux-mingw-w64-64bit.json, --unity=on, gcc, g++)	13m 30s
	3h 55m 36s

Provide feedback