Reapply size optimizations for clang & MSVC, LTO for Mac+Linux #1920

Job	Run time
windows-2019.Debug.shared.VS-16-2019	4m 59s
ubuntu-22.04.Release.shared.g++	2m 15s
windows-2019.Debug.static.VS-16-2019	3m 51s
ubuntu-22.04.Release.static.g++	2m 44s
windows-2019.Release.shared.VS-16-2019	4m 27s
ubuntu-22.04.Release.shared.clang++	3m 9s
windows-2019.Release.static.VS-16-2019	4m 7s
ubuntu-22.04.Release.static.clang++	2m 10s
ubuntu-22.04.Debug.shared.g++	2m 5s
windows-2022.Debug.shared.VS-17-2022	5m 11s
ubuntu-22.04.Debug.static.g++	2m 45s
windows-2022.Debug.static.VS-17-2022	5m 26s
windows-2022.Release.shared.VS-17-2022	4m 40s
ubuntu-22.04.Debug.shared.clang++	2m 46s
windows-2022.Release.static.VS-17-2022	4m 26s
ubuntu-22.04.Debug.static.clang++	2m 56s
ubuntu-20.04.Release.shared.g++	2m 13s
ubuntu-20.04.Release.static.g++	3m 1s
ubuntu-20.04.Release.shared.clang++	2m 10s
ubuntu-20.04.Release.static.clang++	3m 5s
ubuntu-20.04.Debug.shared.g++	2m 56s
ubuntu-20.04.Debug.static.g++	2m 30s
ubuntu-20.04.Debug.shared.clang++	1m 58s
ubuntu-20.04.Debug.static.clang++	2m 16s
macos-latest.Release.shared.g++	3m 49s
macos-latest.Release.static.g++	3m 35s
macos-latest.Release.shared.clang++	4m 25s
macos-latest.Release.static.clang++	2m 55s
macos-latest.Debug.shared.g++	2m 31s
macos-latest.Debug.static.g++	2m 55s
macos-latest.Debug.shared.clang++	3m 33s
macos-latest.Debug.static.clang++	3m 49s
	1h 45m 38s

Provide feedback