-
Notifications
You must be signed in to change notification settings - Fork 4.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
NativeAOT produces very large __LINKEDIT
segment on macOS
#86707
Comments
Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas Issue DetailsDescriptionNativeAOT on macOS typically produces binaries about twice the size with large, almost zero-filled Reproduction Steps
Expected behaviorWindows
Linux
Actual behaviormacOS x64
Regression?No response Known WorkaroundsNo response ConfigurationWindows
Linux
macOS x64
Other informationNo response
|
@filipnavara Do you have any insights about where the large |
Unfortunately not. I'd use XMachOViewer to figure out which link edit block is taking the space. I can look at it in more detail next week after I get back from vacation... |
@ivanpovazan started looking at it this week - in a MAUI app he sent me this segment was 20 MB and I was puzzled what it was when looking at it with a hex editor. A third of the MAUI executable was zeros. |
we've noticed a similar (but much much smaller) behaviour with MAUI iOS template app built with MonoAOT, where the section is around 200kb also zeroed out: https://gist.github.com/ivanpovazan/97ec573836caca3f8347b8eafa2cad14 |
I think the massive section of zeros is due to native stripping. At least going by the Turns out Apple’s https://opensource.apple.com/source/cctools/cctools-973.0.1/misc/strip.c.auto.html Note the call to I took a brief look at No idea if other things would go wrong if we switched from strip to llvm-objcopy, though. Presumably there were reasons not to use it on Apple platforms. But it would be worthwhile to save an unstripped binary and look at what was in that section previously. |
Stripping can be disabled with |
We're likely not invoking the strip tool correctly. I cannot imagine symbol stripping would work like this by design on Apple platforms that really care about size. Is there a way to see how stripping happens on a normal obj-c/swift/c++ Xcode project? |
There's also the reserved room for code signature at the end of the file/__LINKEDIT section. It should not be this big though. |
I think the strip tool is rarely used on Apple platforms. You just tell the linker/compiler to strip it. One of the reasons is that the linker produces ad-hoc code signatures nowadays, and any further modification has to account for the code signature. It is hard to rewrite the signature, but it's even harder to reallocate the space for the signature (you need to change size of __LINKEDIT section, which has descriptor at the beginning of file, but also calculate the size of the signature at the same time, which needs to know the sizes of all headers and sections). |
Strip it and save it into a separate file? (ie make it work just like it works with Windows linkers and .pdbs) What is the command line option for that? |
After some investigation, there seems to be something wrong with the way we are doing native linking, not stripping. Regarding the large sections being zeroed out, I am mainly referring to the: To explain it further, if I do the following with what @nxtn-staged provided as a repro:
The size of the app comes down to: The flag above will force passing to the native linker Additionally, applying the same approach to the MAUI iOS with NativeAOT issue, shaves off the 20MB of zeroes from the binary (similar benefit as described above). This should be considered when setting up Xamarin integration with NativeAOT and we could probably pass to the native linker something like:
in the |
For future reference: That doesn't quite apply to Mach-O. In Apple world the debug symbols are not propagated to the final executable. Instead the executable has an assigned GUID and there's a map that points to the original .o files which have DWARF symbols. |
@ivanpovazan That kinda rings a bell. We output some obnoxious amount of symbols in the Mach-O .o files in the ILCompiler output because the Apple linker is both buggy and picky. It basically needs to find function boundaries and match them with the DWARF exception info. Unlike ELF there are no explicit function symbols with size. It also notoriously misinterprets symbol+offset relocations in context of the debug symbols, which - once again - means outputting extra symbols (for methods with try/catch and try/finally blocks). |
Doing some reading, I don't quite understand what the MacOS standard is, but I agree that it sounds like Mac is different from Linux and it's expected for linkers/Mach-O tools to split debug symbols ahead of time. Separately, it looks like our default is to produce a "flat" file, which at best seems not-preferred and potentially not-supported, instead of a dSYM bundle, which is the preferred one. I think this is because the runtime build wants to upload a single symbol file, and dSYMs are actually directories. We may need to do some fiddling with the infra to get runtime to produce whatever it needs, but produce dSYMs for customers |
Here is another resource explaining the issue and confirms similar approach I gave above with empty I confirmed that doing |
…ies (#18408) # Description This PR reduces the application's SOD (size on disk) by making `__LINKEDIT Export Info` section smaller in the stripped Mach-O binaries. The feature is controlled by `_ExportSymbolsExplicitly` MSBuild property and can be disabled by specifying: `-p:_ExportSymbolsExplicitly=true` Fixes #18332 # Initial problem It has been noticed that during stripping, the strip tool does not resize the export info section after it removes the symbols. Instead it only zeroes out the entries (achieved by calling `prune_trie` function): - https://github.com/apple-oss-distributions/cctools/blob/cctools-986/misc/strip.c - https://github.com/apple-oss-distributions/ld64/blob/ld64-711/src/other/PruneTrie.cpp Thanks @lambdageek for helping to track this down. # Approach As Xamarin build process already collects all the [required symbols][1] needed for the application to run and preserves them during the strip phase, we can use the same file to instruct clang toolchain to export only those symbols via the command line options: `-exported_symbols_list <file>` ([source][2]). This will make the export info section only include what is necessary for the runtime - and at the same time eliminate the problem of the `strip` tool which does not resize stripped symbols. # Investigation setup The issue is observable by building and inspecting the test application: https://github.com/xamarin/xamarin-macios/blob/main/tests/dotnet/MySingleView/MySingleView.csproj and targeting iOS platform in Release mode. ## Results: | Measure | MySingleView - main | MySingleView - this PR | Diff (%) | | :--- | ---: | ---: | ---: | | SOD (bytes) | 13668940 | 13458476 | -1.5% | | .ipa (bytes) | 4829368 | 4827928 | -0.03% | Even though zeroes are compressed well, the SOD is still affected and unused section takes around 1.5% of the most simplistic app size. Much bigger impact has been noted when trying out a MAUI iOS template app with NativeAOT where the `__LINKEDIT Export Info` zeroes take up to 20MB of the SOD, but also with the regular macOS applications: dotnet/runtime#86707 ### Repro current state of MySingleView.app with stripped binary 1. Build the app (you can ignore the need to run the sample, I just did it to make sure the changes do not break anything) ```bash make run-device ``` 2. Print the load commands - [load_cmds_strip.list][3] ```bash otool -l bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > load_cmds_strip.list ``` - We are interested in the export info section: ``` cmd LC_DYLD_INFO_ONLY ... export_off 5942960 export_size 207712 ``` 3. Create a hex dump of the export info section - [hex_dump_strip.list][4] ``` bash xxd -s 5942960 -l 207712 bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > hex_dump_strip.list ``` - NOTE: Notice around ~200kb of zeroes from ~0x005ab490 to ~0x005dda00 4. Verify exported symbols are correct - [dyld_info_strip.list][5] ``` bash dyld_info -exports bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > dyld_info_strip.list ``` ### Repro current state of MySingleView.app with unstripped binary 1. Build the app (the make target preserves the symbols) ```bash make run-device-no-strip ``` 2. Print the load commands - [load_cmds_nostrip.list][6] ```bash otool -l bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > load_cmds_nostrip.list ``` - We are interested in the export info section: ``` cmd LC_DYLD_INFO_ONLY ... export_off 5942960 export_size 207712 ``` 3. Create a hex dump of the export info section - [hex_dump_nostrip.list][7] ``` bash xxd -s 5942960 -l 207712 bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > hex_dump_nostrip.list ``` - Notice that the range: ~ 0x005ab490 to ~ 0x005dda00 now includes exported symbol entries 4. Verify exported symbols are correct - [dyld_info_nostrip.list][8] ``` bash dyld_info -exports bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > dyld_info_nostrip.list ``` ### Repro the new approach 1. Build the app (the make target uses the new approach) ```bash make run-device-export-syms ``` 2. Print the load commands - [load_cmds_export.list][9] ```bash otool -l bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > load_cmds_export.list ``` - We are interested in the export info section ***notice the reduced size of the section***: ``` cmd LC_DYLD_INFO_ONLY ... export_off 5942432 export_size 1048 ``` 3. Create a hex dump of the export info section - [hex_dump_export.list][10] ``` bash xxd -s 5942432 -l 1048 bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > hex_dump_export.list ``` 4. Verify exported symbols are correct - [dyld_info_export.list][11] ``` bash dyld_info -exports bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > dyld_info_export.list ``` --- ## Additional benefits With this approach we could also switch the way strip tool is invoked to always strip all debug and local symbols via `strip -S -x` instead of passing the file with symbols to preserve. This would remove the warning that we are currently getting (which is being ignored): ``` /Applications/Xcode_14.3.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/strip: warning: removing global symbols from a final linked no longer supported. Use -exported_symbols_list at link time when building... ``` ## Other references: - https://github.com/qyang-nj/llios/blob/main/exported_symbol/README.md [1]: https://github.com/xamarin/xamarin-macios/blob/11e7883da04d80c59e4ffbbc955a3e0e0060ff90/tools/dotnet-linker/Steps/GenerateReferencesStep.cs#L38-L44 [2]: https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/DynamicLibraryDesignGuidelines.html [3]: https://gist.github.com/ivanpovazan/d53f8d10be5e4ea9f39a41ea540aa7fa [4]: https://gist.github.com/ivanpovazan/60637422f3ff8cb5f437ddd06a21d9c1 [5]: https://gist.github.com/ivanpovazan/352595ad15c2ac02f38dcb3bd4130642 [6]: https://gist.github.com/ivanpovazan/bf700161f2f3691d1d7381c98d4fa0be [7]: https://gist.github.com/ivanpovazan/44269e4fff5ebd58a4d181451e5c106f [8]: https://gist.github.com/ivanpovazan/38c5afe076502d514a77420af0e10b01 [9]: https://gist.github.com/ivanpovazan/3f663c3c630005f5a578605d48ba807e [10]: https://gist.github.com/ivanpovazan/0bb84f64281d05ab20438aeaed64f13c [11]: https://gist.github.com/ivanpovazan/78b3ba2288f53a2316b9bc46964e7e4f --------- Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
The extra size appears to be the export list, which is zeroed but not removed when the file is stripped. Exes shouldn't have any exported symbols by default, but ld will include everything by default. We'll fix it up using an ld flag. Fixes #86707
Description
NativeAOT on macOS typically produces binaries about twice the size with large, almost zero-filled
__LINKEDIT
segments. Is this expected?Reproduction Steps
Expected behavior
win-x64
linux-x64
Actual behavior
osx-x64
osx-arm64
Regression?
No response
Known Workarounds
No response
Configuration
win-x64
linux-x64
osx-x64
osx-arm64
Other information
No response
The text was updated successfully, but these errors were encountered: