Add `CompressSymbols` opt-out option in NativeAOT #85192

am11 · 2023-04-22T10:54:47Z

Compressed symbols option shaves ~9+ MB off of .dbg file with api template app on linux-arm64.

Even with StripSymbols=false, the "fat binary" with embedded symbols goes from 34.6M to 26.1M.

ghost · 2023-04-22T10:55:00Z

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

Compressed symbols option shaves ~9+ MB off of .dbg file with api template app on linux-arm64.

Even with StripSymbols=false, the "fat binary" with embedded symbols goes from 34.6M to 26.1M.

Fix #85188

Author:	am11
Assignees:	-
Labels:	`area-NativeAOT-coreclr`
Milestone:	-

am11 · 2023-04-22T10:56:53Z

gdb and lldb recognize compressed format, e.g. before and after the compression, the disassembly of an internal method with gdb remained the same:

# with `<LinkerArg Include="-gz=zlib" />` item in project file
$ dotnet publish -p:PublishAot=true -o dist -c Release -p:StripSymbols=true
$ gdb dist/app1 -batch -ex 'disassemble S_P_CoreLib_Internal_Runtime_CompilerHelpers_StartupCodeHelpers__InitializeModules'

Dump of assembler code for function S_P_CoreLib_Internal_Runtime_CompilerHelpers_StartupCodeHelpers__InitializeModules:
   0x0000000000244980 <+0>:	stp	x29, x30, [sp, #-80]!
   0x0000000000244984 <+4>:	stp	x19, x20, [sp, #40]
   0x0000000000244988 <+8>:	stp	x21, x22, [sp, #56]
   0x000000000024498c <+12>:	str	x23, [sp, #72]
   0x0000000000244990 <+16>:	mov	x29, sp
   0x0000000000244994 <+20>:	mov	x19, x0
   0x0000000000244998 <+24>:	mov	x21, x1
   0x000000000024499c <+28>:	mov	w20, w2
   0x00000000002449a0 <+32>:	mov	x22, x3
   0x00000000002449a4 <+36>:	mov	w23, w4
   0x00000000002449a8 <+40>:	add	x0, x29, #0x18
   0x00000000002449ac <+44>:	bl	0x212c8 <RhpReversePInvoke(ReversePInvokeFrame*)>
   0x00000000002449b0 <+48>:	mov	x0, x19
   0x00000000002449b4 <+52>:	bl	0x1e174 <RhpRegisterOsModule(HANDLE)>
   0x00000000002449b8 <+56>:	mov	x0, x19
   0x00000000002449bc <+60>:	mov	x1, x21
   0x00000000002449c0 <+64>:	mov	w2, w20
   0x00000000002449c4 <+68>:	mov	x3, x22
   0x00000000002449c8 <+72>:	mov	w4, w23
   0x00000000002449cc <+76>:	bl	0x244b80 <S_P_CoreLib_Internal_Runtime_CompilerHelpers_StartupCodeHelpers__CreateTypeManagers>
   0x00000000002449d0 <+80>:	mov	x19, x0
   0x00000000002449d4 <+84>:	sxtw	x1, w20
   0x00000000002449d8 <+88>:	adrp	x0, 0xaba000
   0x00000000002449dc <+92>:	add	x0, x0, #0x700
   0x00000000002449e0 <+96>:	bl	0x636e0 <RhpNewArray>
   0x00000000002449e4 <+100>:	mov	x20, x0
   0x00000000002449e8 <+104>:	mov	w21, wzr
   0x00000000002449ec <+108>:	ldr	w22, [x19, #8]
   0x00000000002449f0 <+112>:	cmp	w22, #0x0
   0x00000000002449f4 <+116>:	b.le	0x244a1c <S_P_CoreLib_Internal_Runtime_CompilerHelpers_StartupCodeHelpers__InitializeModules+156>
   0x00000000002449f8 <+120>:	ubfiz	x0, x21, #3, #32
   0x00000000002449fc <+124>:	add	x0, x0, #0x10
   0x0000000000244a00 <+128>:	ldr	x0, [x19, x0]
   0x0000000000244a04 <+132>:	mov	w1, w21
   0x0000000000244a08 <+136>:	mov	x2, x20
   0x0000000000244a0c <+140>:	bl	0x244d80 <S_P_CoreLib_Internal_Runtime_CompilerHelpers_StartupCodeHelpers__InitializeGlobalTablesForModule>
   0x0000000000244a10 <+144>:	add	w21, w21, #0x1
   0x0000000000244a14 <+148>:	cmp	w22, w21
   0x0000000000244a18 <+152>:	b.gt	0x2449f8 <S_P_CoreLib_Internal_Runtime_CompilerHelpers_StartupCodeHelpers__InitializeModules+120>
   0x0000000000244a1c <+156>:	mov	x0, x20
   0x0000000000244a20 <+160>:	mov	w1, #0x2                   	// #2
   0x0000000000244a24 <+164>:	bl	0x216fc0 <S_P_CoreLib_System_Runtime_RuntimeImports__RhHandleAlloc>
   0x0000000000244a28 <+168>:	adrp	x1, 0x8b2000
   0x0000000000244a2c <+172>:	add	x1, x1, #0xf40
   0x0000000000244a30 <+176>:	str	x0, [x1, #16]
   0x0000000000244a34 <+180>:	adrp	x14, 0x894000
   0x0000000000244a38 <+184>:	add	x14, x14, #0x7d0
   0x0000000000244a3c <+188>:	ldr	x14, [x14]
   0x0000000000244a40 <+192>:	add	x14, x14, #0x8
   0x0000000000244a44 <+196>:	mov	x15, x19
   0x0000000000244a48 <+200>:	bl	0x64b38 <RhpAssignRefArm64>
   0x0000000000244a4c <+204>:	str	w22, [x1, #8]
   0x0000000000244a50 <+208>:	mov	w20, wzr
   0x0000000000244a54 <+212>:	cmp	w22, #0x0
   0x0000000000244a58 <+216>:	b.le	0x244a7c <S_P_CoreLib_Internal_Runtime_CompilerHelpers_StartupCodeHelpers__InitializeModules+252>
   0x0000000000244a5c <+220>:	ubfiz	x0, x20, #3, #32
   0x0000000000244a60 <+224>:	add	x0, x0, #0x10
   0x0000000000244a64 <+228>:	ldr	x0, [x19, x0]
   0x0000000000244a68 <+232>:	mov	w1, #0xcd                  	// #205
   0x0000000000244a6c <+236>:	bl	0x244ee0 <S_P_CoreLib_Internal_Runtime_CompilerHelpers_StartupCodeHelpers__RunInitializers>
   0x0000000000244a70 <+240>:	add	w20, w20, #0x1
   0x0000000000244a74 <+244>:	cmp	w22, w20
   0x0000000000244a78 <+248>:	b.gt	0x244a5c <S_P_CoreLib_Internal_Runtime_CompilerHelpers_StartupCodeHelpers__InitializeModules+220>
   0x0000000000244a7c <+252>:	add	x0, x29, #0x18
   0x0000000000244a80 <+256>:	bl	0x21394 <RhpReversePInvokeReturn(ReversePInvokeFrame*)>
   0x0000000000244a84 <+260>:	ldr	x23, [sp, #72]
   0x0000000000244a88 <+264>:	ldp	x21, x22, [sp, #56]
   0x0000000000244a8c <+268>:	ldp	x19, x20, [sp, #40]
   0x0000000000244a90 <+272>:	ldp	x29, x30, [sp], #80
   0x0000000000244a94 <+276>:	ret
End of assembler dump.

jkotas · 2023-04-22T14:40:42Z

Compressed symbols option shaves ~9+ MB off of .dbg file with api

This is less that what I was expecting. I was expecting that this will be as effective as running gzip on the .dbg file. Here are the numbers that I am seeing on .NET 8 P3 with dotnet new api -aot:

.dbg: 37,794,320 bytes
.dbg w/ -gz: 25,524,832 bytes
gzip .dbg: 6,378,687 bytes

Why is the -gz option leaving so much on the table?

MichalPetryka · 2023-04-22T14:49:29Z

Why is the -gz option leaving so much on the table?

Maybe it uses the lowest possible compression level? Could you compare that?

am11 · 2023-04-22T14:54:49Z

Why is the -gz option leaving so much on the table?

Clang's -gz=zlib or llvm-objcopy's --compress-debug-sections=zlib only compress sections, they do not compress the whole file. Also, we do not have control over which compression level is selected.

gzip .dbg: 6,378,687 bytes

Both debuggers, lldb and gdb, are unable to load this file.

am11 · 2023-07-25T07:09:00Z

@jkotas, should we take this in 8.0?

jkotas

@jkotas, should we take this in 8.0?

Sounds reasonable to me. @agocke @MichalStrehovsky Do you have an opinion?

It may have some interactions with #89358

am11 · 2023-07-25T17:20:57Z

      Found 0 warnings and errors
      Found 0 warnings and errors, expected between 11000 and 13000

😮

Not sure if we can attribute it to compression, it's a test bug or bug in dwarfdump tool? Will try to run it locally tonight.

agocke · 2023-07-25T17:59:46Z

My suspicion is a problem with dwarfdump, but it would need investigation.

MichalStrehovsky · 2023-07-26T03:50:56Z

Sounds reasonable to me. @agocke @MichalStrehovsky Do you have an opinion?

Sounds reasonable to me too. I had a look at why we're not getting a great compression and it looks like strings don't get compressed at all.

If dwarfdump doesn't understand these, we could opt the test out of compression.

src/tests/nativeaot/SmokeTests/DwarfDump/Program.cs

am11 · 2023-07-26T04:15:54Z

With llvm-dwarfdump-16 installed from Ubuntu packages, test passed. However, the llvm-dwardump (also v16) we are packaging in SuperFileCheck assets was built without zlib support, so we get these errors:

error: failed to decompress '.debug_abbrev', LLVM was not built with LLVM_ENABLE_ZLIB or did not find zlib at build time
error: failed to decompress '.debug_str', LLVM was not built with LLVM_ENABLE_ZLIB or did not find zlib at build time
error: failed to decompress '.debug_info', LLVM was not built with LLVM_ENABLE_ZLIB or did not find zlib at build time
error: failed to decompress '.debug_aranges', LLVM was not built with LLVM_ENABLE_ZLIB or did not find zlib at build time
error: failed to decompress '.debug_loc', LLVM was not built with LLVM_ENABLE_ZLIB or did not find zlib at build time
error: failed to decompress '.debug_line', LLVM was not built with LLVM_ENABLE_ZLIB or did not find zlib at build time
error: failed to decompress '.debug_loclists', LLVM was not built with LLVM_ENABLE_ZLIB or did not find zlib at build time
error: failed to decompress '.debug_str_offsets', LLVM was not built with LLVM_ENABLE_ZLIB or did not find zlib at build time
error: failed to decompress '.debug_addr', LLVM was not built with LLVM_ENABLE_ZLIB or did not find zlib at build time
error: failed to decompress '.debug_line_str', LLVM was not built with LLVM_ENABLE_ZLIB or did not find zlib at build time
error: failed to decompress '.debug_rnglists', LLVM was not built with LLVM_ENABLE_ZLIB or did not find zlib at build time

that's why the line count on stdout was zero. We can add the zlib enabled llvm-dwarfdump in SuperFileCheck to fix it and temporarily disable this test until the update is flown.

Co-authored-by: Jan Kotas <jkotas@microsoft.com>

am11 · 2023-08-02T21:52:07Z

llvm with zlib was picked up today. NativeAOT legs are passing now.

jkotas · 2023-08-02T22:08:08Z

Thank you!

Add CompressSymbols opt-out option in NativeAOT

f254461

dotnet-issue-labeler bot added the area-NativeAOT-coreclr label Apr 22, 2023

ghost added the community-contribution Indicates that the PR has been added by a community member label Apr 22, 2023

am11 marked this pull request as ready for review April 22, 2023 13:07

am11 requested a review from MichalStrehovsky as a code owner April 22, 2023 13:07

build-analysis bot mentioned this pull request Apr 22, 2023

Tracking issue for CI build timeouts #76454

Closed

am11 requested a review from jkotas April 22, 2023 14:36

jkotas closed this Jul 25, 2023

jkotas reopened this Jul 25, 2023

jkotas approved these changes Jul 25, 2023

View reviewed changes

am11 added 2 commits July 26, 2023 07:07

Merge dotnet/main into feature/nativeaot/symbol..

2644ce7

Report what went wrong in dwarfdump test

0d19626

jkotas reviewed Jul 26, 2023

View reviewed changes

src/tests/nativeaot/SmokeTests/DwarfDump/Program.cs Outdated Show resolved Hide resolved

Update src/tests/nativeaot/SmokeTests/DwarfDump/Program.cs

f4e5b4f

Co-authored-by: Jan Kotas <jkotas@microsoft.com>

am11 mentioned this pull request Jul 26, 2023

Build with zlib on Unix dotnet/llvm-project#456

Merged

am11 closed this Aug 2, 2023

am11 reopened this Aug 2, 2023

jkotas merged commit 30db24b into dotnet:main Aug 2, 2023
114 of 116 checks passed

ghost locked as resolved and limited conversation to collaborators Sep 2, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `CompressSymbols` opt-out option in NativeAOT #85192

Add `CompressSymbols` opt-out option in NativeAOT #85192

am11 commented Apr 22, 2023

ghost commented Apr 22, 2023

am11 commented Apr 22, 2023

jkotas commented Apr 22, 2023

MichalPetryka commented Apr 22, 2023

am11 commented Apr 22, 2023 •

edited

Loading

am11 commented Jul 25, 2023

jkotas left a comment

am11 commented Jul 25, 2023

agocke commented Jul 25, 2023

MichalStrehovsky commented Jul 26, 2023

am11 commented Jul 26, 2023

am11 commented Aug 2, 2023

jkotas commented Aug 2, 2023

Add CompressSymbols opt-out option in NativeAOT #85192

Add CompressSymbols opt-out option in NativeAOT #85192

Conversation

am11 commented Apr 22, 2023

ghost commented Apr 22, 2023

am11 commented Apr 22, 2023

jkotas commented Apr 22, 2023

MichalPetryka commented Apr 22, 2023

am11 commented Apr 22, 2023 • edited Loading

am11 commented Jul 25, 2023

jkotas left a comment

Choose a reason for hiding this comment

am11 commented Jul 25, 2023

agocke commented Jul 25, 2023

MichalStrehovsky commented Jul 26, 2023

am11 commented Jul 26, 2023

am11 commented Aug 2, 2023

jkotas commented Aug 2, 2023

Add `CompressSymbols` opt-out option in NativeAOT #85192

Add `CompressSymbols` opt-out option in NativeAOT #85192

am11 commented Apr 22, 2023 •

edited

Loading