Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ObjWriter in C# #95876

Merged
merged 144 commits into from
Jan 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
144 commits
Select commit Hold shift + click to select a range
6f62905
Move ObjectWriter and ObjectWritingOptions into its own namespace
filipnavara Oct 18, 2022
b3a2198
Add external sources for Melanzana.MachO and LibObjectFile
filipnavara Oct 18, 2022
55094c9
Disable versioning NuGets in external projects (MinVer, Nerdbank.GitV…
filipnavara Oct 18, 2022
ee4bf61
Add first draft of MachO and ELF object writers
filipnavara Oct 10, 2022
a9ead97
Generate relocatable DWARF info in ELF, fix X64 register mapping
filipnavara Oct 19, 2022
1ba7e46
Fix writing base address with relocations
filipnavara Oct 19, 2022
f160dc8
Emit information about static variables, minor DWARF fixes
filipnavara Oct 19, 2022
490495d
Fix generated section layout in symbol table for Mach-O after EmitSym…
filipnavara Oct 19, 2022
685fcb8
Add support for ARM64 ELF
filipnavara Oct 19, 2022
ff0d382
Add ObjectWriter class and derive UnixObjectWriter from it
filipnavara Oct 20, 2022
fc31d42
Add initial take on CoffObjectWriter
filipnavara Oct 21, 2022
bd13d55
Emit control flow guard information
filipnavara Oct 21, 2022
d19459a
Add win-arm64 support
filipnavara Oct 23, 2022
efd0c57
Misc cleanups of the COFF object writer
filipnavara Oct 24, 2022
14a4755
Remove GetSection
filipnavara Oct 24, 2022
ef5ffb4
Add COFF COMDAT support
filipnavara Oct 25, 2022
19501ca
Add support for writing extended COFF object files (> 65279 sections)
filipnavara Oct 25, 2022
88e1002
Wrap some logic in SectionWriter struct; clean up code
filipnavara Oct 25, 2022
6d2d4bd
Emit GNU-stack note in ELF
filipnavara Oct 25, 2022
cdce467
Propagate DWARF5 option
filipnavara Oct 25, 2022
f900b0d
Initial take on CodeView type section builder
filipnavara Oct 26, 2022
1f2473b
Move SectionWriter out of ObjectWriter so it can be used by other bui…
filipnavara Oct 27, 2022
00d86ff
Switch DwarfEhFrame to use UnixObjectWriter
filipnavara Oct 27, 2022
b770f78
WIP: CodeView debugging information
filipnavara Oct 27, 2022
65d59f0
WIP: CodeView line mapping
filipnavara Oct 28, 2022
bc8b8d5
WIP: CodeView restructuring
filipnavara Oct 29, 2022
22ae5a6
CodeView cleanup
filipnavara Oct 31, 2022
cd56a6c
CodeView: Handle continuation records in lists
filipnavara Nov 1, 2022
81210d1
Add comments and asserts
filipnavara Nov 1, 2022
249407c
Emit CodeView symbols correctly when COMDAT is used
filipnavara Nov 1, 2022
904bec4
Minor cleanup in CodeView, add ARM64 register mapping
filipnavara Nov 2, 2022
d72a472
Optimize memory usage when writing section content
filipnavara Nov 2, 2022
0aca151
CodeView: Fix emitting size of big structures
filipnavara Nov 3, 2022
f5a9081
Fix too aggressive zeroing of buffer
filipnavara Nov 3, 2022
87afd74
Make sure not to dereference null returned from _debugInfo.GetLocalVa…
filipnavara Nov 3, 2022
f1ad210
HACK: Do not emit debug information where LLVM ObjWriter would not.
filipnavara Nov 3, 2022
fbc6f3e
Avoid some unnecessary name mangling calls
filipnavara Nov 3, 2022
2ca35a6
Cache mangled names
filipnavara Nov 3, 2022
c5d0547
Fix ArgumentOutOfRangeException when writing DWARF debug info
filipnavara Nov 4, 2022
670a14a
Fix build after rebase
filipnavara Sep 10, 2023
bdc441f
Add support for SectionType.Uninitialized
filipnavara Sep 10, 2023
f7aa827
Add support for iOS-like platforms
filipnavara Sep 10, 2023
2bef42c
Invoke object dumper from shared ObjectWriter
filipnavara Sep 10, 2023
3579987
macOS/iOS: Emit simple compact unwinding information
filipnavara Sep 10, 2023
4567163
Add support for new TLS relocations in ELF
filipnavara Sep 11, 2023
8a6774c
Generate debug info even if managed debug info not present
filipnavara Sep 11, 2023
a9cc9b4
Fix counting the DwarfLineProgramTable layout (backport of https://gi…
filipnavara Sep 11, 2023
7eb39c1
Remove incorrect assert in MachObjectWriter (text section starts at v…
filipnavara Sep 11, 2023
a96157d
Temporarily disable optimization that results in incorrect LSDA point…
filipnavara Sep 11, 2023
0e855a1
Fix emitting unwind tables in Mach-O
filipnavara Sep 11, 2023
aee297e
Emit the MH_SUBSECTIONS_VIA_SYMBOLS header flag for Mach-O
filipnavara Sep 21, 2023
f728fb5
Fix seeking in ObjectWriterStream
filipnavara Sep 21, 2023
d7ef771
Avoid excessive seeks by implementing SliceStream.CopyTo
filipnavara Sep 21, 2023
4b84616
Use ObjectWriterStream in ElfObjectWriter
filipnavara Sep 21, 2023
a8a12b9
Use ObjectWriterStream for Mach-O
filipnavara Sep 21, 2023
18a39ab
Fix setting the UNWIND_HAS_LSDA flag in compact unwind table
filipnavara Sep 21, 2023
64dc26c
COFF:
filipnavara Sep 26, 2023
66d35bd
COFF: Fix off-by-one error when writing more than 0xffff relocations …
filipnavara Sep 26, 2023
e333caa
Make ObjectWriterStream the only supported stream and fix padding emi…
filipnavara Sep 26, 2023
5645f9a
Fix scope overlapping in NAOT DWARF info (backport of https://github.…
filipnavara Sep 27, 2023
4025c09
ELF: Use the address of __managedcode in DWARF info (backport of http…
filipnavara Sep 27, 2023
6e0f1b6
ELF: Fix emitting global symbols
filipnavara Sep 27, 2023
9dc6b63
ELF: Fix TLS support on linux-x64 Release builds
filipnavara Sep 27, 2023
1d76838
Return sorted set from GetUndefinedSymbols to avoid extra sorting
filipnavara Sep 26, 2023
a1922c5
COFF: Fix generating COMDAT section references in big-obj files
filipnavara Sep 27, 2023
3e9fcc7
Generate Mach-O for iOS-like platforms, not ELF
filipnavara Sep 27, 2023
9e95e9b
Fix EH frames on linux-arm64
filipnavara Sep 28, 2023
9a50db5
COFF: Support addend for IMAGE_REL_BASED_ARM64_PAGEOFFSET_12A relocation
filipnavara Sep 30, 2023
6c28f5a
COFF: Emit alignment for .pdata section
filipnavara Sep 30, 2023
b357425
Fix padding emitted in ObjectWriterStream.AppendPadding
filipnavara Sep 30, 2023
febce5a
Fix ObjectWriterStream.Position setter (again)
filipnavara Sep 30, 2023
93e84cb
COFF: Emit .pdata/.xdata as associative COMDAT sections/symbols
filipnavara Sep 30, 2023
0a61577
ELF/LibObjectFile experiment: Add support for building optimized stri…
filipnavara Oct 2, 2023
bfb6db4
LibObjectFile: Add support for >= 0xff00 sections
filipnavara Oct 4, 2023
6f0d770
ELF: Add basic COMDAT support, don't emit empty relocation sections
filipnavara Oct 4, 2023
62c478e
ELF: Emit correct addresses and relocations in debug info, references…
filipnavara Oct 5, 2023
14f5d17
Add missing StringComparer.Ordinal to dictionaries
filipnavara Oct 5, 2023
ec7bc5e
Remove incorrect assert
filipnavara Oct 5, 2023
307ca09
WIP: Rewritten DWARF debug info emitter
filipnavara Oct 18, 2023
e8a19ff
Fix state change in ObjectWriterStream triggering an Assert
filipnavara Oct 20, 2023
fedcab7
WIP: Mach-O/DWARF
filipnavara Oct 20, 2023
5cda00a
DWARF Mach-O relocations done right
filipnavara Oct 20, 2023
5d54858
Do not output debug information for static variables that were trimmed
filipnavara Oct 22, 2023
cd4a543
WIP: Cleanup
filipnavara Oct 22, 2023
cbd7f83
Rework how symbol tables are generated, and how DWARF relocations are…
filipnavara Oct 23, 2023
8530c75
Revert accidental change
filipnavara Oct 23, 2023
0575b0f
Update LibObjectFile to version 0.5.0.
filipnavara Oct 23, 2023
157dbdc
Fix ELF section symbols and DW_AT_low_pc in DWARF debug info
filipnavara Oct 23, 2023
d7f4e2d
Disable MinVer package in LibObjectFile
filipnavara Oct 23, 2023
941e0a3
ELF: Generate symbol table section index if there's more than 0xff00 …
filipnavara Oct 23, 2023
a00cd12
Revert "Do not output debug information for static variables that wer…
filipnavara Oct 23, 2023
ea9360f
Fix typo
filipnavara Oct 23, 2023
81ac747
Fix null dereference in CoffObjectWriter.CreateSection
filipnavara Oct 23, 2023
f1a1bfa
COFF: Add X86 support (tested on SmolSharp.HelloWorld)
filipnavara Oct 24, 2023
07dd8a6
Fix COMDAT symbol values
filipnavara Nov 12, 2023
d89308d
Do not generate broken debug info for non-methods
filipnavara Nov 17, 2023
43d8e3d
Add generic StringTableBuilder and use it in COFF
filipnavara Nov 25, 2023
d92d31a
Fix couple of bugs in last commit:
filipnavara Nov 26, 2023
6ff8ae9
COFF: Add support for '//BBBBBB' section names.
filipnavara Nov 26, 2023
018f727
Add simple ELF writer
filipnavara Dec 6, 2023
e7cda50
Port from main: Fix generating debug info for static fields
filipnavara Dec 6, 2023
a1a9050
Add (untested) x86 support to ElfObjectWriter
filipnavara Dec 7, 2023
75d34e5
Cleanup
filipnavara Dec 7, 2023
aa6e94e
Move Dwarf/CodeView files into their respective directories
filipnavara Dec 8, 2023
0048be7
Make bulk of the ObjectWriter classes internal, improve logging
filipnavara Dec 8, 2023
5c5d6a1
Use IBinaryInteger for WriteLittleEndian helper method
filipnavara Dec 8, 2023
288eedd
Add simple Mach-O object file writer that doesn't use Melanzana.MachO
filipnavara Dec 9, 2023
f17b308
Rename LlvmObjectWriter to LegacyObjectWriter to prevent conflict wit…
filipnavara Dec 9, 2023
004b5c5
Mach-O: Swap the order of symbol table and string table to make strip…
filipnavara Dec 9, 2023
e5d6e72
Fix the CPU type/subtype
filipnavara Dec 10, 2023
d38b91a
Fix Release build
filipnavara Dec 10, 2023
57133a9
Eliminate relocations for known symbols within the same section
filipnavara Dec 11, 2023
93f0650
Fix IMAGE_REL_BASED_ARM64_BRANCH26
filipnavara Dec 11, 2023
a4312a1
Attempt to fix CI errors
filipnavara Dec 11, 2023
130cfe1
Fix incorrect addends in COFF
filipnavara Dec 11, 2023
53c9f00
Fix ELF rela section flags; omit frame symbols for ELF
filipnavara Dec 11, 2023
96635da
Split the writing and reading part of section data
filipnavara Dec 11, 2023
17c3001
Cleanup
filipnavara Dec 11, 2023
6dde29b
Pattern matching consistency
filipnavara Dec 11, 2023
bee3113
Comments, more cleanup
filipnavara Dec 11, 2023
2f7bda3
Refactor ULEB128 writing in CfiCodeToInstructions
filipnavara Dec 13, 2023
df31fc6
Make JamCrc32 table a read-only span with collection initializer
filipnavara Dec 14, 2023
dd168d1
IsOSXLike -> IsApplePlatform
filipnavara Dec 17, 2023
cdb7b63
Follow up on 'using static' for relocations and *Native
filipnavara Dec 17, 2023
deb4df2
Use Relocation.ReadValue/WriteValue instead of manual implementation …
filipnavara Dec 17, 2023
9c24e71
Fix handling of IMAGE_REL_BASED_RELPTR32 in the last commit
filipnavara Dec 17, 2023
2c92fd5
Add missing enum values in Relocation.WriteValue
filipnavara Dec 18, 2023
4cce374
Apply readonly on class members where appropriate
filipnavara Jan 4, 2024
1c10dce
Zero-initialize stackalloc
filipnavara Jan 4, 2024
af416b8
Add comments; few missed readonly members
filipnavara Jan 4, 2024
8e5ed26
Use 'static readonly' for reminder of ObjectNodeSection members
filipnavara Jan 4, 2024
2f3df76
Pass target pointer size to CodeViewTypesBuilder instead of target ar…
filipnavara Jan 4, 2024
e970927
Set attribute values in CodeViewTypesBuilder
filipnavara Jan 4, 2024
666d654
Use upper case hex constants for consistency
filipnavara Jan 4, 2024
a64485e
Move AbbreviationCode out of DwarfAbbrev
filipnavara Jan 5, 2024
c07a3d7
More readonly treatment
filipnavara Jan 5, 2024
a1ad976
Update comment
filipnavara Jan 5, 2024
eb143fd
Consistently throw exception for unsupported architectures
filipnavara Jan 5, 2024
a182ea7
Use the WriteLittleEndian helper method where appropriate
filipnavara Jan 5, 2024
939fa83
Refactor how DwarfFde/DwarfCie are constructed and make them readonly…
filipnavara Jan 5, 2024
47e8094
Add WritePadding method and use it instead of Write(stackalloc byte[.…
filipnavara Jan 5, 2024
96a9e7c
Fix DWARF EH frame building
filipnavara Jan 5, 2024
1fa5074
Revert accidental change
filipnavara Jan 5, 2024
e716e3d
Add code paths for ARM32 to ELF object writer
filipnavara Jan 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ public enum SectionType
Writeable,
Executable,
Uninitialized,
Debug,
}

/// <summary>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ public enum RelocType
// This is a special NGEN-specific relocation type
// for relative pointer (used to make NGen relocation
// section smaller)
IMAGE_REL_SECTION = 0x79, // 16 bit section index containing target

IMAGE_REL_BASED_ARM64_PAGEBASE_REL21 = 0x81, // ADRP
IMAGE_REL_BASED_ARM64_PAGEOFFSET_12A = 0x82, // ADD/ADDS (immediate) with zero shift, for page offset
Expand Down Expand Up @@ -423,9 +424,13 @@ public static unsafe void WriteValue(RelocType relocType, void* location, long v
switch (relocType)
{
case RelocType.IMAGE_REL_BASED_ABSOLUTE:
case RelocType.IMAGE_REL_BASED_ADDR32NB:
case RelocType.IMAGE_REL_BASED_HIGHLOW:
case RelocType.IMAGE_REL_BASED_REL32:
case RelocType.IMAGE_REL_BASED_ADDR32NB:
case RelocType.IMAGE_REL_BASED_RELPTR32:
case RelocType.IMAGE_REL_SECREL:
case RelocType.IMAGE_REL_TLSGD:
case RelocType.IMAGE_REL_TPOFF:
case RelocType.IMAGE_REL_SYMBOL_SIZE:
case RelocType.IMAGE_REL_FILE_ABSOLUTE:
*(int*)location = (int)value;
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

using System;
using System.Buffers;
using System.Buffers.Binary;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Text;

using static ILCompiler.ObjectWriter.CodeViewNative;

namespace ILCompiler.ObjectWriter
{
internal sealed class CodeViewFileTableBuilder
{
private readonly MemoryStream _stringTableWriter = new();
private readonly MemoryStream _fileTableWriter = new();
private readonly Dictionary<string, uint> _fileNameToIndex = new();

public CodeViewFileTableBuilder()
{
// Insert empty entry at the beginning of string table
_stringTableWriter.Write(stackalloc byte[2] { 0, 0 });
}

public uint GetFileIndex(string fileName)
{
if (fileName == "")
{
// Match the placeholder value from LLVM. We need to use a non-empty
// string to ensure that the null terminator of the UTF-8 representation
// is not treated as the terminator of the whole file name table.
fileName = "<stdin>";
filipnavara marked this conversation as resolved.
Show resolved Hide resolved
}

if (_fileNameToIndex.TryGetValue(fileName, out uint fileIndex))
{
return fileIndex;
}
else
{
uint stringTableIndex = (uint)_stringTableWriter.Position;
_stringTableWriter.Write(Encoding.UTF8.GetBytes(fileName));
_stringTableWriter.WriteByte(0);

uint fileTableIndex = (uint)_fileTableWriter.Position;
Span<byte> fileTableEntry = stackalloc byte[sizeof(uint) + sizeof(uint)];
BinaryPrimitives.WriteUInt32LittleEndian(fileTableEntry, stringTableIndex);
BinaryPrimitives.WriteUInt32LittleEndian(fileTableEntry.Slice(4), 0);
_fileTableWriter.Write(fileTableEntry);

_fileNameToIndex.Add(fileName, fileTableIndex);

return fileTableIndex;
}
}

public void Write(SectionWriter sectionWriter)
{
sectionWriter.WriteLittleEndian<uint>((uint)DebugSymbolsSubsectionType.FileChecksums);
sectionWriter.WriteLittleEndian<uint>((uint)_fileTableWriter.Length);
sectionWriter.EmitData(_fileTableWriter.GetBuffer().AsMemory(0, (int)_fileTableWriter.Length));
sectionWriter.WriteLittleEndian<uint>((uint)DebugSymbolsSubsectionType.StringTable);
sectionWriter.WriteLittleEndian<uint>((uint)_stringTableWriter.Length);
sectionWriter.EmitData(_stringTableWriter.GetBuffer().AsMemory(0, (int)_stringTableWriter.Length));
}
}
}

Large diffs are not rendered by default.

Loading
Loading