Portable Data Contract-based DAC in .NET 9+ #99298
Comments
|
Tagging subscribers to this area: @tommcdon Issue DetailsSummaryThe DAC is a .NET runtime component responsible for helping debuggers and other diagnostic tools to make sense of Additionally, the current DAC is not version-resilient. When a new verson of the .NET runtime is release, it ships with a DAC that can access that versions' memory and nothing else. This means that debuggers need to be able to locate, load and run a DAC corresponding to each runtime version. While this might be possible for official releases, it may create complications for source-built .NET runtimes, for unsupported/community-supported platforms etc. Portable Data Contract-based DACAn alternate approach is for the .NET runtime to include a data stream that encodes information about itself: the size and endianness of machine words, the size and offsets of important fields in interesting runtime data structures, the locations of globals that are relevant for diagnostic tools. With access to a data stream, the DAC can be a separate implementation that can abstract over the details of a particular version of the runtime and implement an abstract data contract based diagnostic tooling. (This is abbreviated cDAC) .NET 9 PlanIn .NET 9 we will begin to move the DAC toward a data-contract based approach by enabling the existing DAC to delegate some operations to the cDAC. Our initial focus will be the SOS tool used by the .NET LLDB plugin and by Windbg. Our initial milestone will be the implementation of the A goal for the cDAC implementation is to maintain backward- and platform- compatability with each release: the each future release of the cDAC will be able to access each .NET runtime version >= .NET 9. We do not intend to support .NET <= 8 or .NET Framework using the cDAC. The focus for .NET 9 is to host the DAC on 64-bit desktop platforms - Linux (glibc), MacOS and Windows - but to be able to access a debuggee or crash dump from 32-bit and 64-bit processes on all supported (and ideally community-supported) platforms (including musl-based Linux, arm32, win-x86, etc).
Future plansThe initial cDAC plan will not reduce the complexity of the current design. However once the existing DAC delegates all commands to the cDAC, we may simplify the packaging and distribution of the cDAC - allowing diagnostics tools to obtain a single library that works with all versions of CoreCLR >= .NET 9, as well as other .NET runtimes including NativeAOT and Mono. Special attention needs to be paid to the
|
steveisok
added
the
Epic
Contributes to #100162 which is part of #99298 Follow-up to #99936 that removes "type layout" and "global value" contracts and instead replaces them with a "data descriptor" blob. Conceptually a particular target runtime provides a pair of a logical data descriptor together with a set of algorithmic contract versions. The logical data descriptor is just a single model that defines all the globals and type layouts relevant to the set of algorithmic contract versions. A logical data descriptor is realized by merging two physical data descriptors in a proscribed order. The physical data descriptors provide some subset of the type layouts or global values. The physical data descriptors come in two flavors: - baseline descriptors that are checked into the dotnet/runtime repo and have well -known names - in-proc descriptors that get embedded into a target runtime. Each in-proc descriptor may refer to a baseline and represents a delta applied on top of the baseline. The data contract model works on top of a flattened logical data descriptor. Co-authored-by: Aaron Robinson <arobins@microsoft.com> Co-authored-by: Jan Kotas <jkotas@microsoft.com> Co-authored-by: Noah Falk <noahfalk@users.noreply.github.com>
Building on #100253 , describe an in-memory representation of the toplevel contract descriptor, comprisied of: * some target architecture properties * a data descriptor * a collection of compatible contracts Contributes to #99298 Fixes #99299 --- * [cdac] Physical contract descriptor spec * Add "contracts" to the data descriptor * one runtime per module if there are multiple hosted runtimes, diagnostic tooling should look in each loaded module to discover the contract descriptor * Apply suggestions from code review * Review feedback - put the aux data and descriptor sizes closer to the pointers - Don't include trailing nul `descriptor_size`. Clarify it is counting bytes and that `descriptor` is in UTF-8 - Simplify `DotNetRuntimeContractDescriptor` naming discussion --------- Co-authored-by: Elinor Fung <elfung@microsoft.com>
Contributes to dotnet#100162 which is part of dotnet#99298 Follow-up to dotnet#99936 that removes "type layout" and "global value" contracts and instead replaces them with a "data descriptor" blob. Conceptually a particular target runtime provides a pair of a logical data descriptor together with a set of algorithmic contract versions. The logical data descriptor is just a single model that defines all the globals and type layouts relevant to the set of algorithmic contract versions. A logical data descriptor is realized by merging two physical data descriptors in a proscribed order. The physical data descriptors provide some subset of the type layouts or global values. The physical data descriptors come in two flavors: - baseline descriptors that are checked into the dotnet/runtime repo and have well -known names - in-proc descriptors that get embedded into a target runtime. Each in-proc descriptor may refer to a baseline and represents a delta applied on top of the baseline. The data contract model works on top of a flattened logical data descriptor. Co-authored-by: Aaron Robinson <arobins@microsoft.com> Co-authored-by: Jan Kotas <jkotas@microsoft.com> Co-authored-by: Noah Falk <noahfalk@users.noreply.github.com>
Building on dotnet#100253 , describe an in-memory representation of the toplevel contract descriptor, comprisied of: * some target architecture properties * a data descriptor * a collection of compatible contracts Contributes to dotnet#99298 Fixes dotnet#99299 --- * [cdac] Physical contract descriptor spec * Add "contracts" to the data descriptor * one runtime per module if there are multiple hosted runtimes, diagnostic tooling should look in each loaded module to discover the contract descriptor * Apply suggestions from code review * Review feedback - put the aux data and descriptor sizes closer to the pointers - Don't include trailing nul `descriptor_size`. Clarify it is counting bytes and that `descriptor` is in UTF-8 - Simplify `DotNetRuntimeContractDescriptor` naming discussion --------- Co-authored-by: Elinor Fung <elfung@microsoft.com>
Contributes to dotnet#100162 which is part of dotnet#99298 Follow-up to dotnet#99936 that removes "type layout" and "global value" contracts and instead replaces them with a "data descriptor" blob. Conceptually a particular target runtime provides a pair of a logical data descriptor together with a set of algorithmic contract versions. The logical data descriptor is just a single model that defines all the globals and type layouts relevant to the set of algorithmic contract versions. A logical data descriptor is realized by merging two physical data descriptors in a proscribed order. The physical data descriptors provide some subset of the type layouts or global values. The physical data descriptors come in two flavors: - baseline descriptors that are checked into the dotnet/runtime repo and have well -known names - in-proc descriptors that get embedded into a target runtime. Each in-proc descriptor may refer to a baseline and represents a delta applied on top of the baseline. The data contract model works on top of a flattened logical data descriptor. Co-authored-by: Aaron Robinson <arobins@microsoft.com> Co-authored-by: Jan Kotas <jkotas@microsoft.com> Co-authored-by: Noah Falk <noahfalk@users.noreply.github.com>
Building on dotnet#100253 , describe an in-memory representation of the toplevel contract descriptor, comprisied of: * some target architecture properties * a data descriptor * a collection of compatible contracts Contributes to dotnet#99298 Fixes dotnet#99299 --- * [cdac] Physical contract descriptor spec * Add "contracts" to the data descriptor * one runtime per module if there are multiple hosted runtimes, diagnostic tooling should look in each loaded module to discover the contract descriptor * Apply suggestions from code review * Review feedback - put the aux data and descriptor sizes closer to the pointers - Don't include trailing nul `descriptor_size`. Clarify it is counting bytes and that `descriptor` is in UTF-8 - Simplify `DotNetRuntimeContractDescriptor` naming discussion --------- Co-authored-by: Elinor Fung <elfung@microsoft.com>
lambdageek
changed the title
lambdageek
added
the
tracking
Summary
The DAC is a .NET runtime component responsible for helping debuggers and other diagnostic tools to make sense of
the memory of a .NET runtime process. In the current design, each instance of the .NET runtime is tightly coupled to a DAC:
a runtime running on a a 32-bit little endian architecture can only be accessed by a 32-bit LE DAC, 64-bit with a 64-bit DAC,
etc. Although the current runtime data structures are carefully designed so that a DAC hosted on Windows can make sense
of a Linux or Mac .NET runtime, this is fragile and hard to maintain. This is because the current DAC implementation is actually a special build of the .NET runtime that uses C++ smart pointers to hide the fact
that it is not accessing the memory of the host process, but rather the memory of a remote debuggee .NET process.
Additionally, the current DAC is not version-resilient. When a new verson of the .NET runtime is released, it ships with a DAC that can access that versions' memory and nothing else. This means that debuggers need to be able to locate, load and run a DAC corresponding to each runtime version. While this might be possible for official releases, it may create complications for source-built .NET runtimes, for unsupported/community-supported platforms etc.
Portable Data Contract-based DAC
An alternate approach is for the .NET runtime to include a data stream that encodes information about itself: the size and endianness of machine words, the size and offsets of important fields in interesting runtime data structures, the locations of globals that are relevant for diagnostic tools.
With access to a data stream, the DAC can be a separate implementation that can abstract over the details of a particular version of the runtime and implement an abstract data contract based diagnostic tooling. (This is abbreviated cDAC)
.NET 10 Plan
In .NET 10 we will begin to move the DAC toward a data-contract based approach by enabling the existing DAC to delegate some operations to the cDAC. Our initial focus will be the SOS tool used by the .NET LLDB plugin and by Windbg. Our initial milestone will be the implementation of the
!PrintExceptionSOS command. Followed by the DAC interfaces used by CLRMAA goal for the cDAC implementation is to maintain backward- and platform- compatability with each release: the each future release of the cDAC will be able to access each .NET runtime version >= .NET 9. We do not intend to support .NET <= 8 or .NET Framework using the cDAC.
The focus for .NET 9 is to host the DAC on 64-bit desktop platforms - Linux (glibc), MacOS and Windows - but to be able to access a debuggee or crash dump from 32-bit and 64-bit processes on all supported (and ideally community-supported) platforms (including musl-based Linux, arm32, win-x86, etc).
First milestone
The first milestone is a working
!PrintExceptioncommand based on the cDAC inwindbg/SOS!PrintException#99302Post-.NET 10 work
The overall goal is to implement a subset of the
ISOSDacInterfaceNNIDL interfaces in sospriv.idl via the cDACDOTNET_ENABLE_CDACenvironment variable checkhttps://github.com/mikem8361/diagnostics/blob/clrma/documentation/clrma.md#dac-interfaces-used-by-clrma
Other closed issues
.NET 9 backports
We maintain a branch
feature/9.0-cdac-backportsthat has selected data descriptor and contract changes necessary for the cdacreader to interrogate a net9.0 runtime. The backports to this branch are tracked on #99302.Future work
The initial cDAC plan will not reduce the complexity of the current design. However once the existing DAC delegates all commands to the cDAC, we may simplify the packaging and distribution of the cDAC - allowing diagnostics tools to obtain a single library that works with all versions of CoreCLR >= .NET 9, as well as other .NET runtimes including NativeAOT and Mono.
Special attention needs to be paid to the
crashdumptool. This ships with CoreCLR and NativeAOT and runs on the same host as the .NET runtime. It uses a subset of the DAC in order to save a copycrashdumpThe text was updated successfully, but these errors were encountered: