Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix running tests against our DNNE'd NativeExports project to work in NativeAOT #84578

Merged
merged 6 commits into from
Apr 18, 2023

Conversation

jkoritzinsky
Copy link
Member

In NativeAOT, we'll forego using the DNNE-generated shim and instead will directly call the [UnmanagedCallersOnly] exports from the [LibraryImport]/[DllImport] methods in our tests. We'll use the functionality that was added for the iOS team to export the [UnmanagedCallersOnly] symbols from an assembly that is not the entry-point to include the unmanaged names for the symbols in NativeExports, and we'll use the Direct P/Invoke functionality to redirect the calls to the DNNE-based shim to call the symbols we now have exported from the same image, allowing the native linker to put the pieces together.

Contributes to #84451 by enabling libraries tests with DNNE components to run on our NativeAOT test runs.

@jkoritzinsky jkoritzinsky added area-System.Runtime.InteropServices source-generator Indicates an issue with a source generator feature labels Apr 10, 2023
@ghost ghost assigned jkoritzinsky Apr 10, 2023
@ghost
Copy link

ghost commented Apr 10, 2023

Tagging subscribers to this area: @dotnet/interop-contrib
See info in area-owners.md if you want to be subscribed.

Issue Details

In NativeAOT, we'll forego using the DNNE-generated shim and instead will directly call the [UnmanagedCallersOnly] exports from the [LibraryImport]/[DllImport] methods in our tests. We'll use the functionality that was added for the iOS team to export the [UnmanagedCallersOnly] symbols from an assembly that is not the entry-point to include the unmanaged names for the symbols in NativeExports, and we'll use the Direct P/Invoke functionality to redirect the calls to the DNNE-based shim to call the symbols we now have exported from the same image, allowing the native linker to put the pieces together.

Contributes to #84451 by enabling libraries tests with DNNE components to run on our NativeAOT test runs.

Author: jkoritzinsky
Assignees: -
Labels:

area-System.Runtime.InteropServices, source-generator

Milestone: -

@MichalStrehovsky
Copy link
Member

Thank you for looking into this! Once #84552 merges, could you trigger runtime-extra-platforms legs on this to make sure it works everywhere we use this?

@MichalStrehovsky
Copy link
Member

/azp run runtime-extra-platforms

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@jkoritzinsky
Copy link
Member Author

/azp run runtime-extra-platforms

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@jkoritzinsky
Copy link
Member Author

Looks like the ComInterfaceGenerator.Tests test suite hits an AV when AOTd. I'll take a look.

…e zeroed (CoreCLR does this). The interop source generators depend on this behavior for lazy initialization.
@jkotas
Copy link
Member

jkotas commented Apr 15, 2023

The memory allocated by AllocateTypeAssociatedMemory is expected to be zeroed

We should augment the test

and mention it in the docs.

…e/CompilerServices/RuntimeHelpers.NativeAot.cs

Co-authored-by: Jan Kotas <jkotas@microsoft.com>
@jkoritzinsky
Copy link
Member Author

/azp run runtime-extra-platforms

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@jkoritzinsky
Copy link
Member Author

Looks like there's still some failures on Mac. I'll investigate next time I have a Mac to test with (probably later this week).

@MichalStrehovsky
Copy link
Member

Looks like there's still some failures on Mac. I'll investigate next time I have a Mac to test with (probably later this week).

They also fail on Linux. We could make these only run on Windows for now and cut a bug for non-Windows. Looks like there's at least two issues: https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-84578-merge-0c2106cb7a2b4c4eb9/ComInterfaceGenerator.Tests/1/console.1e0f119a.log?helixlogtype=result

[FAIL] ComInterfaceGenerator.Tests.GeneratedComClassTests.ComInstanceProvidesInterfaceForIndirectlyImplementedComInterface
System.PlatformNotSupportedException : Operation is not supported on this platform.
   at System.Runtime.InteropServices.ComWrappers.GetOrCreateComInterfaceForObject(Object instance, CreateComInterfaceFlags flags) + 0x28
   at ComInterfaceGenerator.Tests.GeneratedComClassTests.ComInstanceProvidesInterfaceForIndirectlyImplementedComInterface() + 0x50
   at ComInterfaceGenerator!<BaseAddress>+0x74e190

looks like we PNSE ComWrappers outside Windows. We already ran into that one: #76005

The sigsegv has a crashdump that can be downaloded with runfo get-helix-payload -j 0c2106cb-7a2b-4c4e-b99f-0d5f63a07d58 -w ComInterfaceGenerator.Tests -o c:\hel\:

02 0000ff4e`61383720 0000ab6b`02bdd7c8     ComInterfaceGenerator!SystemNative_Abort+0x8 [/__w/1/s/src/native/libs/System.Native/pal_threading.c @ 286] 
03 0000ff4e`61383860 0000ab6b`02bdd6b8     ComInterfaceGenerator!S_P_CoreLib_System_RuntimeExceptionHelpers::FailFast+0xb8 [/_/src/coreclr/nativeaot/System.Private.CoreLib/src/System/RuntimeExceptionHelpers.cs @ 281] 
04 0000ff4e`613838f0 0000ab6b`02c5c260     ComInterfaceGenerator!S_P_CoreLib_System_RuntimeExceptionHelpers::RuntimeFailFast+0xf8 [/_/src/coreclr/nativeaot/System.Private.CoreLib/src/System/RuntimeExceptionHelpers.cs @ 226] 
05 0000ff4e`61383930 0000ab6b`02c5c808     ComInterfaceGenerator!S_P_CoreLib_System_Runtime_EH::UnhandledExceptionFailFastViaClasslib+0xa0 [/_/src/coreclr/nativeaot/Runtime.Base/src/System/Runtime/ExceptionHandling.cs @ 205] 
06 0000ff4e`61383d10 0000ab6b`02c5c678     ComInterfaceGenerator!S_P_CoreLib_System_Runtime_EH::DispatchEx+0x128 [/_/src/coreclr/nativeaot/Runtime.Base/src/System/Runtime/ExceptionHandling.cs @ 696] 
07 0000ff4e`61383da0 0000ab6b`02aad638     ComInterfaceGenerator!S_P_CoreLib_System_Runtime_EH::RhThrowEx+0x48 [/_/src/coreclr/nativeaot/Runtime.Base/src/System/Runtime/ExceptionHandling.cs @ 573] 
08 0000ff4e`61383dc0 0000ab6b`02c64fb8     ComInterfaceGenerator!RhpThrowEx+0xbc [/__w/1/s/src/coreclr/nativeaot/Runtime/arm64/ExceptionHandling.S @ 338] 
09 0000ff4e`61384120 0000ab6b`02bc0b80     ComInterfaceGenerator!S_P_CoreLib_System_Runtime_InteropServices_ComWrappers::GetOrCreateComInterfaceForObject+0x28 [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/InteropServices/ComWrappers.PlatformNotSupported.cs @ 32] 
0a 0000ff4e`61384140 0000ab6b`02b50178     ComInterfaceGenerator!Microsoft_Interop_Tests_NativeExports_NativeExports_ComInterfaceGenerator_ArrayMarshalling::CreateComObject+0x50 [/_/src/libraries/System.Runtime.InteropServices/tests/TestAssets/NativeExports/ComInterfaceGenerator/ArrayMarshalling.cs @ 27] 
0b 0000ff4e`61384170 0000ab6b`02dce190     ComInterfaceGenerator!ComInterfaceGenerator_Tests_ComInterfaceGenerator_Tests_IGetIntArrayTests::CallRcwFromGeneratedComInterface+0x28 [/_/src/libraries/System.Runtime.InteropServices/tests/ComInterfaceGenerator.Tests/IGetIntArrayTests.cs @ 25] 
0c 0000ff4e`61384200 0000ab6b`02c665f8     ComInterfaceGenerator!Internal_CompilerGenerated__Module___<DynamicInvoke>None<S_P_CoreLib_System_Void>+0x20 [/_/src/libraries/System.Linq/src/System/Linq/Join.cs @ 64] 

@jkoritzinsky
Copy link
Member Author

Looks like the sigsev might also be the PNSE, but just on the other side of an UnmanagedCallersOnly call. I'll exclude the project on non-Windows NativeAOT for now and work on enabling ComWrappers on NativeAOT there next.

@jkoritzinsky
Copy link
Member Author

/azp run runtime-extra-platforms

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@jkoritzinsky
Copy link
Member Author

The tests enabled by this PR all pass, so I'm going to merge.

@jkoritzinsky jkoritzinsky merged commit ce2838a into dotnet:main Apr 18, 2023
@jkoritzinsky jkoritzinsky deleted the nativeaot-nativeeexports branch April 18, 2023 23:14
@ghost ghost locked as resolved and limited conversation to collaborators May 19, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-System.Runtime.InteropServices source-generator Indicates an issue with a source generator feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants