[java-interop] Windows build system support (#816)

Context: 5c756b1
Context: 08ff4db

Context: https://discord.com/channels/732297728826277939/732297952676020316/819978878948868166
Context: microsoft/vstest#2571
Context: dotnet/msbuild#539 (comment)
Context: #816 (comment)

With the `cmake`-based build system knowledge behind 5c756b1 in
mind, update `src/java-interop` to *also* use a `cmake`-based build
system, then add support so it can build on Windows.

On Windows, this builds both e.g.
a 32-bit `bin\Debug\win-x86\java-interop.dll`
*and* 64-bit `bin\Debug\win-x64\java-interop.dll` libraries.

This allows the `Java.Interop-Tests.dll` unit tests to *also* be run
on Windows, under .NET Core [^0].

Note that this requires updating `java_interop_lib_load()` to also
include [`LOAD_LIBRARY_SEARCH_SYSTEM32`][0]:

  * `LOAD_LIBRARY_SEARCH_SYSTEM32`: `%windows%\system32` is searched
    for the DLL and its dependencies.

This specifically is needed so that `JVM.DLL` dependencies such as
`KERNEL32.DLL` can be resolved, allowing `JVM.DLL` to be loaded.

This allows .NET Core on Windows to run `Java.Interop-Tests.dll`!

	> dotnet test bin\TestDebug-netcoreapp3.1\Java.Interop-Tests.dll
	…
	A total of 1 test files matched the specified pattern.
	  Skipped Dispose_ClearsLocalReferences [11 ms]

	Passed!  - Failed:     0, Passed:   617, Skipped:     1, Total:   618, Duration: 2 s - Java.Interop-Tests.dll (netcoreapp3.1)

Install .NET 5.0.103, as that version is required in order for
`dotnet test Java.Interop-Tests.dll` to reliably use a 64-bit process.

~~~

Aside: So you want to create a file which is built into a
*subdirectory* of `$(OutputPath)`.  The "obvious thing" fails:

	<!-- Fails -->
	<ItemGroup>
	  <None Include="$(OutputPath)win-x64\java-interop.dll" />
	</ItemGroup>

This results in:

	error MSB3030: Could not copy the file…

This can be done by:

 1. Providing `%(Link)` item metadata which contains the relative
    directory and filename, *not* `$(OutputPath)`, and

 2. Using `\` *everywhere*.

Thus:

	<!-- Works -->
	<ItemGroup>
	  <None Include="$(OutputPath)win-x64\java-interop.dll">
	    <Link>win-x64\java-interop.dll</Link>
	  </None>
	</ItemGroup>

This works because:

 1. The [`<AppendTargetPath/> task`][1] uses `%(Link)` as an
    *override* when setting `%(TargetPath)`.

 2. The [`_CopyOutOfDateSourceItemsToOutputDirectory` target][2] uses
    `$(OutDir)%(TargetPath)` for `Copy.DestinationFiles`..

 3. The [`<Copy/>` task][3] *ignores errors* when `SourceFiles[i]` and
    `DestinationFiles[i]` are *identical*.  This is why `\` must be
    used, to ensure that the values are identical.

[^0]: …but not .NET Framework, as .NET Framework on CI is launching a
      32-bit process for the unit tests, while we need a 64-bit
      process to match the `JVM.DLL` we're loading.

[0]: https://docs.microsoft.com/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibraryexw
[1]: https://github.com/dotnet/msbuild/blob/17677364d656a98ce4c6eff73b49eddf24f0fd72/src/Tasks/AssignTargetPath.cs#L74
[2]: https://github.com/dotnet/msbuild/blob/17677364d656a98ce4c6eff73b49eddf24f0fd72/src/Tasks/Microsoft.Common.CurrentVersion.targets#L4965-L4977
[3]: https://github.com/dotnet/msbuild/blob/17677364d656a98ce4c6eff73b49eddf24f0fd72/src/Tasks/Copy.cs#L787-L796

build-tools/automation/azure-pipelines.yaml

@@ -18,7 +18,7 @@ pr:
 variables:
   RunningOnCI: true
   Build.Configuration: Release
-  DotNetCoreVersion: 3.1.300
+  DotNetCoreVersion: 5.0.103
   HostedMacImage: macOS-10.15
   HostedWinVS2019: Hosted Windows 2019 with VS2019
   NetCoreTargetFrameworkPathSuffix: -netcoreapp3.1
@@ -90,6 +90,8 @@ jobs:
   - template: templates\core-build.yaml
 
   - template: templates\core-tests.yaml
+    parameters:
+      runNativeDotnetTests: true
 
   - template: templates\fail-on-issue.yaml
 

build-tools/automation/templates/core-tests.yaml

@@ -69,7 +69,7 @@ steps:
 
 - task: DotNetCoreCLI@2
   displayName: 'Tests: Java.Interop'
-  condition: eq('${{ parameters.runNativeTests }}', 'true')
+  condition: or(eq('${{ parameters.runNativeDotnetTests }}', 'true'), eq('${{ parameters.runNativeTests }}', 'true'))
   inputs:
     command: test
     arguments: bin/Test$(Build.Configuration)$(NetCoreTargetFrameworkPathSuffix)/Java.Interop-Tests.dll

build-tools/scripts/NativeToolchain.targets

@@ -0,0 +1,18 @@
+<?xml version="1.0" encoding="utf-8"?>
+<Project>
+  <Target Name="GetNativeBuildCommands">
+    <ItemGroup Condition=" '$(VSINSTALLROOT)' != '' And Exists('$(VSINSTALLROOT)') ">
+      <_Vcvarsall
+          Include="$(VSINSTALLROOT)\VC\Auxiliary\Build\vcvarsall.bat"
+      />
+    </ItemGroup>
+    <PropertyGroup Condition=" '@(_Vcvarsall->Count())' != '0' ">
+      <_Vcvarsall>%(_Vcvarsall.Identity)</_Vcvarsall>
+      <PrepareNativeToolchain>call "$(_Vcvarsall)" </PrepareNativeToolchain>
+    </PropertyGroup>
+    <PropertyGroup>
+      <CmakeGenerator Condition=" $([MSBuild]::IsOSPlatform ('windows')) ">-G "NMake Makefiles"</CmakeGenerator>
+      <CmakeGenerator Condition=" !$([MSBuild]::IsOSPlatform ('windows')) ">-G "Unix Makefiles"</CmakeGenerator>
+    </PropertyGroup>
+  </Target>
+</Project>

build-tools/scripts/RunCmake.proj

@@ -0,0 +1,73 @@
+<?xml version="1.0" encoding="utf-8"?>
+<Project DefaultTargets="Prepare" ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
+  <Target Name="Cmake"
+      DependsOnTargets="_ValidateCmake;_RunCmake"
+  />
+  <Target Name="_ValidateCmake">
+    <Error
+        Condition=" '$(CmakePath)' == '' "
+        Text="Set the `%24(CmakePath)` property."
+    />
+    <Error
+        Condition=" '$(CmakeGenerator)' == '' "
+        Text="Set the `%24(CmakeGenerator)` property."
+    />
+    <Error
+        Condition=" '$(CmakeSourceDir)' == '' "
+        Text="Set the `%24(CmakeSourceDir)` property."
+    />
+    <Error
+        Condition=" '$(CmakeBuildDir)' == '' "
+        Text="Set the `%24(CmakeBuildDir)` property."
+    />
+  </Target>
+
+  <Target Name="_RunCmake">
+    <PropertyGroup>
+      <_Prepare>$(PrepareNativeToolchain)</_Prepare>
+      <_Prepare Condition=" '$(_Prepare)' != '' And !$(_Prepare.Trim().EndsWith('&amp;&amp;')) ">$(_Prepare) &amp;&amp;</_Prepare>
+      <_SourceDir>$(CmakeSourceDir.Replace('%5c', '/'))</_SourceDir>
+      <_BuildDir>$(CmakeBuildDir.Replace('%5c', '/'))</_BuildDir>
+      <_ExtraArgs>$(CmakeExtraArgs.Replace('%5c', '/'))</_ExtraArgs>
+    </PropertyGroup>
+    <Exec
+        ContinueOnError="WarnAndContinue"
+        Command="$(_Prepare) $(CmakePath) $(CmakeGenerator) -S &quot;$(_SourceDir)&quot; -B &quot;$(_BuildDir)&quot; $(_ExtraArgs) &amp;&amp; $(CmakePath) --build &quot;$(_BuildDir)&quot; -v"
+    />
+    <PropertyGroup>
+      <_CmakeStatus>$(MSBuildLastTaskResult)</_CmakeStatus>
+    </PropertyGroup>
+    <ReadLinesFromFile
+        Condition=" '$(_CmakeStatus)' == 'false' "
+        File="$(CmakeBuildDir)CMakeFiles/CMakeOutput.log">
+      <Output TaskParameter="Lines" ItemName="_CmakeLog" />
+    </ReadLinesFromFile>
+    <Message
+        Condition=" '$(_CmakeStatus)' == 'false' "
+        Text="CMakeOutput.log"
+    />
+    <Message
+        Condition=" '$(_CmakeStatus)' == 'false' "
+        Text="@(_CmakeLog, '
+')"
+    />
+    <ReadLinesFromFile
+        Condition=" '$(_CmakeStatus)' == 'false' "
+        File="$(CmakeBuildDir)CMakeFiles/CMakeError.log">
+      <Output TaskParameter="Lines" ItemName="_CmakeErrorLog" />
+    </ReadLinesFromFile>
+    <Message
+        Condition=" '$(_CmakeStatus)' == 'false' "
+        Text="CMakeError.log"
+    />
+    <Message
+        Condition=" '$(_CmakeStatus)' == 'false' "
+        Text="@(_CmakeErrorLog, '
+')"
+    />
+    <Error
+        Condition=" '$(_CmakeStatus)' == 'false' "
+        Text="`cmake` failed.  See previous messages."
+    />
+  </Target>
+</Project>

src/Java.Runtime.Environment/Java.Interop/JreRuntime.cs

@@ -70,6 +70,15 @@ public JreRuntime CreateJreVM ()
 
 	public class JreRuntime : JniRuntime
 	{
+		static JreRuntime ()
+		{
+			if (Environment.OSVersion.Platform == PlatformID.Win32NT) {
+				var baseDir = Path.GetDirectoryName (typeof (JreRuntime).Assembly.Location);
+				var newDir  = Path.Combine (baseDir, Environment.Is64BitProcess ? "win-x64" : "win-x86");
+				NativeMethods.AddDllDirectory (newDir);
+			}
+		}
+
 		static int CreateJavaVM (out IntPtr javavm, out IntPtr jnienv, ref JavaVMInitArgs args)
 		{
 			return NativeMethods.java_interop_jvm_create (out javavm, out jnienv, ref args);
@@ -168,6 +177,9 @@ partial class NativeMethods {
 
 		[DllImport (JavaInteropLib, CharSet=CharSet.Ansi, CallingConvention=CallingConvention.Cdecl)]
 		internal static extern int java_interop_jvm_create (out IntPtr javavm, out IntPtr jnienv, ref JavaVMInitArgs args);
+
+		[DllImport ("kernel32", CharSet=CharSet.Unicode)]
+		internal static extern int AddDllDirectory (string NewDirectory);
 	}
 }
 

src/java-interop/CMakeLists.txt

@@ -0,0 +1,61 @@
+set(CMAKE_OSX_ARCHITECTURES x86_64 arm64)
+
+project(
+    java-interop
+    DESCRIPTION "Java.Interop native support"
+    HOMEPAGE_URL "https://github.com/xamarin/java.interop/"
+    LANGUAGES CXX C
+)
+
+set(CMAKE_C_STANDARD 11)
+set(CMAKE_CXX_STANDARD 17)
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+set(CMAKE_CXX_EXTENSIONS OFF)
+set(CMAKE_CXX_VISIBILITY_PRESET hidden)
+
+option(ENABLE_MONO_INTEGRATION "Require Mono runtime" OFF)
+
+cmake_minimum_required(VERSION 3.10.2)
+
+set(JAVA_INTEROP_CORE_SOURCES
+    java-interop-dlfcn.cc
+    java-interop-jvm.cc
+    java-interop-logger.cc
+    java-interop-util.cc
+    java-interop.cc
+    ${JNI_C_PATH}
+)
+set(JAVA_INTEROP_MONO_SOURCES
+    java-interop-gc-bridge-mono.cc
+    java-interop-mono.cc
+)
+
+add_compile_definitions("JAVA_INTEROP_DLL_EXPORT")
+add_compile_definitions("JI_DLL_EXPORT")
+
+foreach(dir in ${JDK_INCLUDE_LIST})
+  include_directories(${dir})
+endforeach()
+
+set(LINK_FLAGS "")
+
+if(ENABLE_MONO_INTEGRATION)
+  foreach(dir in ${MONO_INCLUDE_LIST})
+    include_directories(${dir})
+  endforeach()
+  list(APPEND LINK_FLAGS ${MONO_LINK_FLAGS})
+  list(APPEND LINK_FLAGS "-Wl,-undefined -Wl,suppress -Wl,-flat_namespace")
+  set(JAVA_INTEROP_SOURCES ${JAVA_INTEROP_CORE_SOURCES} ${JAVA_INTEROP_MONO_SOURCES})
+else()
+  set(JAVA_INTEROP_SOURCES ${JAVA_INTEROP_CORE_SOURCES})
+endif()
+
+add_library(
+    java-interop
+    SHARED
+    ${JAVA_INTEROP_SOURCES}
+)
+target_link_libraries(
+    java-interop
+    ${LINK_FLAGS}
+)

src/java-interop/Directory.Build.targets

@@ -0,0 +1,100 @@
+<Project>
+
+  <Import Project="..\..\build-tools\scripts\NativeToolchain.targets" />
+
+  <PropertyGroup>
+    <_JavaInteropLibName Condition=" $([MSBuild]::IsOSPlatform ('osx')) ">libjava-interop.dylib</_JavaInteropLibName>
+    <_JavaInteropLibName Condition=" $([MSBuild]::IsOSPlatform ('linux')) ">libjava-interop.so</_JavaInteropLibName>
+    <_JavaInteropLibName Condition=" $([MSBuild]::IsOSPlatform ('windows')) ">java-interop.dll</_JavaInteropLibName>
+  </PropertyGroup>
+
+  <ItemGroup Condition=" $([MSBuild]::IsOSPlatform ('windows')) ">
+    <_JavaInteropNativeLib Include="CMakeLists.txt">
+      <Arch>x86_amd64</Arch>
+      <Dir>win-x64\</Dir>
+    </_JavaInteropNativeLib>
+    <_JavaInteropNativeLib Include="CMakeLists.txt">
+      <Arch>x86</Arch>
+      <Dir>win-x86\</Dir>
+    </_JavaInteropNativeLib>
+  </ItemGroup>
+
+  <ItemGroup Condition=" !$([MSBuild]::IsOSPlatform ('windows')) ">
+    <_JavaInteropNativeLib Include="CMakeLists.txt" />
+  </ItemGroup>
+
+  <ItemGroup>
+    <None Include="@(_JavaInteropNativeLib->'$(OutputPath)%(Dir)$(_JavaInteropLibName)')">
+      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+      <Link>%(Dir)$(_JavaInteropLibName)</Link>
+    </None>
+  </ItemGroup>
+
+  <ItemGroup>
+    <ClInclude Include="*.h" />
+  </ItemGroup>
+
+  <ItemGroup>
+    <ClCompile Include="$(IntermediateOutputPath)jni.c" />
+    <ClCompile Include="*.cc" />
+  </ItemGroup>
+
+  <Target Name="_BuildJni_c"
+      Inputs="$(_JNIEnvGenPath)"
+      Outputs="$(IntermediateOutputPath)jni.c">
+    <MakeDir Directories="$(OutputPath)" />
+    <Exec Command="$(_RunJNIEnvGen) $(IntermediateOutputPath)jni.g.cs $(IntermediateOutputPath)jni.c" />
+  </Target>
+
+  <Target Name="_GetCmakeOptions">
+    <PropertyGroup Condition=" '$(TargetFramework)' == 'net472' And '@(MonoIncludePath->Count())' != '0' ">
+      <_MonoDirs>"-DMONO_INCLUDE_LIST=@(MonoIncludePath, ';')"</_MonoDirs>
+      <_MonoLib>"-DMONO_LINK_FLAGS=$(MonoLibs)"</_MonoLib>
+      <_EnableMono>-DENABLE_MONO_INTEGRATION=ON</_EnableMono>
+    </PropertyGroup>
+    <PropertyGroup>
+      <_JdkDirs>"-DJDK_INCLUDE_LIST=@(JdkIncludePath, ';')"</_JdkDirs>
+      <_Jni_c>"-DJNI_C_PATH=$(MSBuildThisFileDirectory)$(IntermediateOutputPath)jni.c"</_Jni_c>
+      <_ExtraArgs>$([MSBuild]::Escape('$(_JdkDirs) $(_Jni_c) $(_EnableMono) $(_MonoDirs) $(_MonoLib)'))</_ExtraArgs>
+    </PropertyGroup>
+  </Target>
+
+  <Target Name="_BuildLibs"
+      DependsOnTargets="GetNativeBuildCommands;_BuildJni_c;_GetCmakeOptions"
+      BeforeTargets="Build"
+      Inputs="@(_JavaInteropNativeLib);$(MSBuildThisFileFullPath);java-interop.csproj;@(ClInclude);@(ClCompile)"
+      Outputs="$(OutputPath)%(_JavaInteropNativeLib.Dir)$(_JavaInteropLibName)">
+    <MakeDir Directories="$(IntermediateOutputPath)%(_JavaInteropNativeLib.Dir)" />
+    <ItemGroup>
+      <_Cmake
+          Condition=" '$(PrepareNativeToolchain)' != '' "
+          Include="PrepareNativeToolchain=$(PrepareNativeToolchain) %(_JavaInteropNativeLib.Arch)"
+      />
+      <_Cmake Include="CmakePath=$(CmakePath)" />
+      <_Cmake Include="CmakeGenerator=$(CmakeGenerator)" />
+      <_Cmake Include="CmakeSourceDir=$(MSBuildThisFileDirectory)" />
+      <_Cmake Include="CmakeBuildDir=$(MSBuildThisFileDirectory)$(IntermediateOutputPath)%(_JavaInteropNativeLib.Dir)" />
+      <_Cmake Include="CmakeExtraArgs=$(_ExtraArgs)" />
+    </ItemGroup>
+    <MSBuild
+        Projects="..\..\build-tools\scripts\RunCmake.proj"
+        Properties="@(_Cmake)"
+        Targets="Cmake"
+    />
+    <MakeDir Directories="$(OutputPath)%(_JavaInteropNativeLib.Dir)" />
+    <ItemGroup>
+      <_Libs Include="$(IntermediateOutputPath)%(_JavaInteropNativeLib.Dir)$(_JavaInteropLibName)*" />
+    </ItemGroup>
+    <Copy
+        SourceFiles="@(_Libs)"
+        DestinationFolder="$(OutputPath)%(_JavaInteropNativeLib.Dir)"
+    />
+    <Touch Files="$(OutputPath)%(_JavaInteropNativeLib.Dir)$(_JavaInteropLibName)" />
+  </Target>
+
+  <Target Name="_Clean"
+      AfterTargets="Clean">
+    <Delete Files="@(None)" />
+  </Target>
+
+</Project>