From ce2bc689a19cb45f7d7bfdd371c16c54b018a020 Mon Sep 17 00:00:00 2001 From: Marek Habersack Date: Mon, 10 Feb 2020 20:40:22 +0000 Subject: [PATCH] [build] Optimize managed <-> java type lookups (#3992) Typemap data is used to correlate JNI type names to .NET Assembly- Qualified Type Names, and vice versa: java/lang/Object <=> Java.Lang.Object, Mono.Android Typemap data is used from `JNIEnv.GetJniName()` for managed-to-JNI lookups, and from `TypeManager.GetJavaToManagedType()` for JNI-to-managed lookups. When [typemap files were first introduced][0], they relied on: 1. A string-oriented mapping from Java type names to .NET Assembly Qualified names and vice versa; and 2. A binary search via **bsearch**(3) over this table to find the associated type, using the source type as the "key". (The introduction of `libxamarin-app.so` (decfbccf) merely moved the (formerly separate) typemap data into `libxamarin-app.so` for Release config builds -- Debug builds continued using separate typemap files -- but didn't otherwise change how these mappings work.) This approach works very well at the expense of data size -- shorter strings are 0-padded to a common width -- and slightly degraded performance because of the requirement to perform string comparisons. Furthermore, the managed-to-JNI lookup required that Reflection is used to obtain the Assembly Qualified type name (`Type.AssemblyQualifiedName`), while the JNI-to-managed lookup likewise requires some Reflection to obtain `Type` instances (via `Type.GetType()`). Rework the typemap data in an effort to reduce Reflection use: For the managed-to-JNI mapping, use the combination of `type.Module.ModuleVersionId` and `Type.MetadataToken` -- a GUID and an int -- instead of using `Type.AssemblyQualifiedName`. This allows us to perform the binary search over a set of 20 bytes (16 bytes for the UUID and 4 bytes for the token ID). JNI-to-managed lookups still need to rely on a binary search across strings, but instead of mapping the JNI name to an Assembly-Qualified Type Name and using `Type.GetType()`, we instead map the JNI name to the same GUID+token pair via a new internal call which uses `mono_class_get()` & `mono_type_get_object()` to return the `Type`. As a result of this fundamental change, `libxamarin-app.so` decreases in size, and app startup time is reduced. For a Release configuration build of `tests/Xamarin.Forms-Performance-Integration`, `libs/arm64-v8a/libxamarin-app.so` shrinks from 377KB to 104KB (!), and on a Pixel 3 XL app the `ActivityTaskManager: Displayed` time was reduced from 805ms to 789ms (`$(AndroidEnablePreloadAssemblies)`=True), a nearly 10% improvement. Build time is also minimally impacted; `` task time is reduced from 389ms to 247ms for the Xamarin.Forms build. ~~ Fast Deployment ~~ When Xamarin.Android Fast Deployment is *not* used for Debug builds (which is the case for OSS builds of xamarin-android), the typemap generation and deployment is identical for both Release and Debug builds: `libxamarin-app.so` contains the new typemap information. In commercial Xamarin.Android builds which use Fast Deployment, the typemap data is instead stored in two sets of files: * `typemap.index`: stores the mapping from module GUIDs to assembly filenames. * `*.typemap`: One file per .NET *module*, contain both the JNI-to- managed and managed-to-JNI maps, the latter using indexes into the Java to managed maps. All of these files are loaded during Debug app startup and used to construct a dataset which is then searched during all the lookups. ~~ File Formats ~~ All data in all file formats is little-endian. The `typemap.index` file stores the mapping from GUIDs to module filenames such as `Mono.Android.dll`. The file format in pseudo-C++: struct TypemapIndexHeader { byte magic [4]; // "XATI" uint32_t format_version; uint32_t entry_count; uint32_t module_filename_width; TypemapIndexEntry entries [entry_count]; }; struct TypemapIndexEntry { UUID module_uuid; // 16 bytes byte file_name [TypemapIndexHeader::module_filename_width]; }; `TypemapIndexHeader::module_filename_width` is the maximum filename length of any entry within `TypemapIndexEntry::file_name` + 1 for a terminating `NUL`. There is no order required within `TypemapIndexHeader::entries`. `TypemapIndexEntry::file_name` is `NUL` padded, filling the entire array until the next `TypemapIndexEntry` entry. The `*.typemap` file stores the mappings from JNI type names to module GUID and type token pairs. The file format in pseudo-C++: struct TypemapFileHeader { byte magic [4]; // "XATM" uint32_t format_version; GUID module_uuid; uint32_t entry_count; uint32_t duplicate_count; uint32_t jni_name_width; uint32_t assembly_name_size; byte assembly_name [assembly_name_size]; TypemapFileJavaToManagedEntry java_to_managed [entry_count]; TypemapFileManagedToJavaEntry managed_to_java [entry_count]; TypemapFileManagedToJavaEntry duplicates [duplicate_count]; }; struct TypemapFileJavaToManagedEntry { byte jni_name [TypemapFileHeader::jni_name_width]; uint32_t managed_type_token; }; struct TypemapFileManagedToJavaEntry { uint32_t managed_type_token; uint32_t java_to_managed_index; }; `TypemapFileHeader::duplicate_count` may be 0. `TypemapFileJavaToManagedEntry::jni_name` is `NUL` padded. `TypemapFileJavaToManagedEntry::managed_type_token` is the value of `Type.MetadataToken`. `TypemapFileManagedToJavaEntry::java_to_managed_index` is the index within `TypemapFileHeader::java_to_managed` that contains the JNI name. [0]: https://github.com/xamarin/monodroid/commit/e69b76e2b86f92fa8440756c44ad3e2205563ac7 [1]: https://github.com/xamarin/java.interop/blob/3226a4b57ad84574a69a151a310b077cfe69ee19/src/Java.Interop.Tools.JavaCallableWrappers/Java.Interop.Tools.JavaCallableWrappers/TypeNameMapGenerator.cs#L16-L56 --- Documentation/guides/messages/README.md | 1 + Documentation/guides/messages/xa4308.md | 18 + .../Steps/Step_DownloadMonoArchive.cs | 50 +- .../Android.Runtime/AndroidRuntime.cs | 12 +- src/Mono.Android/Android.Runtime/JNIEnv.cs | 53 +- src/Mono.Android/Java.Interop/TypeManager.cs | 16 +- .../Properties/AssemblyInfo.cs.in | 4 + .../Java.Interop-Tests.csproj | 5 +- .../Properties/AssemblyInfo.cs | 30 - .../Test/Java.Interop/JnienvTest.cs | 26 +- .../Mono.Android-Test.Library.csproj | 1 - .../Properties/AssemblyInfo.cs | 30 - .../Test/Mono.Android-Tests.csproj | 2 + .../Android/Xamarin.Android.Designer.targets | 12 +- .../Properties/Resources.Designer.cs | 8 + .../Properties/Resources.resx | 5 +- .../Properties/xlf/Resources.cs.xlf | 5 + .../Properties/xlf/Resources.de.xlf | 5 + .../Properties/xlf/Resources.es.xlf | 5 + .../Properties/xlf/Resources.fr.xlf | 5 + .../Properties/xlf/Resources.it.xlf | 5 + .../Properties/xlf/Resources.ja.xlf | 5 + .../Properties/xlf/Resources.ko.xlf | 5 + .../Properties/xlf/Resources.pl.xlf | 5 + .../Properties/xlf/Resources.pt-BR.xlf | 5 + .../Properties/xlf/Resources.ru.xlf | 5 + .../Properties/xlf/Resources.tr.xlf | 5 + .../Properties/xlf/Resources.zh-Hans.xlf | 5 + .../Properties/xlf/Resources.zh-Hant.xlf | 5 + .../Tasks/GenerateJavaStubs.cs | 170 ++--- .../Tasks/GeneratePackageManagerJava.cs | 13 +- .../Tasks/PrepareAbiItems.cs | 54 +- .../Tasks/RemoveRegisterAttribute.cs | 8 +- .../IncrementalBuildTest.cs | 8 +- .../Utilities/EnvironmentHelper.cs | 29 +- .../ARMNativeAssemblerTargetProvider.cs | 6 + ...pplicationConfigNativeAssemblyGenerator.cs | 27 +- .../Utilities/MonoAndroidHelper.cs | 17 +- .../NativeAssemblerTargetProvider.cs | 20 +- .../Utilities/NativeAssemblyDataStream.cs | 235 ------- .../Utilities/NativeAssemblyGenerator.cs | 154 ++++- .../Utilities/NativeTypeMappingData.cs | 60 ++ .../Utilities/TypeMapGenerator.cs | 423 ++++++++++++ .../TypeMappingNativeAssemblyGenerator.cs | 264 +++++-- .../X86NativeAssemblerTargetProvider.cs | 6 + .../Xamarin.Android.Common.targets | 37 +- src/monodroid/jni/android-system.cc | 23 +- src/monodroid/jni/android-system.hh | 3 +- src/monodroid/jni/application_dso_stub.cc | 14 +- src/monodroid/jni/cpp-util.hh | 11 - src/monodroid/jni/embedded-assemblies-zip.cc | 13 - src/monodroid/jni/embedded-assemblies.cc | 649 ++++++++++++++---- src/monodroid/jni/embedded-assemblies.hh | 30 +- src/monodroid/jni/external-api.cc | 10 +- src/monodroid/jni/monodroid-glue-internal.hh | 1 + src/monodroid/jni/monodroid-glue.cc | 24 +- src/monodroid/jni/xamarin-app.h | 42 -- src/monodroid/jni/xamarin-app.hh | 86 +++ .../Tests/InstantRunTest.cs | 7 +- .../Mono.Android-TestsAppBundle.csproj | 2 + .../Mono.Android-TestsMultiDex.csproj | 2 + .../Environment.txt | 7 +- .../MSBuildDeviceIntegration.csv | 6 +- 63 files changed, 1939 insertions(+), 860 deletions(-) create mode 100644 Documentation/guides/messages/xa4308.md delete mode 100644 src/Mono.Android/Test/Java.Interop-Tests/Properties/AssemblyInfo.cs delete mode 100644 src/Mono.Android/Test/Mono.Android-Test.Library/Properties/AssemblyInfo.cs delete mode 100644 src/Xamarin.Android.Build.Tasks/Utilities/NativeAssemblyDataStream.cs create mode 100644 src/Xamarin.Android.Build.Tasks/Utilities/NativeTypeMappingData.cs create mode 100644 src/Xamarin.Android.Build.Tasks/Utilities/TypeMapGenerator.cs delete mode 100644 src/monodroid/jni/xamarin-app.h create mode 100644 src/monodroid/jni/xamarin-app.hh diff --git a/Documentation/guides/messages/README.md b/Documentation/guides/messages/README.md index e5a2d79770b..c5aca1639a9 100644 --- a/Documentation/guides/messages/README.md +++ b/Documentation/guides/messages/README.md @@ -104,6 +104,7 @@ ms.date: 01/24/2020 + [XA4305](xa4305.md): MultiDex is enabled, but '{nameof (MultiDexMainDexListFile)}' was not specified. + [XA4306](xa4306.md): R8 does not support \`@(MultiDexMainDexList)\` files when android:minSdkVersion >= 21 + [XA4307](xa4307.md): Invalid ProGuard configuration file. ++ [XA4308](xa4308.md): Failed to generate type maps ## XA5xxx: GCC and toolchain diff --git a/Documentation/guides/messages/xa4308.md b/Documentation/guides/messages/xa4308.md new file mode 100644 index 00000000000..d1f474665e3 --- /dev/null +++ b/Documentation/guides/messages/xa4308.md @@ -0,0 +1,18 @@ +--- +title: Xamarin.Android error XA4308 +description: XA4308 error code +ms.date: 02/07/2020 +--- +# Xamarin.Android error XA4308 + +## Issue + +The `GenerateJavaStubs` task was unable to generate type maps. Detailed diagnostic will be found before this +error in the build log. + +## Solution + +Consider submitting a [bug][bug] if you are getting this warning under +normal circumstances. + +[bug]: https://github.com/xamarin/xamarin-android/wiki/Submitting-Bugs,-Feature-Requests,-and-Pull-Requests diff --git a/build-tools/xaprepare/xaprepare/Steps/Step_DownloadMonoArchive.cs b/build-tools/xaprepare/xaprepare/Steps/Step_DownloadMonoArchive.cs index c5bf663aa79..0598e2387f3 100644 --- a/build-tools/xaprepare/xaprepare/Steps/Step_DownloadMonoArchive.cs +++ b/build-tools/xaprepare/xaprepare/Steps/Step_DownloadMonoArchive.cs @@ -60,26 +60,39 @@ async Task DownloadMonoArchive (Context context) localPath = Path.Combine (context.Properties.GetRequiredValue (KnownProperties.AndroidToolchainCacheDirectory), archiveFileName); } - bool result = await DownloadAndUpackIfNeeded ( - context, - "Mono", - context.MonoArchiveCustomUrl, - localPath, - archiveFileName, - Configurables.Paths.MonoSDKSOutputDir - ); + bool result = false; + for (uint i = 0; i < 3; i++) { + result = await DownloadAndUpackIfNeeded ( + context, + "Mono", + context.MonoArchiveCustomUrl, + localPath, + archiveFileName, + Configurables.Paths.MonoSDKSOutputDir + ); + + if (result) + break; + } if (!result) return false; - return await DownloadAndUpackIfNeeded ( - context, - "Windows Mono", - customUrl: null, - localPath: Configurables.Paths.MonoArchiveWindowsLocalPath, - archiveFileName: Configurables.Paths.MonoArchiveWindowsFileName, - destinationDirectory: Configurables.Paths.BCLWindowsOutputDir - ); + for (uint i = 0; i < 3; i++) { + result = await DownloadAndUpackIfNeeded ( + context, + "Windows Mono", + customUrl: null, + localPath: Configurables.Paths.MonoArchiveWindowsLocalPath, + archiveFileName: Configurables.Paths.MonoArchiveWindowsFileName, + destinationDirectory: Configurables.Paths.BCLWindowsOutputDir + ); + + if (result) + break; + } + + return result; } async Task DownloadAndUpackIfNeeded (Context context, string name, string customUrl, string localPath, string archiveFileName, string destinationDirectory) @@ -109,7 +122,7 @@ async Task DownloadAndUpackIfNeeded (Context context, string name, string await Download (context, url, localPath, $"{name} Archive", archiveFileName, downloadStatus); if (!File.Exists (localPath)) { - Log.InfoLine ($"Download of {name} archive from {url} failed, Mono will be rebuilt"); + Log.InfoLine ($"Download of {name} archive from {url} failed"); return false; } } @@ -118,7 +131,8 @@ async Task DownloadAndUpackIfNeeded (Context context, string name, string if (!await Utilities.Unpack (localPath, tempDir, cleanDestinatioBeforeUnpacking: true)) { Utilities.DeleteFileSilent (localPath); Utilities.DeleteDirectorySilent (destinationDirectory); - Log.WarningLine ($"Failed to unpack {name} archive {localPath}, Mono will be rebuilt"); + Log.WarningLine ($"Failed to unpack {name} archive {localPath}"); + Utilities.DeleteFileSilent (localPath); return false; } diff --git a/src/Mono.Android/Android.Runtime/AndroidRuntime.cs b/src/Mono.Android/Android.Runtime/AndroidRuntime.cs index 478d72dd581..f23cf9f5358 100644 --- a/src/Mono.Android/Android.Runtime/AndroidRuntime.cs +++ b/src/Mono.Android/Android.Runtime/AndroidRuntime.cs @@ -231,9 +231,9 @@ protected override IEnumerable GetTypesForSimpleReference (string jniSimpl protected override string GetSimpleReference (Type type) { - var j = JNIEnv.monodroid_typemap_managed_to_java (type.FullName + ", " + type.Assembly.GetName ().Name); - if (j != IntPtr.Zero) { - return Marshal.PtrToStringAnsi (j); + string j = JNIEnv.TypemapManagedToJava (type); + if (j != null) { + return j; } if (JNIEnv.IsRunningOnDesktop) { return JavaNativeTypeManager.ToJniName (type); @@ -243,9 +243,9 @@ protected override string GetSimpleReference (Type type) protected override IEnumerable GetSimpleReferences (Type type) { - var j = JNIEnv.monodroid_typemap_managed_to_java (type.FullName + ", " + type.Assembly.GetName ().Name); - if (j != IntPtr.Zero) { - yield return Marshal.PtrToStringAnsi (j); + string j = JNIEnv.TypemapManagedToJava (type); + if (j != null) { + yield return j; } if (JNIEnv.IsRunningOnDesktop) { yield return JavaNativeTypeManager.ToJniName (type); diff --git a/src/Mono.Android/Android.Runtime/JNIEnv.cs b/src/Mono.Android/Android.Runtime/JNIEnv.cs index 842837271e7..4b3bc8d3bf8 100644 --- a/src/Mono.Android/Android.Runtime/JNIEnv.cs +++ b/src/Mono.Android/Android.Runtime/JNIEnv.cs @@ -57,10 +57,14 @@ public static partial class JNIEnv { static UncaughtExceptionHandler defaultUncaughtExceptionHandler; internal static bool IsRunningOnDesktop; + internal static bool LogTypemapMissStackTrace; static AndroidRuntime androidRuntime; static BoundExceptionType BoundExceptionType; + [ThreadStatic] + static byte[] mvid_bytes; + internal static AndroidValueManager AndroidValueManager; [DllImport ("__Internal", CallingConvention = CallingConvention.Cdecl)] @@ -146,6 +150,8 @@ internal static unsafe void Initialize (JnienvInitializeArgs* args) partial_timing_sequence = monodroid_timing_start (null); } + LogTypemapMissStackTrace = (args->logCategories & (uint)LogCategories.Assembly) != 0; + gref_gc_threshold = args->grefGcThreshold; java_vm = args->javaVm; @@ -632,16 +638,55 @@ public static string GetClassNameFromInstance (IntPtr jobject) } [DllImport ("__Internal", CallingConvention = CallingConvention.Cdecl)] - internal static extern IntPtr monodroid_typemap_managed_to_java (string managed); + static extern IntPtr monodroid_typemap_managed_to_java (byte[] mvid, int token); + + internal static void LogTypemapTrace (StackTrace st) + { + string trace = st.ToString ()?.Trim (); + if (String.IsNullOrEmpty (trace)) + return; + + monodroid_log (LogLevel.Warn, LogCategories.Assembly, "typemap: called from"); + foreach (string line in trace.Split ('\n')) { + monodroid_log (LogLevel.Warn, LogCategories.Assembly, line); + } + } + + internal static string TypemapManagedToJava (Type type) + { + if (mvid_bytes == null) + mvid_bytes = new byte[16]; + + Span mvid = new Span(mvid_bytes); + byte[] mvid_slow = null; + if (!type.Module.ModuleVersionId.TryWriteBytes (mvid)) { + monodroid_log (LogLevel.Warn, LogCategories.Default, $"Failed to obtain module MVID using the fast method, falling back to the slow one"); + mvid_slow = type.Module.ModuleVersionId.ToByteArray (); + } + + IntPtr ret = monodroid_typemap_managed_to_java (mvid_slow == null ? mvid_bytes : mvid_slow, type.MetadataToken); + + if (ret == IntPtr.Zero) { + if (LogTypemapMissStackTrace) { + monodroid_log (LogLevel.Warn, LogCategories.Default, $"typemap: failed to map managed type to Java type: {type.AssemblyQualifiedName} (Module ID: {type.Module.ModuleVersionId}; Type token: {type.MetadataToken})"); + LogTypemapTrace (new StackTrace (true)); + } + + return null; + } + + return Marshal.PtrToStringAnsi (ret); + } public static string GetJniName (Type type) { if (type == null) throw new ArgumentNullException ("type"); - var java = monodroid_typemap_managed_to_java (type.FullName + ", " + type.Assembly.GetName ().Name); - return java == IntPtr.Zero + + string java = TypemapManagedToJava (type); + return java == null ? JavaNativeTypeManager.ToJniName (type) - : Marshal.PtrToStringAnsi (java); + : java; } public static IntPtr ToJniHandle (IJavaObject value) diff --git a/src/Mono.Android/Java.Interop/TypeManager.cs b/src/Mono.Android/Java.Interop/TypeManager.cs index 0cf4265f62c..ebab6f33f49 100644 --- a/src/Mono.Android/Java.Interop/TypeManager.cs +++ b/src/Mono.Android/Java.Interop/TypeManager.cs @@ -2,6 +2,7 @@ using System.Collections.Generic; using System.Reflection; using System.Linq; +using System.Runtime.CompilerServices; using System.Runtime.InteropServices; using Java.Interop.Tools.TypeNameMappings; @@ -203,22 +204,25 @@ static Exception CreateJavaLocationException () return new JavaLocationException (loc.ToString ()); } - [DllImport ("__Internal", CallingConvention = CallingConvention.Cdecl)] - internal static extern IntPtr monodroid_typemap_java_to_managed (string java); + [MethodImplAttribute(MethodImplOptions.InternalCall)] + static extern Type monodroid_typemap_java_to_managed (string java_type_name); internal static Type GetJavaToManagedType (string class_name) { - var t = monodroid_typemap_java_to_managed (class_name); - if (t != IntPtr.Zero) - return Type.GetType (Marshal.PtrToStringAnsi (t)); + Type type = monodroid_typemap_java_to_managed (class_name); + if (type != null) + return type; if (!JNIEnv.IsRunningOnDesktop) { + // Miss message is logged in the native runtime + if (JNIEnv.LogTypemapMissStackTrace) + JNIEnv.LogTypemapTrace (new System.Diagnostics.StackTrace (true)); return null; } __TypeRegistrations.RegisterPackages (); - var type = (Type) null; + type = null; int ls = class_name.LastIndexOf ('/'); var package = ls >= 0 ? class_name.Substring (0, ls) : ""; List> mappers; diff --git a/src/Mono.Android/Properties/AssemblyInfo.cs.in b/src/Mono.Android/Properties/AssemblyInfo.cs.in index 3063db7a1b8..960d373e6fa 100644 --- a/src/Mono.Android/Properties/AssemblyInfo.cs.in +++ b/src/Mono.Android/Properties/AssemblyInfo.cs.in @@ -27,3 +27,7 @@ using System.Runtime.CompilerServices; [assembly: System.Runtime.CompilerServices.TypeForwardedToAttribute(typeof(System.Drawing.Size))] [assembly: System.Runtime.CompilerServices.TypeForwardedToAttribute(typeof(System.Drawing.SizeF))] [assembly: System.Runtime.CompilerServices.TypeForwardedToAttribute(typeof(System.Drawing.SystemColors))] +[assembly: InternalsVisibleTo("Mono.Android-Tests, PublicKey=0024000004800000940000000602000000240000525341310004000011000000438ac2a5acfbf16cbd2b2b47a62762f273df9cb2795ceccdf77d10bf508e69e7a362ea7a45455bbf3ac955e1f2e2814f144e5d817efc4c6502cc012df310783348304e3ae38573c6d658c234025821fda87a0be8a0d504df564e2c93b2b878925f42503e9d54dfef9f9586d9e6f38a305769587b1de01f6c0410328b2c9733db")] +[assembly: InternalsVisibleTo("Java.Interop-Tests, PublicKey=0024000004800000940000000602000000240000525341310004000011000000438ac2a5acfbf16cbd2b2b47a62762f273df9cb2795ceccdf77d10bf508e69e7a362ea7a45455bbf3ac955e1f2e2814f144e5d817efc4c6502cc012df310783348304e3ae38573c6d658c234025821fda87a0be8a0d504df564e2c93b2b878925f42503e9d54dfef9f9586d9e6f38a305769587b1de01f6c0410328b2c9733db")] +[assembly: InternalsVisibleTo("Mono.Android-TestsMultiDex, PublicKey=0024000004800000940000000602000000240000525341310004000011000000438ac2a5acfbf16cbd2b2b47a62762f273df9cb2795ceccdf77d10bf508e69e7a362ea7a45455bbf3ac955e1f2e2814f144e5d817efc4c6502cc012df310783348304e3ae38573c6d658c234025821fda87a0be8a0d504df564e2c93b2b878925f42503e9d54dfef9f9586d9e6f38a305769587b1de01f6c0410328b2c9733db")] +[assembly: InternalsVisibleTo("Mono.Android-TestsAppBundle, PublicKey=0024000004800000940000000602000000240000525341310004000011000000438ac2a5acfbf16cbd2b2b47a62762f273df9cb2795ceccdf77d10bf508e69e7a362ea7a45455bbf3ac955e1f2e2814f144e5d817efc4c6502cc012df310783348304e3ae38573c6d658c234025821fda87a0be8a0d504df564e2c93b2b878925f42503e9d54dfef9f9586d9e6f38a305769587b1de01f6c0410328b2c9733db")] diff --git a/src/Mono.Android/Test/Java.Interop-Tests/Java.Interop-Tests.csproj b/src/Mono.Android/Test/Java.Interop-Tests/Java.Interop-Tests.csproj index a587304a775..fe28f678c9a 100644 --- a/src/Mono.Android/Test/Java.Interop-Tests/Java.Interop-Tests.csproj +++ b/src/Mono.Android/Test/Java.Interop-Tests/Java.Interop-Tests.csproj @@ -17,6 +17,8 @@ Off v10.0 true + true + ..\..\..\..\product.snk @@ -48,12 +50,11 @@ - ..\..\..\..\packages\mono.linq.expressions.2.0.0\lib\netstandard2.0\Mono.Linq.Expressions.dll + ..\..\..\..\packages\Mono.Linq.Expressions.2.0.0\lib\netstandard2.0\Mono.Linq.Expressions.dll - diff --git a/src/Mono.Android/Test/Java.Interop-Tests/Properties/AssemblyInfo.cs b/src/Mono.Android/Test/Java.Interop-Tests/Properties/AssemblyInfo.cs deleted file mode 100644 index 887802784b2..00000000000 --- a/src/Mono.Android/Test/Java.Interop-Tests/Properties/AssemblyInfo.cs +++ /dev/null @@ -1,30 +0,0 @@ -using System.Reflection; -using System.Runtime.CompilerServices; -using System.Runtime.InteropServices; -using Android.App; - -// General Information about an assembly is controlled through the following -// set of attributes. Change these attribute values to modify the information -// associated with an assembly. -[assembly: AssemblyTitle("Java.Interop_Tests")] -[assembly: AssemblyDescription("")] -[assembly: AssemblyConfiguration("")] -[assembly: AssemblyCompany("")] -[assembly: AssemblyProduct("Java.Interop_Tests")] -[assembly: AssemblyCopyright("Copyright © 2018")] -[assembly: AssemblyTrademark("")] -[assembly: AssemblyCulture("")] -[assembly: ComVisible(false)] - -// Version information for an assembly consists of the following four values: -// -// Major Version -// Minor Version -// Build Number -// Revision -// -// You can specify all the values or you can default the Build and Revision Numbers -// by using the '*' as shown below: -// [assembly: AssemblyVersion("1.0.*")] -[assembly: AssemblyVersion("1.0.0.0")] -[assembly: AssemblyFileVersion("1.0.0.0")] diff --git a/src/Mono.Android/Test/Java.Interop/JnienvTest.cs b/src/Mono.Android/Test/Java.Interop/JnienvTest.cs index a4f90046cba..d6b54c45c97 100644 --- a/src/Mono.Android/Test/Java.Interop/JnienvTest.cs +++ b/src/Mono.Android/Test/Java.Interop/JnienvTest.cs @@ -1,5 +1,6 @@ using System; using System.Linq; +using System.Runtime.CompilerServices; using System.Runtime.InteropServices; using System.Threading; @@ -373,32 +374,27 @@ public void MoarThreadingTests () Assert.IsNull (ignore_t2, string.Format ("No exception should be thrown [t2]! Got: {0}", ignore_t2)); } - [DllImport ("__Internal", CallingConvention = CallingConvention.Cdecl)] - static extern IntPtr monodroid_typemap_java_to_managed (string java); - [Test] public void JavaToManagedTypeMapping () { - var m = monodroid_typemap_java_to_managed ("android/content/res/Resources"); - Assert.AreNotEqual (IntPtr.Zero, m); - m = monodroid_typemap_java_to_managed ("this/type/does/not/exist"); - Assert.AreEqual (IntPtr.Zero, m); + Type m = Java.Interop.TypeManager.GetJavaToManagedType ("android/content/res/Resources"); + Assert.AreNotEqual (null, m); + m = Java.Interop.TypeManager.GetJavaToManagedType ("this/type/does/not/exist"); + Assert.AreEqual (null, m); } [DllImport ("__Internal", CallingConvention = CallingConvention.Cdecl)] - static extern IntPtr monodroid_typemap_managed_to_java (string java); - - string GetTypeName (Type type) - { - return type.FullName + ", " + type.Assembly.GetName ().Name; - } + static extern IntPtr monodroid_typemap_managed_to_java (byte[] mvid, int token); [Test] public void ManagedToJavaTypeMapping () { - var m = monodroid_typemap_managed_to_java (GetTypeName (typeof (Activity))); + Type type = typeof(Activity); + var m = monodroid_typemap_managed_to_java (type.Module.ModuleVersionId.ToByteArray (), type.MetadataToken); Assert.AreNotEqual (IntPtr.Zero, m, "`Activity` subclasses Java.Lang.Object, it should be in the typemap!"); - m = monodroid_typemap_managed_to_java (GetTypeName (typeof (JnienvTest))); + + type = typeof (JnienvTest); + m = monodroid_typemap_managed_to_java (type.Module.ModuleVersionId.ToByteArray (), type.MetadataToken); Assert.AreEqual (IntPtr.Zero, m, "`JnienvTest` does *not* subclass Java.Lang.Object, it should *not* be in the typemap!"); } diff --git a/src/Mono.Android/Test/Mono.Android-Test.Library/Mono.Android-Test.Library.csproj b/src/Mono.Android/Test/Mono.Android-Test.Library/Mono.Android-Test.Library.csproj index 80a77968eac..98d84f6debf 100644 --- a/src/Mono.Android/Test/Mono.Android-Test.Library/Mono.Android-Test.Library.csproj +++ b/src/Mono.Android/Test/Mono.Android-Test.Library/Mono.Android-Test.Library.csproj @@ -46,7 +46,6 @@ - diff --git a/src/Mono.Android/Test/Mono.Android-Test.Library/Properties/AssemblyInfo.cs b/src/Mono.Android/Test/Mono.Android-Test.Library/Properties/AssemblyInfo.cs deleted file mode 100644 index ea0d06d7073..00000000000 --- a/src/Mono.Android/Test/Mono.Android-Test.Library/Properties/AssemblyInfo.cs +++ /dev/null @@ -1,30 +0,0 @@ -using System.Reflection; -using System.Runtime.CompilerServices; -using System.Runtime.InteropServices; -using Android.App; - -// General Information about an assembly is controlled through the following -// set of attributes. Change these attribute values to modify the information -// associated with an assembly. -[assembly: AssemblyTitle("Mono.Android_Test.Library")] -[assembly: AssemblyDescription("")] -[assembly: AssemblyConfiguration("")] -[assembly: AssemblyCompany("")] -[assembly: AssemblyProduct("Mono.Android_Test.Library")] -[assembly: AssemblyCopyright("Copyright © 2018")] -[assembly: AssemblyTrademark("")] -[assembly: AssemblyCulture("")] -[assembly: ComVisible(false)] - -// Version information for an assembly consists of the following four values: -// -// Major Version -// Minor Version -// Build Number -// Revision -// -// You can specify all the values or you can default the Build and Revision Numbers -// by using the '*' as shown below: -// [assembly: AssemblyVersion("1.0.*")] -[assembly: AssemblyVersion("1.0.0.0")] -[assembly: AssemblyFileVersion("1.0.0.0")] diff --git a/src/Mono.Android/Test/Mono.Android-Tests.csproj b/src/Mono.Android/Test/Mono.Android-Tests.csproj index 96118c34a18..76e6a21f8fc 100644 --- a/src/Mono.Android/Test/Mono.Android-Tests.csproj +++ b/src/Mono.Android/Test/Mono.Android-Tests.csproj @@ -18,6 +18,8 @@ Properties\AndroidManifest.xml armeabi-v7a;x86 False + true + ..\..\..\product.snk v10.0 d8 diff --git a/src/Xamarin.Android.Build.Tasks/MSBuild/Xamarin/Android/Xamarin.Android.Designer.targets b/src/Xamarin.Android.Build.Tasks/MSBuild/Xamarin/Android/Xamarin.Android.Designer.targets index 3f4ce816da6..2a5ee84f13a 100644 --- a/src/Xamarin.Android.Build.Tasks/MSBuild/Xamarin/Android/Xamarin.Android.Designer.targets +++ b/src/Xamarin.Android.Build.Tasks/MSBuild/Xamarin/Android/Xamarin.Android.Designer.targets @@ -56,8 +56,8 @@ Copyright (C) 2016 Xamarin. All rights reserved. Include="@(ResolvedUserAssemblies->'$(MonoAndroidIntermediateAssemblyDir)%(Filename)%(Extension)')" /> <_ResolvedUserMonoAndroidAssembliesForDesigner - Include="@(ResolvedUserAssemblies)" - Condition="'%(ResolvedUserAssemblies.TargetFrameworkIdentifier)' == 'MonoAndroid' Or '%(ResolvedUserAssemblies.HasMonoAndroidReference)' == 'True'" + Include="@(_ResolvedUserAssemblies)" + Condition="'%(_ResolvedUserAssemblies.TargetFrameworkIdentifier)' == 'MonoAndroid' Or '%(_ResolvedUserAssemblies.HasMonoAndroidReference)' == 'True'" /> @@ -86,6 +86,8 @@ Copyright (C) 2016 Xamarin. All rights reserved. AndroidSdkDir="$(_AndroidSdkDirectory)" PackageName="$(_AndroidPackage)" OutputDirectory="$(IntermediateOutputPath)android" + TypemapOutputDirectory="$(_NativeAssemblySourceDir)" + GenerateNativeAssembly="false" MergedAndroidManifestOutput="$(_ManifestOutput)" UseSharedRuntime="$(AndroidUseSharedRuntime)" EmbedAssemblies="$(EmbedAssembliesIntoApk)" @@ -94,7 +96,8 @@ Copyright (C) 2016 Xamarin. All rights reserved. ApplicationJavaClass="$(AndroidApplicationJavaClass)" FrameworkDirectories="$(_XATargetFrameworkDirectories);$(_XATargetFrameworkDirectories)Facades" AcwMapFile="$(_AcwMapFile)" - SupportedAbis="$(_BuildTargetAbis)"> + SupportedAbis="$(_BuildTargetAbis)" + InstantRunEnabled="$(_InstantRunEnabled)"> + EnablePreloadAssembliesDefault="$(_AndroidEnablePreloadAssembliesDefault)" + InstantRunEnabled="$(_InstantRunEnabled)"> diff --git a/src/Xamarin.Android.Build.Tasks/Properties/Resources.Designer.cs b/src/Xamarin.Android.Build.Tasks/Properties/Resources.Designer.cs index 144836d16cd..cfd6318ad96 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/Resources.Designer.cs +++ b/src/Xamarin.Android.Build.Tasks/Properties/Resources.Designer.cs @@ -303,6 +303,14 @@ internal static string XA4305_File_Missing { } } + /// Looks up a localized string similar to Failed to generate type maps. + /// + internal static string XA4308 { + get { + return ResourceManager.GetString("XA4308", resourceCulture); + } + } + /// /// Looks up a localized string similar to Missing Android NDK toolchains directory '{0}'. Please install the Android NDK.. /// diff --git a/src/Xamarin.Android.Build.Tasks/Properties/Resources.resx b/src/Xamarin.Android.Build.Tasks/Properties/Resources.resx index 9c203bb59e4..8c54034b470 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/Resources.resx +++ b/src/Xamarin.Android.Build.Tasks/Properties/Resources.resx @@ -238,6 +238,9 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. {0} - The path of the missing directory @@ -268,4 +271,4 @@ {0} - The managed type name {1} - The exception message of the associated exception - \ No newline at end of file + diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.cs.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.cs.xlf index c81c114f8c2..c748de2638d 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.cs.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.cs.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.de.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.de.xlf index 5f7c6618d2d..570747d4d2f 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.de.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.de.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.es.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.es.xlf index dd6a13df338..dcb7dc84679 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.es.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.es.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.fr.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.fr.xlf index a4b1eadf814..83fad9636cf 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.fr.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.fr.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.it.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.it.xlf index 35986cc9035..4ab899e99d1 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.it.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.it.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.ja.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.ja.xlf index 2e150b819a4..1557560da43 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.ja.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.ja.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.ko.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.ko.xlf index bdc43885ac8..c987644bb4f 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.ko.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.ko.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.pl.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.pl.xlf index 0a7f3f1c707..a2874bbf844 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.pl.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.pl.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.pt-BR.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.pt-BR.xlf index f7251109ee8..f513ad17b34 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.pt-BR.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.pt-BR.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.ru.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.ru.xlf index 1f0229a480d..f7225aa2f42 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.ru.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.ru.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.tr.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.tr.xlf index c5c030af400..5c66871f3b3 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.tr.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.tr.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.zh-Hans.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.zh-Hans.xlf index 01293297d69..2188eb2d0d5 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.zh-Hans.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.zh-Hans.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.zh-Hant.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.zh-Hant.xlf index 6984cbcc842..c4a92aa9f98 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.zh-Hant.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.zh-Hant.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Tasks/GenerateJavaStubs.cs b/src/Xamarin.Android.Build.Tasks/Tasks/GenerateJavaStubs.cs index fcc277569f5..5664b18aaf5 100644 --- a/src/Xamarin.Android.Build.Tasks/Tasks/GenerateJavaStubs.cs +++ b/src/Xamarin.Android.Build.Tasks/Tasks/GenerateJavaStubs.cs @@ -42,6 +42,12 @@ public class GenerateJavaStubs : AndroidTask [Required] public string [] SupportedAbis { get; set; } + [Required] + public string TypemapOutputDirectory { get; set; } + + [Required] + public bool GenerateNativeAssembly { get; set; } + public string ManifestTemplate { get; set; } public string[] MergedManifestDocuments { get; set; } @@ -71,6 +77,9 @@ public class GenerateJavaStubs : AndroidTask public string ApplicationJavaClass { get; set; } + [Output] + public string [] GeneratedBinaryTypeMaps { get; set; } + internal const string AndroidSkipJavaStubGeneration = "AndroidSkipJavaStubGeneration"; public override bool RunTask () @@ -92,8 +101,6 @@ public override bool RunTask () // by ensuring that the target outputs have been deleted. Files.DeleteFile (MergedAndroidManifestOutput, Log); Files.DeleteFile (AcwMapFile, Log); - Files.DeleteFile (Path.Combine (OutputDirectory, "typemap.jm"), Log); - Files.DeleteFile (Path.Combine (OutputDirectory, "typemap.mj"), Log); } return !Log.HasLoggedErrors; @@ -110,73 +117,95 @@ void Run (DirectoryAssemblyResolver res) } // Put every assembly we'll need in the resolver + bool hasExportReference = false; + bool haveMonoAndroid = false; + var allTypemapAssemblies = new HashSet (StringComparer.OrdinalIgnoreCase); + var userAssemblies = new Dictionary (StringComparer.OrdinalIgnoreCase); foreach (var assembly in ResolvedAssemblies) { - if (bool.TryParse (assembly.GetMetadata (AndroidSkipJavaStubGeneration), out bool value) && value) { + bool value; + if (bool.TryParse (assembly.GetMetadata (AndroidSkipJavaStubGeneration), out value) && value) { Log.LogDebugMessage ($"Skipping Java Stub Generation for {assembly.ItemSpec}"); continue; } + + bool addAssembly = false; + string fileName = Path.GetFileName (assembly.ItemSpec); + if (!hasExportReference && String.Compare ("Mono.Android.Export.dll", fileName, StringComparison.OrdinalIgnoreCase) == 0) { + hasExportReference = true; + addAssembly = true; + } else if (!haveMonoAndroid && String.Compare ("Mono.Android.dll", fileName, StringComparison.OrdinalIgnoreCase) == 0) { + haveMonoAndroid = true; + addAssembly = true; + } else if (MonoAndroidHelper.FrameworkAssembliesToTreatAsUserAssemblies.Contains (fileName)) { + if (!bool.TryParse (assembly.GetMetadata (AndroidSkipJavaStubGeneration), out value) || !value) { + string name = Path.GetFileNameWithoutExtension (fileName); + if (!userAssemblies.ContainsKey (name)) + userAssemblies.Add (name, assembly.ItemSpec); + addAssembly = true; + } + } + + if (addAssembly) { + allTypemapAssemblies.Add (assembly.ItemSpec); + } + res.Load (assembly.ItemSpec); } - // However we only want to look for JLO types in user code - List assemblies = new List (); + // However we only want to look for JLO types in user code for Java stub code generation foreach (var asm in ResolvedUserAssemblies) { if (bool.TryParse (asm.GetMetadata (AndroidSkipJavaStubGeneration), out bool value) && value) { Log.LogDebugMessage ($"Skipping Java Stub Generation for {asm.ItemSpec}"); continue; } - if (!assemblies.All (x => Path.GetFileName (x) != Path.GetFileName (asm.ItemSpec))) - continue; - Log.LogDebugMessage ($"Adding {asm.ItemSpec} to assemblies."); - assemblies.Add (asm.ItemSpec); - } - foreach (var asm in MonoAndroidHelper.GetFrameworkAssembliesToTreatAsUserAssemblies (ResolvedAssemblies)) { - if (bool.TryParse (asm.GetMetadata (AndroidSkipJavaStubGeneration), out bool value) && value) { - Log.LogDebugMessage ($"Skipping Java Stub Generation for {asm.ItemSpec}"); - continue; - } - if (!assemblies.All (x => Path.GetFileName (x) != Path.GetFileName (asm.ItemSpec))) - continue; - Log.LogDebugMessage ($"Adding {asm.ItemSpec} to assemblies."); - assemblies.Add (asm.ItemSpec); + allTypemapAssemblies.Add (asm.ItemSpec); + userAssemblies.Add (Path.GetFileNameWithoutExtension (asm.ItemSpec), asm.ItemSpec); } // Step 1 - Find all the JLO types var scanner = new JavaTypeScanner (this.CreateTaskLogger ()) { ErrorOnCustomJavaObject = ErrorOnCustomJavaObject, }; - var all_java_types = scanner.GetJavaTypes (assemblies, res); - WriteTypeMappings (all_java_types); + List allJavaTypes = scanner.GetJavaTypes (allTypemapAssemblies, res); - var java_types = all_java_types - .Where (t => !JavaTypeScanner.ShouldSkipJavaCallableWrapperGeneration (t)) - .ToArray (); + // Step 2 - Generate type maps + // Type mappings need to use all the assemblies, always. + WriteTypeMappings (res, allJavaTypes); - // Step 2 - Generate Java stub code + var javaTypes = new List (); + foreach (TypeDefinition td in allJavaTypes) { + if (!userAssemblies.ContainsKey (td.Module.Assembly.Name.Name) || JavaTypeScanner.ShouldSkipJavaCallableWrapperGeneration (td)) { + continue; + } + + javaTypes.Add (td); + } + + // Step 3 - Generate Java stub code var success = Generator.CreateJavaSources ( Log, - java_types, + javaTypes, Path.Combine (OutputDirectory, "src"), ApplicationJavaClass, AndroidSdkPlatform, UseSharedRuntime, int.Parse (AndroidSdkPlatform) <= 10, - ResolvedAssemblies.Any (assembly => Path.GetFileName (assembly.ItemSpec) == "Mono.Android.Export.dll")); + hasExportReference); if (!success) return; // We need to save a map of .NET type -> ACW type for resource file fixups - var managed = new Dictionary (java_types.Length, StringComparer.Ordinal); - var java = new Dictionary (java_types.Length, StringComparer.Ordinal); + var managed = new Dictionary (javaTypes.Count, StringComparer.Ordinal); + var java = new Dictionary (javaTypes.Count, StringComparer.Ordinal); var managedConflicts = new Dictionary> (0, StringComparer.Ordinal); var javaConflicts = new Dictionary> (0, StringComparer.Ordinal); // Allocate a MemoryStream with a reasonable guess at its capacity - using (var stream = new MemoryStream (java_types.Length * 32)) + using (var stream = new MemoryStream (javaTypes.Count * 32)) using (var acw_map = new StreamWriter (stream)) { - foreach (var type in java_types) { + foreach (TypeDefinition type in javaTypes) { string managedKey = type.FullName.Replace ('/', '.'); string javaKey = JavaNativeTypeManager.ToJniName (type).Replace ('/', '.'); @@ -241,7 +270,7 @@ void Run (DirectoryAssemblyResolver res) manifest.PackageName = PackageName; manifest.ApplicationName = ApplicationName ?? PackageName; manifest.Placeholders = ManifestPlaceholders; - manifest.Assemblies.AddRange (assemblies); + manifest.Assemblies.AddRange (userAssemblies.Values); manifest.Resolver = res; manifest.SdkDir = AndroidSdkDir; manifest.SdkVersion = AndroidSdkPlatform; @@ -250,7 +279,7 @@ void Run (DirectoryAssemblyResolver res) manifest.NeedsInternet = NeedsInternet; manifest.InstantRunEnabled = InstantRunEnabled; - var additionalProviders = manifest.Merge (Log, all_java_types, ApplicationJavaClass, EmbedAssemblies, BundledWearApplicationName, MergedManifestDocuments); + var additionalProviders = manifest.Merge (Log, allJavaTypes, ApplicationJavaClass, EmbedAssemblies, BundledWearApplicationName, MergedManifestDocuments); using (var stream = new MemoryStream ()) { manifest.Save (Log, stream); @@ -272,7 +301,7 @@ void Run (DirectoryAssemblyResolver res) // Create additional application java sources. StringWriter regCallsWriter = new StringWriter (); regCallsWriter.WriteLine ("\t\t// Application and Instrumentation ACWs must be registered first."); - foreach (var type in java_types) { + foreach (var type in javaTypes) { if (JavaNativeTypeManager.IsApplication (type) || JavaNativeTypeManager.IsInstrumentation (type)) { string javaKey = JavaNativeTypeManager.ToJniName (type).Replace ('/', '.'); regCallsWriter.WriteLine ("\t\tmono.android.Runtime.register (\"{0}\", {1}.class, {1}.__md_methods);", @@ -301,75 +330,12 @@ void SaveResource (string resource, string filename, string destDir, Func types) - { - void logger (TraceLevel level, string value) => Log.LogDebugMessage (value); - TypeNameMapGenerator createTypeMapGenerator () => UseSharedRuntime ? - new TypeNameMapGenerator (types, logger) : - new TypeNameMapGenerator (ResolvedAssemblies.Select (p => p.ItemSpec), logger); - using (var gen = createTypeMapGenerator ()) { - using (var ms = new MemoryStream ()) { - UpdateWhenChanged (Path.Combine (OutputDirectory, "typemap.jm"), "jm", ms, gen.WriteJavaToManaged); - UpdateWhenChanged (Path.Combine (OutputDirectory, "typemap.mj"), "mj", ms, gen.WriteManagedToJava); - } - } - } - - void UpdateWhenChanged (string path, string type, MemoryStream ms, Action generator) + void WriteTypeMappings (DirectoryAssemblyResolver resolver, List types) { - if (!EmbedAssemblies) { - ms.SetLength (0); - generator (ms); - MonoAndroidHelper.CopyIfStreamChanged (ms, path); - } - - string dataFilePath = $"{path}.inc"; - using (var stream = new NativeAssemblyDataStream ()) { - if (EmbedAssemblies) { - generator (stream); - stream.EndOfFile (); - MonoAndroidHelper.CopyIfStreamChanged (stream, dataFilePath); - } else { - stream.EmptyFile (); - } - - var generatedFiles = new List (); - string mappingFieldName = $"{type}_typemap"; - string dataFileName = Path.GetFileName (dataFilePath); - NativeAssemblerTargetProvider asmTargetProvider; - var utf8Encoding = new UTF8Encoding (false); - foreach (string abi in SupportedAbis) { - ms.SetLength (0); - switch (abi.Trim ()) { - case "armeabi-v7a": - asmTargetProvider = new ARMNativeAssemblerTargetProvider (is64Bit: false); - break; - - case "arm64-v8a": - asmTargetProvider = new ARMNativeAssemblerTargetProvider (is64Bit: true); - break; - - case "x86": - asmTargetProvider = new X86NativeAssemblerTargetProvider (is64Bit: false); - break; - - case "x86_64": - asmTargetProvider = new X86NativeAssemblerTargetProvider (is64Bit: true); - break; - - default: - throw new InvalidOperationException ($"Unknown ABI {abi}"); - } - - var asmgen = new TypeMappingNativeAssemblyGenerator (asmTargetProvider, stream, dataFileName, stream.MapByteCount, mappingFieldName); - asmgen.EmbedAssemblies = EmbedAssemblies; - string asmFileName = $"{path}.{abi.Trim ()}.s"; - using (var sw = new StreamWriter (ms, utf8Encoding, bufferSize: 8192, leaveOpen: true)) { - asmgen.Write (sw, dataFileName); - MonoAndroidHelper.CopyIfStreamChanged (ms, asmFileName); - } - } - } + var tmg = new TypeMapGenerator ((string message) => Log.LogDebugMessage (message), SupportedAbis); + if (!tmg.Generate (types, TypemapOutputDirectory, GenerateNativeAssembly)) + throw new XamarinAndroidException (4308, Properties.Resources.XA4308); + GeneratedBinaryTypeMaps = tmg.GeneratedBinaryTypeMaps.ToArray (); } } } diff --git a/src/Xamarin.Android.Build.Tasks/Tasks/GeneratePackageManagerJava.cs b/src/Xamarin.Android.Build.Tasks/Tasks/GeneratePackageManagerJava.cs index 45a59759110..2764f083cc2 100644 --- a/src/Xamarin.Android.Build.Tasks/Tasks/GeneratePackageManagerJava.cs +++ b/src/Xamarin.Android.Build.Tasks/Tasks/GeneratePackageManagerJava.cs @@ -58,6 +58,9 @@ public class GeneratePackageManagerJava : AndroidTask [Required] public bool EnablePreloadAssembliesDefault { get; set; } + [Required] + public bool InstantRunEnabled { get; set; } + public string BoundExceptionType { get; set; } public string PackageNamingPolicy { get; set; } @@ -252,7 +255,8 @@ void AddEnvironment () foreach (string abi in SupportedAbis) { ms.SetLength (0); NativeAssemblerTargetProvider asmTargetProvider; - string asmFileName = Path.Combine (EnvironmentOutputDirectory, $"environment.{abi.ToLowerInvariant ()}.s"); + string baseAsmFilePath = Path.Combine (EnvironmentOutputDirectory, $"environment.{abi.ToLowerInvariant ()}"); + string asmFilePath = $"{baseAsmFilePath}.s"; switch (abi.Trim ()) { case "armeabi-v7a": asmTargetProvider = new ARMNativeAssemblerTargetProvider (false); @@ -274,7 +278,7 @@ void AddEnvironment () throw new InvalidOperationException ($"Unknown ABI {abi}"); } - var asmgen = new ApplicationConfigNativeAssemblyGenerator (asmTargetProvider, environmentVariables, systemProperties) { + var asmgen = new ApplicationConfigNativeAssemblyGenerator (asmTargetProvider, baseAsmFilePath, environmentVariables, systemProperties) { IsBundledApp = IsBundledApplication, UsesMonoAOT = usesMonoAOT, UsesMonoLLVM = EnableLLVM, @@ -284,11 +288,12 @@ void AddEnvironment () BrokenExceptionTransitions = brokenExceptionTransitions, PackageNamingPolicy = pnp, BoundExceptionType = boundExceptionType, + InstantRunEnabled = InstantRunEnabled, }; using (var sw = new StreamWriter (ms, utf8Encoding, bufferSize: 8192, leaveOpen: true)) { - asmgen.Write (sw, asmFileName); - MonoAndroidHelper.CopyIfStreamChanged (ms, asmFileName); + asmgen.Write (sw); + MonoAndroidHelper.CopyIfStreamChanged (ms, asmFilePath); } } diff --git a/src/Xamarin.Android.Build.Tasks/Tasks/PrepareAbiItems.cs b/src/Xamarin.Android.Build.Tasks/Tasks/PrepareAbiItems.cs index 00517cdbc0b..d298dfd4ce0 100644 --- a/src/Xamarin.Android.Build.Tasks/Tasks/PrepareAbiItems.cs +++ b/src/Xamarin.Android.Build.Tasks/Tasks/PrepareAbiItems.cs @@ -1,4 +1,5 @@ using System; +using System.IO; using System.Collections.Generic; using Microsoft.Build.Framework; @@ -8,27 +9,68 @@ namespace Xamarin.Android.Tasks { public class PrepareAbiItems : AndroidTask { + const string ArmV7a = "armeabi-v7a"; + const string TypeMapBase = "typemaps"; + const string EnvBase = "environment"; + public override string TaskPrefix => "PAI"; [Required] public string [] BuildTargetAbis { get; set; } [Required] - public string ItemNamePattern { get; set; } + public string NativeSourcesDir { get; set; } + + [Required] + public bool TypeMapMode { get; set; } [Output] - public ITaskItem[] OutputItems { get; set; } + public ITaskItem[] AssemblySources { get; set; } + + [Output] + public ITaskItem[] AssemblyIncludes { get; set; } public override bool RunTask () { - var items = new List (); + var sources = new List (); + var includes = new List (); + bool haveSharedSource = false; + bool haveArmV7SharedSource = false; + string baseName = TypeMapMode ? TypeMapBase : EnvBase; + TaskItem item; foreach (string abi in BuildTargetAbis) { - var item = new TaskItem (ItemNamePattern.Replace ("@abi@", abi)); + if (TypeMapMode) { + if (String.Compare (ArmV7a, abi, StringComparison.Ordinal) == 0) + haveArmV7SharedSource = true; + else + haveSharedSource = true; + } + + item = new TaskItem (Path.Combine (NativeSourcesDir, $"{baseName}.{abi}.s")); item.SetMetadata ("abi", abi); - items.Add (item); + sources.Add (item); + + if (!TypeMapMode) + continue; + + item = new TaskItem (Path.Combine (NativeSourcesDir, $"{baseName}.{abi}-managed.inc")); + item.SetMetadata ("abi", abi); + includes.Add (item); + } + + if (haveArmV7SharedSource) { + item = new TaskItem (Path.Combine (NativeSourcesDir, $"{baseName}.{ArmV7a}-shared.inc")); + item.SetMetadata ("abi", ArmV7a); + includes.Add (item); + } + + if (haveSharedSource) { + item = new TaskItem (Path.Combine (NativeSourcesDir, $"{baseName}.shared.inc")); + includes.Add (item); } - OutputItems = items.ToArray (); + AssemblySources = sources.ToArray (); + AssemblyIncludes = includes.ToArray (); return !Log.HasLoggedErrors; } } diff --git a/src/Xamarin.Android.Build.Tasks/Tasks/RemoveRegisterAttribute.cs b/src/Xamarin.Android.Build.Tasks/Tasks/RemoveRegisterAttribute.cs index e2883f77e7c..4163db2e921 100644 --- a/src/Xamarin.Android.Build.Tasks/Tasks/RemoveRegisterAttribute.cs +++ b/src/Xamarin.Android.Build.Tasks/Tasks/RemoveRegisterAttribute.cs @@ -19,21 +19,17 @@ public class RemoveRegisterAttribute : AndroidTask [Required] public ITaskItem[] ShrunkFrameworkAssemblies { get; set; } - public bool Deterministic { get; set; } - public override bool RunTask () { // Find Mono.Android.dll var mono_android = ShrunkFrameworkAssemblies.First (f => Path.GetFileNameWithoutExtension (f.ItemSpec) == "Mono.Android").ItemSpec; - var writerParameters = new WriterParameters { - DeterministicMvid = Deterministic, - }; + using (var assembly = AssemblyDefinition.ReadAssembly (mono_android, new ReaderParameters { ReadWrite = true })) { // Strip out [Register] attributes foreach (TypeDefinition type in assembly.MainModule.Types) ProcessType (type); - assembly.Write (writerParameters); + assembly.Write (); } return true; diff --git a/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/IncrementalBuildTest.cs b/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/IncrementalBuildTest.cs index cc221f64c11..d790cfbf28e 100644 --- a/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/IncrementalBuildTest.cs +++ b/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/IncrementalBuildTest.cs @@ -805,10 +805,10 @@ public void GenerateJavaStubsAndAssembly ([Values (true, false)] bool isRelease) readonly string [] ExpectedAssemblyFiles = new [] { Path.Combine ("android", "environment.armeabi-v7a.o"), Path.Combine ("android", "environment.armeabi-v7a.s"), - Path.Combine ("android", "typemap.mj.armeabi-v7a.o"), - Path.Combine ("android", "typemap.mj.armeabi-v7a.s"), - Path.Combine ("android", "typemap.jm.armeabi-v7a.o"), - Path.Combine ("android", "typemap.jm.armeabi-v7a.s"), + Path.Combine ("android", "typemaps.armeabi-v7a-managed.inc"), + Path.Combine ("android", "typemaps.armeabi-v7a-shared.inc"), + Path.Combine ("android", "typemaps.armeabi-v7a.o"), + Path.Combine ("android", "typemaps.armeabi-v7a.s"), Path.Combine ("app_shared_libraries", "armeabi-v7a", "libxamarin-app.so") }; diff --git a/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Utilities/EnvironmentHelper.cs b/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Utilities/EnvironmentHelper.cs index d253dc90264..a81b9aefbe7 100644 --- a/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Utilities/EnvironmentHelper.cs +++ b/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Utilities/EnvironmentHelper.cs @@ -22,17 +22,18 @@ public sealed class ApplicationConfig public bool uses_assembly_preload; public bool is_a_bundled_app; public bool broken_exception_transitions; + public bool instant_run_enabled; public byte bound_stream_io_exception_type; public uint package_naming_policy; public uint environment_variable_count; public uint system_property_count; public string android_package_name; }; - const uint ApplicationConfigFieldCount = 10; + const uint ApplicationConfigFieldCount = 11; static readonly object ndkInitLock = new object (); static readonly char[] readElfFieldSeparator = new [] { ' ', '\t' }; - static readonly Regex stringLabelRegex = new Regex ("^\\.L\\.str\\.[0-9]+:", RegexOptions.Compiled); + static readonly Regex stringLabelRegex = new Regex ("^\\.L\\.env\\.str\\.[0-9]+:", RegexOptions.Compiled); static readonly HashSet expectedPointerTypes = new HashSet (StringComparer.Ordinal) { ".long", @@ -49,10 +50,11 @@ public sealed class ApplicationConfig "app_environment_variables", "app_system_properties", "application_config", - "jm_typemap", - "jm_typemap_header", - "mj_typemap", - "mj_typemap_header", + "map_modules", + "map_module_count", + "java_type_count", + "java_name_width", + "map_java", "mono_aot_mode_name", }; @@ -136,27 +138,32 @@ static ApplicationConfig ReadApplicationConfig (string envFile) ret.broken_exception_transitions = ConvertFieldToBool ("broken_exception_transitions", envFile, i, field [1]); break; - case 5: // bound_stream_io_exception_type: byte / .byte + case 5: // instant_run_enabled: bool / .byte + AssertFieldType (envFile, ".byte", field [0], i); + ret.instant_run_enabled = ConvertFieldToBool ("instant_run_enabled", envFile, i, field [1]); + break; + + case 6: // bound_stream_io_exception_type: byte / .byte AssertFieldType (envFile, ".byte", field [0], i); ret.bound_stream_io_exception_type = ConvertFieldToByte ("bound_stream_io_exception_type", envFile, i, field [1]); break; - case 6: // package_naming_policy: uint32_t / .word | .long + case 7: // package_naming_policy: uint32_t / .word | .long Assert.IsTrue (expectedUInt32Types.Contains (field [0]), $"Unexpected uint32_t field type in '{envFile}:{i}': {field [0]}"); ret.package_naming_policy = ConvertFieldToUInt32 ("package_naming_policy", envFile, i, field [1]); break; - case 7: // environment_variable_count: uint32_t / .word | .long + case 8: // environment_variable_count: uint32_t / .word | .long Assert.IsTrue (expectedUInt32Types.Contains (field [0]), $"Unexpected uint32_t field type in '{envFile}:{i}': {field [0]}"); ret.environment_variable_count = ConvertFieldToUInt32 ("environment_variable_count", envFile, i, field [1]); break; - case 8: // system_property_count: uint32_t / .word | .long + case 9: // system_property_count: uint32_t / .word | .long Assert.IsTrue (expectedUInt32Types.Contains (field [0]), $"Unexpected uint32_t field type in '{envFile}:{i}': {field [0]}"); ret.system_property_count = ConvertFieldToUInt32 ("system_property_count", envFile, i, field [1]); break; - case 9: // android_package_name: string / [pointer type] + case 10: // android_package_name: string / [pointer type] Assert.IsTrue (expectedPointerTypes.Contains (field [0]), $"Unexpected pointer field type in '{envFile}:{i}': {field [0]}"); pointers.Add (field [1].Trim ()); break; diff --git a/src/Xamarin.Android.Build.Tasks/Utilities/ARMNativeAssemblerTargetProvider.cs b/src/Xamarin.Android.Build.Tasks/Utilities/ARMNativeAssemblerTargetProvider.cs index 2791da006a0..e563f64ffe6 100644 --- a/src/Xamarin.Android.Build.Tasks/Utilities/ARMNativeAssemblerTargetProvider.cs +++ b/src/Xamarin.Android.Build.Tasks/Utilities/ARMNativeAssemblerTargetProvider.cs @@ -5,9 +5,15 @@ namespace Xamarin.Android.Tasks { class ARMNativeAssemblerTargetProvider : NativeAssemblerTargetProvider { + const string ARMV7a = "armeabi-v7a"; + const string ARMV8a = "arm64-v8a"; + public override bool Is64Bit { get; } public override string PointerFieldType { get; } public override string TypePrefix { get; } + public override string AbiName => Is64Bit ? ARMV8a : ARMV7a; + public override uint MapModulesAlignBits => Is64Bit ? 3u : 2u; + public override uint MapJavaAlignBits { get; } = 2; public ARMNativeAssemblerTargetProvider (bool is64Bit) { diff --git a/src/Xamarin.Android.Build.Tasks/Utilities/ApplicationConfigNativeAssemblyGenerator.cs b/src/Xamarin.Android.Build.Tasks/Utilities/ApplicationConfigNativeAssemblyGenerator.cs index 456db8704ba..1c5b70cf0f2 100644 --- a/src/Xamarin.Android.Build.Tasks/Utilities/ApplicationConfigNativeAssemblyGenerator.cs +++ b/src/Xamarin.Android.Build.Tasks/Utilities/ApplicationConfigNativeAssemblyGenerator.cs @@ -20,11 +20,12 @@ class ApplicationConfigNativeAssemblyGenerator : NativeAssemblyGenerator public string AndroidPackageName { get; set; } public bool BrokenExceptionTransitions { get; set; } public global::Android.Runtime.BoundExceptionType BoundExceptionType { get; set; } + public bool InstantRunEnabled { get; set; } public PackageNamingPolicy PackageNamingPolicy { get; set; } - public ApplicationConfigNativeAssemblyGenerator (NativeAssemblerTargetProvider targetProvider, IDictionary environmentVariables, IDictionary systemProperties) - : base (targetProvider) + public ApplicationConfigNativeAssemblyGenerator (NativeAssemblerTargetProvider targetProvider, string baseFileName, IDictionary environmentVariables, IDictionary systemProperties) + : base (targetProvider, baseFileName) { if (environmentVariables != null) this.environmentVariables = new SortedDictionary (environmentVariables, StringComparer.Ordinal); @@ -44,7 +45,7 @@ protected override void WriteSymbols (StreamWriter output) WriteData (output, AndroidPackageName, stringLabel); WriteDataSection (output, "application_config"); - WriteSymbol (output, "application_config", TargetProvider.GetStructureAlignment (true), fieldAlignBytes: 4, isGlobal: true, alwaysWriteSize: true, structureWriter: () => { + WriteSymbol (output, "application_config", TargetProvider.GetStructureAlignment (true), packed: false, isGlobal: true, alwaysWriteSize: true, structureWriter: () => { // Order of fields and their type must correspond *exactly* to that in // src/monodroid/jni/xamarin-app.h ApplicationConfig structure WriteCommentLine (output, "uses_mono_llvm"); @@ -62,6 +63,9 @@ protected override void WriteSymbols (StreamWriter output) WriteCommentLine (output, "broken_exception_transitions"); size += WriteData (output, BrokenExceptionTransitions); + WriteCommentLine (output, "instant_run_enabled"); + size += WriteData (output, InstantRunEnabled); + WriteCommentLine (output, "bound_exception_type"); size += WriteData (output, (byte)BoundExceptionType); @@ -74,13 +78,8 @@ protected override void WriteSymbols (StreamWriter output) WriteCommentLine (output, "system_property_count"); size += WriteData (output, systemProperties == null ? 0 : systemProperties.Count * 2); - // After uses_embedded_dsos was removed, we need padding on 64-bit - if (TargetProvider.Is64Bit) { - size += WriteDataPadding (output, 4); - } - WriteCommentLine (output, "android_package_name"); - size += WritePointer (output, stringLabel); + size += WritePointer (output, MakeLocalLabel (stringLabel)); return size; }); @@ -88,7 +87,7 @@ protected override void WriteSymbols (StreamWriter output) stringLabel = GetStringLabel (); WriteData (output, MonoAOTMode ?? String.Empty, stringLabel); WriteDataSection (output, "mono_aot_mode_name"); - WritePointer (output, stringLabel, "mono_aot_mode_name", isGlobal: true); + WritePointer (output, MakeLocalLabel (stringLabel), "mono_aot_mode_name", isGlobal: true); WriteNameValueStringArray (output, "app_environment_variables", environmentVariables); WriteNameValueStringArray (output, "app_system_properties", systemProperties); @@ -98,7 +97,7 @@ void WriteNameValueStringArray (StreamWriter output, string label, SortedDiction { if (entries == null || entries.Count == 0) { WriteDataSection (output, label); - WriteSymbol (output, label, TargetProvider.GetStructureAlignment (true), fieldAlignBytes: 4, isGlobal: true, alwaysWriteSize: true, structureWriter: null); + WriteSymbol (output, label, TargetProvider.GetStructureAlignment (true), packed: false, isGlobal: true, alwaysWriteSize: true, structureWriter: null); return; } @@ -117,11 +116,11 @@ void WriteNameValueStringArray (StreamWriter output, string label, SortedDiction } WriteDataSection (output, label); - WriteSymbol (output, label, TargetProvider.GetStructureAlignment (true), fieldAlignBytes: 4, isGlobal: true, alwaysWriteSize: true, structureWriter: () => { + WriteSymbol (output, label, TargetProvider.GetStructureAlignment (true), packed: false, isGlobal: true, alwaysWriteSize: true, structureWriter: () => { uint size = 0; foreach (string l in entry_labels) { - size += WritePointer (output, l); + size += WritePointer (output, MakeLocalLabel (l)); } return size; @@ -136,7 +135,7 @@ void WriteDataSection (StreamWriter output, string tag) string GetStringLabel () { stringCounter++; - return $".L.str.{stringCounter}"; + return $"env.str.{stringCounter}"; } }; } diff --git a/src/Xamarin.Android.Build.Tasks/Utilities/MonoAndroidHelper.cs b/src/Xamarin.Android.Build.Tasks/Utilities/MonoAndroidHelper.cs index a5f778d5af4..0cc1af3b8d8 100644 --- a/src/Xamarin.Android.Build.Tasks/Utilities/MonoAndroidHelper.cs +++ b/src/Xamarin.Android.Build.Tasks/Utilities/MonoAndroidHelper.cs @@ -307,7 +307,7 @@ public static bool IsFrameworkAssembly (string assembly, bool checkSdkPath) { if (IsSharedRuntimeAssembly (assembly)) { #if MSBUILD - bool treatAsUser = Array.BinarySearch (FrameworkAssembliesToTreatAsUserAssemblies, Path.GetFileName (assembly), StringComparer.OrdinalIgnoreCase) >= 0; + bool treatAsUser = FrameworkAssembliesToTreatAsUserAssemblies.Contains (Path.GetFileName (assembly)); // Framework assemblies don't come from outside the SDK Path; // user assemblies do if (checkSdkPath && treatAsUser && TargetFrameworkDirectories != null) { @@ -527,10 +527,14 @@ public static bool IsRawResourcePath (string projectPath) #if MSBUILD internal static IEnumerable GetFrameworkAssembliesToTreatAsUserAssemblies (ITaskItem[] resolvedAssemblies) - { - return resolvedAssemblies - .Where (f => Array.BinarySearch (FrameworkAssembliesToTreatAsUserAssemblies, Path.GetFileName (f.ItemSpec), StringComparer.OrdinalIgnoreCase) >= 0) - .Select(p => p); + { + var ret = new List (); + foreach (ITaskItem item in resolvedAssemblies) { + if (FrameworkAssembliesToTreatAsUserAssemblies.Contains (Path.GetFileName (item.ItemSpec))) + ret.Add (item); + } + + return ret; } #endif @@ -543,8 +547,7 @@ internal static IEnumerable GetFrameworkAssembliesToTreatAsUserAssemb "Mono.Data.Sqlite.dll", "Mono.Posix.dll", }; - // MUST BE SORTED CASE-INSENSITIVE - internal static readonly string[] FrameworkAssembliesToTreatAsUserAssemblies = { + internal static readonly HashSet FrameworkAssembliesToTreatAsUserAssemblies = new HashSet (StringComparer.OrdinalIgnoreCase) { "Mono.Android.Support.v13.dll", "Mono.Android.Support.v4.dll", "Xamarin.Android.NUnitLite.dll", diff --git a/src/Xamarin.Android.Build.Tasks/Utilities/NativeAssemblerTargetProvider.cs b/src/Xamarin.Android.Build.Tasks/Utilities/NativeAssemblerTargetProvider.cs index d3158abf8fc..6b379232078 100644 --- a/src/Xamarin.Android.Build.Tasks/Utilities/NativeAssemblerTargetProvider.cs +++ b/src/Xamarin.Android.Build.Tasks/Utilities/NativeAssemblerTargetProvider.cs @@ -8,6 +8,9 @@ abstract class NativeAssemblerTargetProvider public abstract bool Is64Bit { get; } public abstract string PointerFieldType { get; } public abstract string TypePrefix { get; } + public abstract string AbiName { get; } + public abstract uint MapModulesAlignBits { get; } + public abstract uint MapJavaAlignBits { get; } public virtual string MapType () { @@ -30,22 +33,27 @@ abstract class NativeAssemblerTargetProvider public virtual uint GetTypeSize () { - if (typeof(T) == typeof(byte)) + return GetTypeSize (typeof(T)); + } + + public virtual uint GetTypeSize (Type type) + { + if (type == typeof(byte)) return 1u; - if (typeof(T) == typeof(bool)) + if (type == typeof(bool)) return 1u; - if (typeof(T) == typeof(string)) + if (type == typeof(string)) return GetPointerSize(); - if (typeof(T) == typeof(Int32) || typeof(T) == typeof(UInt32)) + if (type == typeof(Int32) || type == typeof(UInt32)) return 4u; - if (typeof(T) == typeof(Int64) || typeof(T) == typeof(UInt64)) + if (type == typeof(Int64) || type == typeof(UInt64)) return 8u; - throw new InvalidOperationException ($"Unable to map managed type {typeof(T)} to native assembly type"); + throw new InvalidOperationException ($"Unable to map managed type {type} to native assembly type"); } public virtual uint GetStructureAlignment (bool hasPointers) diff --git a/src/Xamarin.Android.Build.Tasks/Utilities/NativeAssemblyDataStream.cs b/src/Xamarin.Android.Build.Tasks/Utilities/NativeAssemblyDataStream.cs deleted file mode 100644 index f0e3e6526a4..00000000000 --- a/src/Xamarin.Android.Build.Tasks/Utilities/NativeAssemblyDataStream.cs +++ /dev/null @@ -1,235 +0,0 @@ -using System; -using System.Collections.Generic; -using System.Security.Cryptography; -using System.IO; -using System.Text; -using Java.Interop.Tools.JavaCallableWrappers; - -namespace Xamarin.Android.Tasks -{ - class NativeAssemblyDataStream : Stream - { - const uint MapVersionFound = 0x0001; - const uint MapEntryCountFound = 0x0002; - const uint MapEntryLengthFound = 0x0004; - const uint MapValueOffsetFound = 0x0008; - const uint MapEverythingFound = MapVersionFound | MapEntryCountFound | MapEntryLengthFound | MapValueOffsetFound; - - const string MapFieldVersion = "version"; - const string MapFieldEntryCount = "entry-count"; - const string MapFieldEntryLength = "entry-len"; - const string MapFieldValueOffset = "value-offset"; - - const uint MaxFieldsInRow = 32; - - MemoryStream outputStream; - StreamWriter outputWriter; - bool firstWrite = true; - uint rowFieldCounter; - HashAlgorithm hashAlgorithm; - bool hashingFinished; - uint byteCount = 0; - - public int MapVersion { get; set; } = -1; - public int MapEntryCount { get; set; } = -1; - public int MapEntryLength { get; set; } = -1; - public int MapValueOffset { get; set; } = -1; - public uint MapByteCount => byteCount; - - public override bool CanRead => outputStream.CanRead; - public override bool CanSeek => outputStream.CanSeek; - public override bool CanWrite => outputStream.CanWrite; - public override long Length => outputStream.Length; - - public override long Position { - get => outputStream.Position; - set => outputStream.Position = value; - } - - public NativeAssemblyDataStream () - { - outputStream = new MemoryStream (); - outputWriter = new StreamWriter (outputStream, new UTF8Encoding (false)); - hashAlgorithm = new Crc64 (); - hashAlgorithm.Initialize (); - } - - public byte[] GetStreamHash () - { - if (!hashingFinished) { - hashAlgorithm.TransformFinalBlock (new byte[0], 0, 0); - hashingFinished = true; - } - - return hashAlgorithm.Hash; - } - - protected override void Dispose (bool disposing) - { - base.Dispose (disposing); - if (!disposing) - return; - - outputWriter.Dispose (); - outputWriter = null; - outputStream = null; - } - - public void EndOfFile () - { - outputWriter.WriteLine (); - Flush (); - } - - public void EmptyFile () - { - MapVersion = 1; - MapEntryCount = 0; - MapEntryLength = 0; - MapValueOffset = 0; - } - - public override void Flush () - { - outputWriter.Flush (); - outputStream.Flush (); - } - - public override int Read (byte[] buffer, int offset, int count) - { - return outputStream.Read (buffer, offset, count); - } - - public override long Seek (long offset, SeekOrigin origin) - { - return outputStream.Seek (offset, origin); - } - - public override void SetLength (long value) - { - outputStream.SetLength (value); - } - - public override void Write (byte[] buffer, int offset, int count) - { - if (buffer == null) - throw new ArgumentNullException (nameof (buffer)); - if (offset < 0) - throw new ArgumentOutOfRangeException (nameof (offset)); - if (count < 0) - throw new ArgumentOutOfRangeException (nameof (count)); - if (offset + count > buffer.Length) - throw new ArgumentException ($"The sum of the '{nameof (offset)}' and '{nameof (count)}' arguments is greater than the buffer length"); - if (outputWriter == null) - throw new ObjectDisposedException (this.GetType ().Name); - - if (!hashingFinished) - hashAlgorithm.TransformBlock (buffer, offset, count, null, 0); - - if (firstWrite) { - // Kind of a backward thing to do, but I wanted to avoid having to modfy (and bump the - // submodule of) Java.Interop. This should be Safe Enough™ (until it's not) - string line = Encoding.UTF8.GetString (buffer, offset, count); - - // Format used by Java.Interop.Tools.JavaCallableWrappers.TypeNameMapGenerator: - // - // "version=1\u0000entry-count={0}\u0000entry-len={1}\u0000value-offset={2}\u0000" - // - string[] parts = line.Split ('\u0000'); - if (parts.Length != 5) - HeaderFormatError ("invalid number of fields"); - - // Let's be just slightly flexible :P - uint foundMask = 0; - foreach (string p in parts) { - string field = p.Trim (); - if (String.IsNullOrEmpty (field)) - continue; - - string[] fieldParts = field.Split(new char[]{'='}, 2); - if (fieldParts.Length != 2) - HeaderFormatError ($"invalid field format '{field}'"); - - switch (fieldParts [0]) { - case MapFieldVersion: - foundMask |= MapVersionFound; - MapVersion = GetHeaderInteger (fieldParts[0], fieldParts [1]); - break; - - case MapFieldEntryCount: - foundMask |= MapEntryCountFound; - MapEntryCount = GetHeaderInteger (fieldParts[0], fieldParts [1]); - break; - - case MapFieldEntryLength: - foundMask |= MapEntryLengthFound; - MapEntryLength = GetHeaderInteger (fieldParts[0], fieldParts [1]); - break; - - case MapFieldValueOffset: - foundMask |= MapValueOffsetFound; - MapValueOffset = GetHeaderInteger (fieldParts[0], fieldParts [1]); - break; - - default: - HeaderFormatError ($"unknown field '{fieldParts [0]}'"); - break; - } - } - - if (foundMask != MapEverythingFound) { - var missingFields = new List (); - if ((foundMask & MapVersionFound) == 0) - missingFields.Add (MapFieldVersion); - if ((foundMask & MapEntryCountFound) == 0) - missingFields.Add (MapFieldEntryCount); - if ((foundMask & MapEntryLengthFound) == 0) - missingFields.Add (MapFieldEntryLength); - if ((foundMask & MapValueOffsetFound) == 0) - missingFields.Add (MapFieldValueOffset); - - var sb = new StringBuilder ("missing header field"); - if (missingFields.Count > 1) - sb.Append ('s'); - sb.Append (": "); - sb.Append (String.Join (", ", missingFields)); - HeaderFormatError (sb.ToString ()); - } - - firstWrite = false; - rowFieldCounter = 0; - return; - } - - for (int i = 0; i < count; i++) { - byteCount++; - if (rowFieldCounter == 0) - outputWriter.Write ("\t.byte "); - - byte b = buffer [offset + i]; - if (rowFieldCounter > 0) - outputWriter.Write (", "); - outputWriter.Write ($"0x{b:x02}"); - - rowFieldCounter++; - if (rowFieldCounter > MaxFieldsInRow) { - rowFieldCounter = 0; - outputWriter.WriteLine (); - } - } - - void HeaderFormatError (string whatsWrong) - { - throw new InvalidOperationException ($"Java.Interop.Tools.JavaCallableWrappers.TypeNameMapGenerator header format changed: {whatsWrong}"); - } - - int GetHeaderInteger (string name, string value) - { - int ret; - if (!Int32.TryParse (value, out ret)) - HeaderFormatError ($"failed to parse integer value from '{value}' for field '{name}'"); - return ret; - } - } - }; -} diff --git a/src/Xamarin.Android.Build.Tasks/Utilities/NativeAssemblyGenerator.cs b/src/Xamarin.Android.Build.Tasks/Utilities/NativeAssemblyGenerator.cs index 17bfca35db7..3b8b8a54d5c 100644 --- a/src/Xamarin.Android.Build.Tasks/Utilities/NativeAssemblyGenerator.cs +++ b/src/Xamarin.Android.Build.Tasks/Utilities/NativeAssemblyGenerator.cs @@ -7,39 +7,49 @@ namespace Xamarin.Android.Tasks abstract class NativeAssemblyGenerator { uint structureByteCount; - uint structureAlignBytes; + bool structureIsPacked = false; bool writingStructure = false; protected string Indent { get; } = "\t"; protected NativeAssemblerTargetProvider TargetProvider { get; } + protected string TypemapsIncludeFile { get; } + protected string SharedIncludeFile { get; } + public string MainSourceFile { get; } - protected NativeAssemblyGenerator (NativeAssemblerTargetProvider targetProvider) + protected NativeAssemblyGenerator (NativeAssemblerTargetProvider targetProvider, string baseFilePath, bool sharedIncludeUsesAbiPrefix = false) { if (targetProvider == null) throw new ArgumentNullException (nameof (targetProvider)); + if (String.IsNullOrEmpty (baseFilePath)) + throw new ArgumentException ("must not be null or empty", nameof (baseFilePath)); + TargetProvider = targetProvider; + if (sharedIncludeUsesAbiPrefix) + SharedIncludeFile = $"{baseFilePath}.{targetProvider.AbiName}-shared.inc"; + else + SharedIncludeFile = $"{baseFilePath}.shared.inc"; + TypemapsIncludeFile = $"{baseFilePath}.{targetProvider.AbiName}-managed.inc"; + MainSourceFile = $"{baseFilePath}.{targetProvider.AbiName}.s"; } - public void Write (StreamWriter output, string outputFileName) + public void Write (StreamWriter output) { if (output == null) throw new ArgumentNullException (nameof (output)); - WriteFileHeader (output, outputFileName); + WriteFileHeader (output); WriteSymbols (output); + WriteFileFooter (output); output.Flush (); } protected virtual void WriteSymbols (StreamWriter output) - { - } + {} protected void WriteEndLine (StreamWriter output, string comment = null, bool indent = true) { if (!String.IsNullOrEmpty (comment)) { - if (indent) - output.Write (Indent); - WriteComment (output, comment); + WriteComment (output, comment, indent); } output.WriteLine (); } @@ -51,16 +61,19 @@ protected void WriteComment (StreamWriter output, string comment, bool indent = protected void WriteCommentLine (StreamWriter output, string comment, bool indent = true) { - WriteComment (output, comment); + WriteComment (output, comment, indent); output.WriteLine (); } - protected virtual void WriteFileHeader (StreamWriter output, string outputFileName) + protected virtual void WriteFileHeader (StreamWriter output) { TargetProvider.WriteFileHeader (output, Indent); - output.WriteLine ($"{Indent}.file{Indent}\"{Path.GetFileName (outputFileName)}\""); + output.WriteLine ($"{Indent}.file{Indent}\"{Path.GetFileName (MainSourceFile)}\""); } + protected virtual void WriteFileFooter (StreamWriter output) + {} + protected virtual void WriteSection (StreamWriter output, string sectionName, bool hasStrings, bool writable) { output.Write ($"{Indent}.section{Indent}{sectionName},\"a"); // a - section is allocatable @@ -106,7 +119,19 @@ protected void WriteSymbol (StreamWriter output, string label, uint size, bool i WriteSymbol (output, null, null, label, size, alignBits: 0, isGlobal: isGlobal, isObject: isObject, alwaysWriteSize: alwaysWriteSize); } - protected void WriteSymbol (StreamWriter output, string symbolName, uint alignBits, uint fieldAlignBytes, bool isGlobal, bool alwaysWriteSize, Func structureWriter) + protected void WriteSymbol (StreamWriter output, string label, uint size, uint alignBits, bool isGlobal, bool isObject, bool alwaysWriteSize) + { + WriteSymbol (output, null, null, label, size, alignBits, isGlobal: isGlobal, isObject: isObject, alwaysWriteSize: alwaysWriteSize); + } + + protected void WriteSymbol (StreamWriter output, string symbolName, uint alignBits, bool packed, bool isGlobal, bool alwaysWriteSize, Func structureWriter) + { + WriteStructureSymbol (output, symbolName, alignBits, isGlobal); + uint size = WriteStructure (output, packed, structureWriter); + WriteStructureSize (output, symbolName, size, alwaysWriteSize); + } + + protected void WriteStructureSymbol (StreamWriter output, string symbolName, uint alignBits, bool isGlobal) { output.WriteLine ($"{Indent}.type{Indent}{symbolName}, {TargetProvider.TypePrefix}object"); if (alignBits > 0) @@ -115,14 +140,28 @@ protected void WriteSymbol (StreamWriter output, string symbolName, uint alignBi output.WriteLine ($"{Indent}.global{Indent}{symbolName}"); if (!String.IsNullOrEmpty (symbolName)) output.WriteLine ($"{symbolName}:"); + } + + protected void WriteStructureSize (StreamWriter output, string symbolName, uint size, bool alwaysWriteSize = false) + { + if (size == 0 && !alwaysWriteSize) + return; + if (String.IsNullOrEmpty (symbolName)) + throw new ArgumentException ("symbol name must be non-empty in order to write structure size", nameof (symbolName)); + + output.WriteLine ($"{Indent}.size{Indent}{symbolName}, {size}"); + } + + protected uint WriteStructure (StreamWriter output, bool packed, Func structureWriter) + { writingStructure = true; structureByteCount = 0; - structureAlignBytes = fieldAlignBytes; + structureIsPacked = packed; uint size = structureWriter != null ? structureWriter () : 0u; writingStructure = false; - if (alwaysWriteSize || size > 0) - output.WriteLine ($"{Indent}.size{Indent}{symbolName}, {size}"); + + return size; } protected virtual string QuoteValue (T value) @@ -136,31 +175,38 @@ protected void WriteSymbol (StreamWriter output, string symbolName, uint alignBi return $"{value}"; } - protected uint WritePointer (StreamWriter output, string targetName, string label = null, bool isGlobal = false) + protected uint WritePointer (StreamWriter output, string targetName = null, string label = null, bool isGlobal = false) { - if (String.IsNullOrEmpty (targetName)) - throw new ArgumentException ("must not be null or empty", nameof (targetName)); - - uint fieldSize = UpdateSize (output, targetName); - WriteSymbol (output, TargetProvider.PointerFieldType, targetName, label, size: 0, alignBits: 0, isGlobal: isGlobal, isObject: false, alwaysWriteSize: false); + uint fieldSize = UpdateSize (output, targetName ?? String.Empty); + WriteSymbol (output, TargetProvider.PointerFieldType, targetName ?? "0", label, size: 0, alignBits: 0, isGlobal: isGlobal, isObject: false, alwaysWriteSize: false); return fieldSize; } uint WriteDataPadding (StreamWriter output, uint nextFieldSize, uint sizeSoFar) { - if (!writingStructure || structureAlignBytes <= 1 || nextFieldSize == 1) + if (!writingStructure || structureIsPacked || nextFieldSize == 1) return 0; - uint alignment = sizeSoFar % structureAlignBytes; + uint modulo; + if (TargetProvider.Is64Bit) { + modulo = nextFieldSize < 8 ? 4u : 8u; + } else { + modulo = 4u; + } + + uint alignment = sizeSoFar % modulo; if (alignment == 0) return 0; - uint nbytes = structureAlignBytes - alignment; + uint nbytes = modulo - alignment; return WriteDataPadding (output, nbytes); } protected uint WriteDataPadding (StreamWriter output, uint nbytes) { + if (nbytes == 0) + return 0; + output.WriteLine ($"{Indent}.zero{Indent}{nbytes}"); return nbytes; } @@ -173,24 +219,78 @@ protected uint WriteData (StreamWriter output, string value, string label, bool value = String.Empty; WriteSection (output, $".rodata.{label}", hasStrings: true, writable: false); - WriteSymbol (output, value, label, size: (ulong)(value.Length + 1), alignBits: 0, isGlobal: isGlobal, isObject: true, alwaysWriteSize: true); + WriteSymbol (output, value, isGlobal ? label : MakeLocalLabel (label), size: (ulong)(value.Length + 1), alignBits: 0, isGlobal: isGlobal, isObject: true, alwaysWriteSize: true); return TargetProvider.GetTypeSize (value); } + protected uint WriteAsciiData (StreamWriter output, string value, uint padToWidth = 0) + { + if (value == null) + value = String.Empty; + + uint size = (uint)output.Encoding.GetByteCount (value); + if (size > 0) { + output.WriteLine ($"{Indent}.ascii{Indent}\"{value}\""); + } else if (padToWidth == 0) { + return WriteDataPadding (output, 1); + } + + if (padToWidth > size) + size += WriteDataPadding (output, padToWidth - size); + + return size; + } + + protected string MakeLocalLabel (string label) + { + return $".L.{label}"; + } + + uint UpdateSize (StreamWriter output, T[] value) + { + uint typeSize = TargetProvider.GetTypeSize (); + uint fieldSize = typeSize * (uint)value.Length; + + fieldSize += WriteDataPadding (output, fieldSize, structureByteCount); + structureByteCount += fieldSize; + + return fieldSize; + } + uint UpdateSize (StreamWriter output, T value) { uint fieldSize; - if (typeof (T) == typeof (string)) + Type t = typeof(T); + if (t == typeof (string)) fieldSize = TargetProvider.GetPointerSize (); else fieldSize = TargetProvider.GetTypeSize (value); + fieldSize += WriteDataPadding (output, fieldSize, structureByteCount); structureByteCount += fieldSize; return fieldSize; } + protected uint WriteData (StreamWriter output, byte[] value, string label = null, bool isGlobal = false) + { + uint fieldSize = UpdateSize (output, value); + var symbolValue = new StringBuilder (); + bool first = true; + foreach (byte b in value) { + if (!first) + symbolValue.Append (", "); + else + first = false; + + symbolValue.Append ($"0x{b:x02}"); + } + WriteSymbol (output, TargetProvider.MapType (), symbolValue.ToString (), symbolName: label, size: 0, alignBits: 0, isGlobal: isGlobal, isObject: false, alwaysWriteSize: false); + + return fieldSize; + } + protected uint WriteData (StreamWriter output, byte value, string label = null, bool isGlobal = false) { uint fieldSize = UpdateSize (output, value); diff --git a/src/Xamarin.Android.Build.Tasks/Utilities/NativeTypeMappingData.cs b/src/Xamarin.Android.Build.Tasks/Utilities/NativeTypeMappingData.cs new file mode 100644 index 00000000000..c915d6d46a8 --- /dev/null +++ b/src/Xamarin.Android.Build.Tasks/Utilities/NativeTypeMappingData.cs @@ -0,0 +1,60 @@ +using System; +using System.Collections.Generic; +using System.Linq; + +namespace Xamarin.Android.Tasks +{ + class NativeTypeMappingData + { + public TypeMapGenerator.ModuleData[] Modules { get; } + public IDictionary AssemblyNames { get; } + public string[] JavaTypeNames { get; } + public TypeMapGenerator.TypeMapEntry[] JavaTypes { get; } + + public uint MapModuleCount { get; } + public uint JavaTypeCount { get; } + public uint JavaNameWidth { get; } + + public NativeTypeMappingData (Action logger, TypeMapGenerator.ModuleData[] modules, int javaNameWidth) + { + Modules = modules ?? throw new ArgumentNullException (nameof (modules)); + + MapModuleCount = (uint)modules.Length; + JavaNameWidth = (uint)javaNameWidth; + + AssemblyNames = new Dictionary (StringComparer.Ordinal); + + var tempJavaTypes = new Dictionary (StringComparer.Ordinal); + int managedStringCounter = 0; + var moduleComparer = new TypeMapGenerator.ModuleUUIDArrayComparer (); + + foreach (TypeMapGenerator.ModuleData data in modules) { + data.AssemblyNameLabel = $"map_aname.{managedStringCounter++}"; + AssemblyNames.Add (data.AssemblyNameLabel, data.AssemblyName); + + int moduleIndex = Array.BinarySearch (modules, data, moduleComparer); + if (moduleIndex < 0) + throw new InvalidOperationException ($"Unable to map module with MVID {data.Mvid} to array index"); + + foreach (TypeMapGenerator.TypeMapEntry entry in data.Types) { + entry.ModuleIndex = moduleIndex; + if (tempJavaTypes.ContainsKey (entry.JavaName)) + continue; + tempJavaTypes.Add (entry.JavaName, entry); + } + } + + var javaNames = tempJavaTypes.Keys.ToArray (); + Array.Sort (javaNames, StringComparer.Ordinal); + + var javaTypes = new TypeMapGenerator.TypeMapEntry[javaNames.Length]; + for (int i = 0; i < javaNames.Length; i++) { + javaTypes[i] = tempJavaTypes[javaNames[i]]; + } + + JavaTypes = javaTypes; + JavaTypeNames = javaNames; + JavaTypeCount = (uint)JavaTypes.Length; + } + } +} diff --git a/src/Xamarin.Android.Build.Tasks/Utilities/TypeMapGenerator.cs b/src/Xamarin.Android.Build.Tasks/Utilities/TypeMapGenerator.cs new file mode 100644 index 00000000000..f397f57019c --- /dev/null +++ b/src/Xamarin.Android.Build.Tasks/Utilities/TypeMapGenerator.cs @@ -0,0 +1,423 @@ +using System; +using System.IO; +using System.Collections.Generic; +using System.Linq; +using System.Text; + +using Mono.Cecil; + +namespace Xamarin.Android.Tasks +{ + class TypeMapGenerator + { + const string TypeMapMagicString = "XATM"; // Xamarin Android TypeMap + const string TypeMapIndexMagicString = "XATI"; // Xamarin Android Typemap Index + const uint TypeMapFormatVersion = 1; // Keep in sync with the value in src/monodroid/jni/xamarin-app.hh + + internal sealed class ModuleUUIDArrayComparer : IComparer + { + int Compare (byte[] left, byte[] right) + { + int ret; + + for (int i = 0; i < 16; i++) { + ret = left[i].CompareTo (right[i]); + if (ret != 0) + return ret; + } + + return 0; + } + + public int Compare (ModuleData left, ModuleData right) + { + return Compare (left.MvidBytes, right.MvidBytes); + } + } + + internal sealed class TypeMapEntryArrayComparer : IComparer + { + public int Compare (TypeMapEntry left, TypeMapEntry right) + { + return String.CompareOrdinal (left.JavaName, right.JavaName); + } + } + + internal sealed class TypeMapEntry + { + public string JavaName; + public int JavaNameLength; + public string ManagedTypeName; + public uint Token; + public int AssemblyNameIndex = -1; + public int ModuleIndex = -1; + } + + internal sealed class ModuleData + { + public Guid Mvid; + public byte[] MvidBytes; + public AssemblyDefinition Assembly; + public TypeMapEntry[] Types; + public Dictionary DuplicateTypes; + public string AssemblyName; + public string AssemblyNameLabel; + public string OutputFilePath; + + public Dictionary TypesScratch; + } + + Action logger; + Encoding outputEncoding; + byte[] moduleMagicString; + byte[] typemapIndexMagicString; + string[] supportedAbis; + + public IList GeneratedBinaryTypeMaps { get; } = new List (); + + public TypeMapGenerator (Action logger, string[] supportedAbis) + { + this.logger = logger ?? throw new ArgumentNullException (nameof (logger)); + if (supportedAbis == null) + throw new ArgumentNullException (nameof (supportedAbis)); + this.supportedAbis = supportedAbis; + + outputEncoding = new UTF8Encoding (false); + moduleMagicString = outputEncoding.GetBytes (TypeMapMagicString); + typemapIndexMagicString = outputEncoding.GetBytes (TypeMapIndexMagicString); + } + + public bool Generate (List javaTypes, string outputDirectory, bool generateNativeAssembly) + { + if (String.IsNullOrEmpty (outputDirectory)) + throw new ArgumentException ("must not be null or empty", nameof (outputDirectory)); + + if (!Directory.Exists (outputDirectory)) + Directory.CreateDirectory (outputDirectory); + + int assemblyId = 0; + int maxJavaNameLength = 0; + int maxModuleFileNameLength = 0; + var knownAssemblies = new Dictionary (StringComparer.Ordinal); + var tempModules = new Dictionary (); + Dictionary moduleCounter = null; + var mvidCache = new Dictionary (); + + foreach (TypeDefinition td in javaTypes) { + string assemblyName = td.Module.Assembly.FullName; + + if (!knownAssemblies.ContainsKey (assemblyName)) { + assemblyId++; + knownAssemblies.Add (assemblyName, assemblyId); + } + + // We must NOT use Guid here! The reason is that Guid sort order is different than its corresponding + // byte array representation and on the runtime we need the latter in order to be able to binary search + // through the module array. + byte[] moduleUUID; + if (!mvidCache.TryGetValue (td.Module.Mvid, out moduleUUID)) { + moduleUUID = td.Module.Mvid.ToByteArray (); + mvidCache.Add (td.Module.Mvid, moduleUUID); + } + + ModuleData moduleData; + if (!tempModules.TryGetValue (moduleUUID, out moduleData)) { + if (moduleCounter == null) + moduleCounter = new Dictionary (); + + moduleData = new ModuleData { + Mvid = td.Module.Mvid, + MvidBytes = moduleUUID, + Assembly = td.Module.Assembly, + AssemblyName = td.Module.Assembly.Name.Name, + TypesScratch = new Dictionary (StringComparer.Ordinal), + DuplicateTypes = new Dictionary (), + }; + tempModules.Add (moduleUUID, moduleData); + + if (!generateNativeAssembly) { + int moduleNum; + if (!moduleCounter.TryGetValue (moduleData.Assembly, out moduleNum)) { + moduleNum = 0; + moduleCounter [moduleData.Assembly] = 0; + } else { + moduleNum++; + moduleCounter [moduleData.Assembly] = moduleNum; + } + + string fileName = $"{moduleData.Assembly.Name.Name}.{moduleNum}.typemap"; + moduleData.OutputFilePath = Path.Combine (outputDirectory, fileName); + if (maxModuleFileNameLength < fileName.Length) + maxModuleFileNameLength = fileName.Length; + } + } + + string javaName = Java.Interop.Tools.TypeNameMappings.JavaNativeTypeManager.ToJniName (td); + var entry = new TypeMapEntry { + JavaName = javaName, + JavaNameLength = outputEncoding.GetByteCount (javaName), + ManagedTypeName = td.FullName, + Token = td.MetadataToken.ToUInt32 (), + AssemblyNameIndex = knownAssemblies [assemblyName] + }; + + if (generateNativeAssembly) { + if (entry.JavaNameLength > maxJavaNameLength) + maxJavaNameLength = entry.JavaNameLength; + } + + if (moduleData.TypesScratch.ContainsKey (entry.JavaName)) { + // This is disabled because it costs a lot of time (around 150ms per standard XF Integration app + // build) and has no value for the end user. The message is left here because it may be useful to us + // in our devloop at some point. + //logger ($"Warning: duplicate Java type name '{entry.JavaName}' in assembly '{moduleData.AssemblyName}' (new token: {entry.Token})."); + moduleData.DuplicateTypes.Add (entry.Token, entry); + } else + moduleData.TypesScratch.Add (entry.JavaName, entry); + } + + var modules = tempModules.Values.ToArray (); + Array.Sort (modules, new ModuleUUIDArrayComparer ()); + + var typeMapEntryComparer = new TypeMapEntryArrayComparer (); + foreach (ModuleData module in modules) { + if (module.TypesScratch.Count == 0) { + module.Types = new TypeMapEntry[0]; + continue; + } + + module.Types = module.TypesScratch.Values.ToArray (); + Array.Sort (module.Types, typeMapEntryComparer); + } + + NativeTypeMappingData data; + if (!generateNativeAssembly) { + string typeMapIndexPath = Path.Combine (outputDirectory, "typemap.index"); + // Try to approximate the index size: + // 16 bytes for the header + // 16 bytes (UUID) + filename length per each entry + using (var ms = new MemoryStream (16 + (modules.Length * (16 + 128)))) { + using (var indexWriter = new BinaryWriter (ms)) { + OutputModules (outputDirectory, modules, indexWriter, maxModuleFileNameLength + 1); + indexWriter.Flush (); + MonoAndroidHelper.CopyIfStreamChanged (ms, typeMapIndexPath); + } + } + GeneratedBinaryTypeMaps.Add (typeMapIndexPath); + + data = new NativeTypeMappingData (logger, new ModuleData[0], 0); + } else { + data = new NativeTypeMappingData (logger, modules, maxJavaNameLength + 1); + } + + NativeAssemblerTargetProvider asmTargetProvider; + bool sharedBitsWritten = false; + bool sharedIncludeUsesAbiPrefix; + foreach (string abi in supportedAbis) { + sharedIncludeUsesAbiPrefix = false; + switch (abi.Trim ()) { + case "armeabi-v7a": + asmTargetProvider = new ARMNativeAssemblerTargetProvider (is64Bit: false); + sharedIncludeUsesAbiPrefix = true; // ARMv7a is "special", it uses different directive prefix + // than the others and the "shared" code won't build for it + break; + + case "arm64-v8a": + asmTargetProvider = new ARMNativeAssemblerTargetProvider (is64Bit: true); + break; + + case "x86": + asmTargetProvider = new X86NativeAssemblerTargetProvider (is64Bit: false); + break; + + case "x86_64": + asmTargetProvider = new X86NativeAssemblerTargetProvider (is64Bit: true); + break; + + default: + throw new InvalidOperationException ($"Unknown ABI {abi}"); + } + + var generator = new TypeMappingNativeAssemblyGenerator (asmTargetProvider, data, Path.Combine (outputDirectory, "typemaps"), sharedBitsWritten, sharedIncludeUsesAbiPrefix); + + using (var ms = new MemoryStream ()) { + using (var sw = new StreamWriter (ms, outputEncoding)) { + generator.Write (sw); + sw.Flush (); + MonoAndroidHelper.CopyIfStreamChanged (ms, generator.MainSourceFile); + if (!sharedIncludeUsesAbiPrefix) + sharedBitsWritten = true; + } + } + } + return true; + } + + // Binary index file format, all data is little-endian: + // + // [Magic string] # XATI + // [Format version] # 32-bit unsigned integer, 4 bytes + // [Entry count] # 32-bit unsigned integer, 4 bytes + // [Module file name width] # 32-bit unsigned integer, 4 bytes + // [Index entries] # Format described below, [Entry count] entries + // + // Index entry format: + // + // [Module UUID][File name] + // + // Where: + // + // [Module UUID] is 16 bytes long + // [File name] is right-padded with characters to the [Module file name width] boundary. + // + void OutputModules (string outputDirectory, ModuleData[] modules, BinaryWriter indexWriter, int moduleFileNameWidth) + { + var moduleCounter = new Dictionary (); + + indexWriter.Write (typemapIndexMagicString); + indexWriter.Write (TypeMapFormatVersion); + indexWriter.Write (modules.Length); + indexWriter.Write (moduleFileNameWidth); + + foreach (ModuleData data in modules) { + OutputModule (outputDirectory, data.MvidBytes, data, moduleCounter); + indexWriter.Write (data.MvidBytes); + + string outputFilePath = Path.GetFileName (data.OutputFilePath); + indexWriter.Write (outputEncoding.GetBytes (outputFilePath)); + PadField (indexWriter, outputFilePath.Length, moduleFileNameWidth); + } + } + + void OutputModule (string outputDirectory, byte[] moduleUUID, ModuleData moduleData, Dictionary moduleCounter) + { + if (moduleData.Types.Length == 0) + return; + + int initialStreamSize = + (36 + 128) + // Static header size + assembly file name + ((128 + 4) * moduleData.Types.Length) + // java-to-managed size + (8 * moduleData.Types.Length) + // managed-to-java size + (8 * moduleData.DuplicateTypes.Count); // managed-to-java duplicates; + + using (var ms = new MemoryStream (initialStreamSize)) { + using (var bw = new BinaryWriter (ms)) { + OutputModule (bw, moduleUUID, moduleData); + bw.Flush (); + MonoAndroidHelper.CopyIfStreamChanged (ms, moduleData.OutputFilePath); + } + } + GeneratedBinaryTypeMaps.Add (moduleData.OutputFilePath); + } + + // Binary file format, all data is little-endian: + // + // [Magic string] # XATM + // [Format version] # 32-bit integer, 4 bytes + // [Module UUID] # 16 bytes + // [Entry count] # unsigned 32-bit integer, 4 bytes + // [Duplicate count] # unsigned 32-bit integer, 4 bytes (might be 0) + // [Java type name width] # unsigned 32-bit integer, 4 bytes + // [Assembly name size] # unsigned 32-bit integer, 4 bytes + // [Assembly name] # Non-null terminated assembly name + // [Java-to-managed map] # Format described below, [Entry count] entries + // [Managed-to-java map] # Format described below, [Entry count] entries + // [Managed-to-java duplicates map] # Map of unique managed IDs which point to the same Java type name (might be empty) + // + // Java-to-managed map format: + // + // [Java type name][Managed type token ID] + // + // Each name is padded with to the width specified in the [Java type name width] field above. + // Names are written without the size prefix, instead they are always terminated with a nul character + // to make it easier and faster to handle by the native runtime. + // + // Each token ID is an unsigned 32-bit integer, 4 bytes + // + // + // Managed-to-java map format: + // + // [Managed type token ID][Java type name table index] + // + // Both fields are unsigned 32-bit integers, to a total of 8 bytes per entry. Index points into the + // [Java-to-managed map] table above. + // + // Managed-to-java duplicates map format: + // + // Format is identical to [Managed-to-java] above. + // + void OutputModule (BinaryWriter bw, byte[] moduleUUID, ModuleData moduleData) + { + bw.Write (moduleMagicString); + bw.Write (TypeMapFormatVersion); + bw.Write (moduleUUID); + + var javaNames = new Dictionary (StringComparer.Ordinal); + var managedTypes = new Dictionary (); + int maxJavaNameLength = 0; + + foreach (TypeMapEntry entry in moduleData.Types) { + javaNames.Add (entry.JavaName, entry.Token); + if (entry.JavaNameLength > maxJavaNameLength) + maxJavaNameLength = entry.JavaNameLength; + + managedTypes.Add (entry.Token, 0); + } + + var javaNameList = javaNames.Keys.ToList (); + foreach (TypeMapEntry entry in moduleData.Types) { + var javaIndex = (uint)javaNameList.IndexOf (entry.JavaName); + managedTypes[entry.Token] = javaIndex; + } + + bw.Write (javaNames.Count); + bw.Write (moduleData.DuplicateTypes.Count); + bw.Write (maxJavaNameLength + 1); + + string assemblyName = moduleData.Assembly.Name.Name; + bw.Write (assemblyName.Length); + bw.Write (outputEncoding.GetBytes (assemblyName)); + + var sortedJavaNames = javaNames.Keys.ToArray (); + Array.Sort (sortedJavaNames, StringComparer.Ordinal); + foreach (string typeName in sortedJavaNames) { + byte[] bytes = outputEncoding.GetBytes (typeName); + bw.Write (bytes); + PadField (bw, bytes.Length, maxJavaNameLength + 1); + bw.Write (javaNames[typeName]); + } + + WriteManagedTypes (managedTypes); + if (moduleData.DuplicateTypes.Count == 0) + return; + + var managedDuplicates = new Dictionary (); + foreach (var kvp in moduleData.DuplicateTypes) { + uint javaIndex = kvp.Key; + uint typeId = kvp.Value.Token; + + managedDuplicates.Add (javaIndex, typeId); + } + + WriteManagedTypes (managedDuplicates); + + void WriteManagedTypes (IDictionary types) + { + var sortedTokens = types.Keys.ToArray (); + Array.Sort (sortedTokens); + + foreach (uint token in sortedTokens) { + bw.Write (token); + bw.Write (types[token]); + } + } + } + + void PadField (BinaryWriter bw, int valueWidth, int maxWidth) + { + for (int i = 0; i < maxWidth - valueWidth; i++) { + bw.Write ((byte)0); + } + } + } +} diff --git a/src/Xamarin.Android.Build.Tasks/Utilities/TypeMappingNativeAssemblyGenerator.cs b/src/Xamarin.Android.Build.Tasks/Utilities/TypeMappingNativeAssemblyGenerator.cs index 5e1c82f1c5a..790fe071af5 100644 --- a/src/Xamarin.Android.Build.Tasks/Utilities/TypeMappingNativeAssemblyGenerator.cs +++ b/src/Xamarin.Android.Build.Tasks/Utilities/TypeMappingNativeAssemblyGenerator.cs @@ -1,75 +1,239 @@ using System; +using System.Collections.Generic; using System.IO; -using Xamarin.Android.Tools; +using System.Linq; namespace Xamarin.Android.Tasks { class TypeMappingNativeAssemblyGenerator : NativeAssemblyGenerator { - NativeAssemblyDataStream dataStream; - string dataFileName; - uint dataSize; - string mappingFieldName; + readonly string baseFileName; + readonly NativeTypeMappingData mappingData; + readonly bool sharedBitsWritten; - public TypeMappingNativeAssemblyGenerator (NativeAssemblerTargetProvider targetProvider, NativeAssemblyDataStream dataStream, string dataFileName, uint dataSize, string mappingFieldName) - : base (targetProvider) + public TypeMappingNativeAssemblyGenerator (NativeAssemblerTargetProvider targetProvider, NativeTypeMappingData mappingData, string baseFileName, bool sharedBitsWritten, bool sharedIncludeUsesAbiPrefix = false) + : base (targetProvider, baseFileName, sharedIncludeUsesAbiPrefix) { - if (dataStream == null) - throw new ArgumentNullException (nameof (dataStream)); - if (String.IsNullOrEmpty (dataFileName)) - throw new ArgumentException ("must not be null or empty", nameof (dataFileName)); - if (String.IsNullOrEmpty (mappingFieldName)) - throw new ArgumentException ("must not be null or empty", nameof (mappingFieldName)); - - this.dataStream = dataStream; - this.dataFileName = dataFileName; - this.dataSize = dataSize; - this.mappingFieldName = mappingFieldName; + this.mappingData = mappingData ?? throw new ArgumentNullException (nameof (mappingData)); + + if (String.IsNullOrEmpty (baseFileName)) + throw new ArgumentException("must not be null or empty", nameof (baseFileName)); + + this.baseFileName = baseFileName; + this.sharedBitsWritten = sharedIncludeUsesAbiPrefix ? false : sharedBitsWritten; } - public bool EmbedAssemblies { get; set; } + protected override void WriteSymbols (StreamWriter output) + { + output.WriteLine (); + WriteHeaderField (output, "map_module_count", mappingData.MapModuleCount); + + output.WriteLine (); + WriteHeaderField (output, "java_type_count", mappingData.JavaTypeCount); + + output.WriteLine (); + WriteHeaderField (output, "java_name_width", mappingData.JavaNameWidth); + + bool haveAssemblyNames = mappingData.AssemblyNames.Count > 0; + bool haveModules = mappingData.Modules.Length > 0; - protected override void WriteFileHeader (StreamWriter output, string outputFileName) + output.WriteLine (); + if (haveAssemblyNames) { + output.WriteLine ($"{Indent}.include{Indent}\"{Path.GetFileName (SharedIncludeFile)}\""); + } else { + WriteCommentLine (output, $"No shared data present, {Path.GetFileName (SharedIncludeFile)} not generated"); + } + + if (haveModules) { + output.WriteLine ($"{Indent}.include{Indent}\"{Path.GetFileName (TypemapsIncludeFile)}\""); + } else { + WriteCommentLine (output, $"No modules defined, {Path.GetFileName (TypemapsIncludeFile)} not generated"); + } + output.WriteLine (); + + if (!sharedBitsWritten && haveAssemblyNames) { + using (var ms = new MemoryStream ()) { + using (var sharedOutput = new StreamWriter (ms, output.Encoding)) { + WriteAssemblyNames (sharedOutput); + sharedOutput.Flush (); + MonoAndroidHelper.CopyIfStreamChanged (ms, SharedIncludeFile); + } + } + } + + if (haveModules) { + using (var ms = new MemoryStream ()) { + using (var mapOutput = new StreamWriter (ms, output.Encoding)) { + WriteMapModules (output, mapOutput, "map_modules"); + mapOutput.Flush (); + MonoAndroidHelper.CopyIfStreamChanged (ms, TypemapsIncludeFile); + } + } + } else { + WriteMapModules (output, null, "map_modules"); + } + + WriteJavaMap (output, "map_java"); + } + + void WriteAssemblyNames (StreamWriter output) { - // The hash is written to make sure the assembly file which includes the data one is - // actually different whenever the data changes. Relying on mapping header values for this - // purpose would not be enough since the only change to the mapping might be a single-character - // change in one of the type names and we must be sure the assembly is rebuilt in all cases, - // thus the hash. - WriteEndLine (output, $"Data Hash: {Files.ToHexString (dataStream.GetStreamHash ())}", false); - base.WriteFileHeader (output, outputFileName); + foreach (var kvp in mappingData.AssemblyNames) { + string label = kvp.Key; + string name = kvp.Value; + + WriteData (output, name, label, isGlobal: false); + output.WriteLine (); + } } - protected override void WriteSymbols (StreamWriter output) + void WriteManagedMaps (StreamWriter output, string moduleSymbolName, IEnumerable entries) + { + if (entries == null) + return; + + var tokens = new Dictionary (); + foreach (TypeMapGenerator.TypeMapEntry entry in entries) { + int idx = Array.BinarySearch (mappingData.JavaTypeNames, entry.JavaName, StringComparer.Ordinal); + if (idx < 0) + throw new InvalidOperationException ($"Could not map entry '{entry.JavaName}' to array index"); + + tokens[entry.Token] = (uint)idx; + } + + WriteSection (output, $".rodata.{moduleSymbolName}", hasStrings: false, writable: false); + WriteStructureSymbol (output, moduleSymbolName, alignBits: 0, isGlobal: false); + + uint size = 0; + var sortedTokens = tokens.Keys.ToArray (); + Array.Sort (sortedTokens); + + foreach (uint token in sortedTokens) { + size += WriteStructure (output, packed: false, structureWriter: () => WriteManagedMapEntry (output, token, tokens[token])); + } + + WriteStructureSize (output, moduleSymbolName, size); + output.WriteLine (); + } + + uint WriteManagedMapEntry (StreamWriter output, uint token, uint javaMapIndex) { - WriteMappingHeader (output, dataStream, mappingFieldName); - WriteCommentLine (output, "Mapping data"); - WriteSymbol (output, mappingFieldName, dataSize, isGlobal: true, isObject: true, alwaysWriteSize: true); - if (EmbedAssemblies) { - output.WriteLine ($"{Indent}.include{Indent}\"{dataFileName}\""); + uint size = WriteData (output, token); + size += WriteData (output, javaMapIndex); + + return size; + } + + void WriteMapModules (StreamWriter output, StreamWriter mapOutput, string symbolName) + { + WriteCommentLine (output, "Managed to Java map: START", indent: false); + WriteSection (output, $".data.rel.{symbolName}", hasStrings: false, writable: true); + WriteStructureSymbol (output, symbolName, alignBits: TargetProvider.MapModulesAlignBits, isGlobal: true); + + uint size = 0; + int moduleCounter = 0; + foreach (TypeMapGenerator.ModuleData data in mappingData.Modules) { + string mapName = $"module{moduleCounter++}_managed_to_java"; + string duplicateMapName; + + if (data.DuplicateTypes.Count == 0) + duplicateMapName = null; + else + duplicateMapName = $"{mapName}_duplicates"; + + size += WriteStructure (output, packed: false, structureWriter: () => WriteMapModule (output, mapName, duplicateMapName, data)); + if (mapOutput != null) { + WriteManagedMaps (mapOutput, mapName, data.Types); + if (data.DuplicateTypes.Count > 0) + WriteManagedMaps (mapOutput, duplicateMapName, data.DuplicateTypes.Values); + } } + + WriteStructureSize (output, symbolName, size); + WriteCommentLine (output, "Managed to Java map: END", indent: false); + output.WriteLine (); } - void WriteMappingHeader (StreamWriter output, NativeAssemblyDataStream dataStream, string mappingFieldName) + uint WriteMapModule (StreamWriter output, string mapName, string duplicateMapName, TypeMapGenerator.ModuleData data) { + uint size = 0; + WriteCommentLine (output, $"module_uuid: {data.Mvid}"); + size += WriteData (output, data.MvidBytes); + + WriteCommentLine (output, "entry_count"); + size += WriteData (output, data.Types.Length); + + WriteCommentLine (output, "duplicate_count"); + size += WriteData (output, data.DuplicateTypes.Count); + + WriteCommentLine (output, "map"); + size += WritePointer (output, mapName); + + WriteCommentLine (output, "duplicate_map"); + size += WritePointer (output, duplicateMapName); + + WriteCommentLine (output, $"assembly_name: {data.AssemblyName}"); + size += WritePointer (output, MakeLocalLabel (data.AssemblyNameLabel)); + + WriteCommentLine (output, "image"); + size += WritePointer (output); + + // These two are used only in Debug builds with Instant Run enabled, but for simplicity we always output + // them. + WriteCommentLine (output, "java_name_width"); + size += WriteData (output, (uint)0); + + WriteCommentLine (output, "java_map"); + size += WritePointer (output); + output.WriteLine (); - WriteCommentLine (output, "Mapping header"); - WriteSection (output, $".data.{mappingFieldName}", hasStrings: false, writable: true); - WriteSymbol (output, $"{mappingFieldName}_header", alignBits: 2, fieldAlignBytes: 4, isGlobal: true, alwaysWriteSize: true, structureWriter: () => { - WriteCommentLine (output, "version"); - WriteData (output, dataStream.MapVersion); - - WriteCommentLine (output, "entry-count"); - WriteData (output, dataStream.MapEntryCount); - - WriteCommentLine (output, "entry-length"); - WriteData (output, dataStream.MapEntryLength); - - WriteCommentLine (output, "value-offset"); - WriteData (output, dataStream.MapValueOffset); - return 16; - }); + + return size; + } + + void WriteJavaMap (StreamWriter output, string symbolName) + { + WriteCommentLine (output, "Java to managed map: START", indent: false); + WriteSection (output, $".rodata.{symbolName}", hasStrings: false, writable: false); + WriteStructureSymbol (output, symbolName, alignBits: TargetProvider.MapJavaAlignBits, isGlobal: true); + + uint size = 0; + int entryCount = 0; + foreach (TypeMapGenerator.TypeMapEntry entry in mappingData.JavaTypes) { + size += WriteJavaMapEntry (output, entry, entryCount++); + } + + WriteStructureSize (output, symbolName, size); + WriteCommentLine (output, "Java to managed map: END", indent: false); + output.WriteLine (); + } + + uint WriteJavaMapEntry (StreamWriter output, TypeMapGenerator.TypeMapEntry entry, int entryIndex) + { + uint size = 0; + + WriteCommentLine (output, $"#{entryIndex}"); + WriteCommentLine (output, "module_index"); + size += WriteData (output, entry.ModuleIndex); + + WriteCommentLine (output, "type_token_id"); + size += WriteData (output, entry.Token); + + WriteCommentLine (output, "java_name"); + size += WriteAsciiData (output, entry.JavaName, mappingData.JavaNameWidth); + output.WriteLine (); + + return size; + } + + void WriteHeaderField (StreamWriter output, string name, uint value) + { + WriteCommentLine (output, $"{name}: START", indent: false); + WriteSection (output, $".rodata.{name}", hasStrings: false, writable: false); + WriteSymbol (output, name, size: 4, alignBits: 2, isGlobal: true, isObject: true, alwaysWriteSize: true); + WriteData (output, value); + WriteCommentLine (output, $"{name}: END", indent: false); } } } diff --git a/src/Xamarin.Android.Build.Tasks/Utilities/X86NativeAssemblerTargetProvider.cs b/src/Xamarin.Android.Build.Tasks/Utilities/X86NativeAssemblerTargetProvider.cs index f6f2a33144a..bb36f000526 100644 --- a/src/Xamarin.Android.Build.Tasks/Utilities/X86NativeAssemblerTargetProvider.cs +++ b/src/Xamarin.Android.Build.Tasks/Utilities/X86NativeAssemblerTargetProvider.cs @@ -4,9 +4,15 @@ namespace Xamarin.Android.Tasks { class X86NativeAssemblerTargetProvider : NativeAssemblerTargetProvider { + const string X86 = "x86"; + const string X86_64 = "x86_64"; + public override bool Is64Bit { get; } public override string PointerFieldType { get; } public override string TypePrefix { get; } = "@"; + public override string AbiName => Is64Bit ? X86_64 : X86; + public override uint MapModulesAlignBits => Is64Bit ? 4u : 2u; + public override uint MapJavaAlignBits => Is64Bit ? 4u : 2u; public X86NativeAssemblerTargetProvider (bool is64Bit) { diff --git a/src/Xamarin.Android.Build.Tasks/Xamarin.Android.Common.targets b/src/Xamarin.Android.Build.Tasks/Xamarin.Android.Common.targets index f8f2c88398f..38d83b6168a 100644 --- a/src/Xamarin.Android.Build.Tasks/Xamarin.Android.Common.targets +++ b/src/Xamarin.Android.Build.Tasks/Xamarin.Android.Common.targets @@ -351,7 +351,8 @@ Copyright (C) 2011-2012 Xamarin. All rights reserved. <_AndroidEnablePreloadAssembliesDefault>True $(_AndroidEnablePreloadAssembliesDefault) - <_NativeAssemblySourceDir>$(IntermediateOutputPath)android\ + <_NativeAssemblySourceDir>$(IntermediateOutputPath)android\ + <_AndroidUseNewTypemaps>True @@ -1162,8 +1163,6 @@ because xbuild doesn't support framework reference assemblies. <_RemoveRegisterFlag>$(MonoAndroidIntermediateAssemblyDir)shrunk\shrunk.flag <_AcwMapFile>$(IntermediateOutputPath)acw-map.txt <_CustomViewMapFile>$(IntermediateOutputPath)customview-map.txt - <_AndroidTypeMappingJavaToManaged>$(IntermediateOutputPath)android\typemap.jm - <_AndroidTypeMappingManagedToJava>$(IntermediateOutputPath)android\typemap.mj $(RootNamespace) <_AndroidLintConfigFile>$(IntermediateOutputPath)lint.xml <_AndroidResourceDesignerFile Condition=" '$(AndroidUseIntermediateDesignerFile)' == 'True' ">$(IntermediateOutputPath)$(_AndroidResourceDesigner) @@ -2039,23 +2038,19 @@ because xbuild doesn't support framework reference assemblies. - - - - - + BuildTargetAbis="@(_BuildTargetAbis)" + NativeSourcesDir="$(_NativeAssemblySourceDir)" + TypeMapMode="True"> + + <_GenerateJavaStubsDependsOnTargets> _SetLatestTargetFrameworkVersion; - _PrepareNativeAssemblySources; _PrepareAssemblies; + _PrepareNativeAssemblySources; $(_AfterPrepareAssemblies); @@ -2088,6 +2083,8 @@ because xbuild doesn't support framework reference assemblies. PackageName="$(_AndroidPackage)" ManifestPlaceholders="$(AndroidManifestPlaceholders)" OutputDirectory="$(IntermediateOutputPath)android" + TypemapOutputDirectory="$(_NativeAssemblySourceDir)" + GenerateNativeAssembly="!$(_InstantRunEnabled)" MergedAndroidManifestOutput="$(_ManifestOutput)" UseSharedRuntime="$(AndroidUseSharedRuntime)" EmbedAssemblies="$(EmbedAssembliesIntoApk)" @@ -2097,10 +2094,13 @@ because xbuild doesn't support framework reference assemblies. FrameworkDirectories="$(_XATargetFrameworkDirectories);$(_XATargetFrameworkDirectories)Facades" AcwMapFile="$(_AcwMapFile)" SupportedAbis="@(_BuildTargetAbis)"> + + + @@ -2157,8 +2157,9 @@ because xbuild doesn't support framework reference assemblies. - + NativeSourcesDir="$(_NativeAssemblySourceDir)" + TypeMapMode="false"> + @@ -2222,6 +2223,7 @@ because xbuild doesn't support framework reference assemblies. EnablePreloadAssembliesDefault="$(_AndroidEnablePreloadAssembliesDefault)" PackageNamingPolicy="$(AndroidPackageNamingPolicy)" BoundExceptionType="$(AndroidBoundExceptionType)" + InstantRunEnabled="$(_InstantRunEnabled)" > @@ -2535,7 +2537,6 @@ because xbuild doesn't support framework reference assemblies. @@ -2578,7 +2579,7 @@ because xbuild doesn't support framework reference assemblies. - + <_NativeAssemblyTarget Include="@(_TypeMapAssemblySource->Replace('.s', '.o'))"> %(_TypeMapAssemblySource.abi) @@ -2591,7 +2592,7 @@ because xbuild doesn't support framework reference assemblies. (fileStat.st_size), 1); - value = new char[r]; - size_t nread = fread (value, 1, static_cast(fileStat.st_size), fp); - if (nread == 0 || nread != static_cast(fileStat.st_size)) { - log_warn(LOG_DEFAULT, "While reading file %s: expected to read %u bytes, actually read %u bytes", path, r, nread); - } - } - fclose (fp); - } - return r; -} - #if defined (DEBUG) || !defined (ANDROID) size_t AndroidSystem::_monodroid_get_system_property_from_file (const char *path, char **value) diff --git a/src/monodroid/jni/android-system.hh b/src/monodroid/jni/android-system.hh index 6c6812a5982..13408ed3f2c 100644 --- a/src/monodroid/jni/android-system.hh +++ b/src/monodroid/jni/android-system.hh @@ -9,7 +9,7 @@ #include "util.hh" #include "cppcompat.hh" -#include "xamarin-app.h" +#include "xamarin-app.hh" #include "shared-constants.hh" #include "basic-android-system.hh" @@ -45,7 +45,6 @@ namespace xamarin::android::internal void create_update_dir (char *override_dir); int monodroid_get_system_property (const char *name, char **value); size_t monodroid_get_system_property_from_overrides (const char *name, char ** value); - size_t monodroid_read_file_into_memory (const char *path, char *&value); char* get_bundled_app (JNIEnv *env, jstring dir); int count_override_assemblies (); long get_gref_gc_threshold (); diff --git a/src/monodroid/jni/application_dso_stub.cc b/src/monodroid/jni/application_dso_stub.cc index 459bb600391..2cc3c860551 100644 --- a/src/monodroid/jni/application_dso_stub.cc +++ b/src/monodroid/jni/application_dso_stub.cc @@ -1,16 +1,16 @@ #include #include -#include "xamarin-app.h" +#include "xamarin-app.hh" -// This file MUST have valid values everywhere - the DSO it is compiled into is loaded by the +// This file MUST have "valid" values everywhere - the DSO it is compiled into is loaded by the // designer on desktop. +const uint32_t map_module_count = 0; +const uint32_t java_type_count = 0; +const uint32_t java_name_width = 0; -TypeMapHeader jm_typemap_header = { 1, 0, 0, 0 }; -uint8_t jm_typemap[] = { 0 }; - -TypeMapHeader mj_typemap_header = { 1, 0, 0, 0 }; -uint8_t mj_typemap[] = { 0 }; +const TypeMapModule map_modules[] = {}; +const TypeMapJava map_java[] = {}; ApplicationConfig application_config = { .uses_mono_llvm = false, diff --git a/src/monodroid/jni/cpp-util.hh b/src/monodroid/jni/cpp-util.hh index 686c86da5b7..7195f2c33c0 100644 --- a/src/monodroid/jni/cpp-util.hh +++ b/src/monodroid/jni/cpp-util.hh @@ -51,17 +51,6 @@ namespace xamarin::android return tp; } - protected: - void set_pointer (T* ptr) noexcept - { - tp = ptr; - } - - void set_pointer (const T* ptr) noexcept - { - tp = ptr; - } - private: T *tp = nullptr; }; diff --git a/src/monodroid/jni/embedded-assemblies-zip.cc b/src/monodroid/jni/embedded-assemblies-zip.cc index 8755a65c6bb..fa5e0193d04 100644 --- a/src/monodroid/jni/embedded-assemblies-zip.cc +++ b/src/monodroid/jni/embedded-assemblies-zip.cc @@ -74,19 +74,6 @@ EmbeddedAssemblies::zip_load_entries (int fd, const char *apk_name, monodroid_sh if (compression_method != 0) continue; -#if defined (DEBUG) - if (utils.ends_with (file_name, ".jm")) { - md_mmap_info map_info = md_mmap_apk_file (fd, data_offset, file_size, file_name, apk_name); - add_type_mapping (&java_to_managed_maps, apk_name, file_name, (const char*)map_info.area); - continue; - } - - if (utils.ends_with (file_name, ".mj")) { - md_mmap_info map_info = md_mmap_apk_file (fd, data_offset, file_size, file_name, apk_name); - add_type_mapping (&managed_to_java_maps, apk_name, file_name, (const char*)map_info.area); - continue; - } -#endif if (strncmp (prefix, file_name, prefix_len) != 0) continue; diff --git a/src/monodroid/jni/embedded-assemblies.cc b/src/monodroid/jni/embedded-assemblies.cc index 9c703e8c70d..19f4467a544 100644 --- a/src/monodroid/jni/embedded-assemblies.cc +++ b/src/monodroid/jni/embedded-assemblies.cc @@ -10,6 +10,7 @@ #include #include #include +#include #include #include @@ -22,26 +23,36 @@ #include "embedded-assemblies.hh" #include "globals.hh" #include "monodroid-glue.hh" -#include "xamarin-app.h" +#include "xamarin-app.hh" #include "cpp-util.hh" -namespace xamarin::android::internal { -#if defined (DEBUG) || !defined (ANDROID) - struct TypeMappingInfo { - char *source_apk; - char *source_entry; - int num_entries; - int entry_length; - int value_offset; - const char *mapping; - TypeMappingInfo *next; - }; -#endif // DEBUG || !ANDROID -} - using namespace xamarin::android; using namespace xamarin::android::internal; +// A utility class which allows us to manage memory allocated by `mono_guid_to_string` in an elegant way. We can create +// temporary instances of this class in calls to e.g. `log_debug` which are executed ONLY when debug logging is enabled +class MonoGuidString +{ +public: + explicit MonoGuidString (const uint8_t *id) noexcept + { + guid = mono_guid_to_string (id); + } + + ~MonoGuidString () + { + ::free (guid); + } + + const char* get () const noexcept + { + return guid; + } + +private: + char *guid = nullptr; +}; + void EmbeddedAssemblies::set_assemblies_prefix (const char *prefix) { if (assemblies_prefix_override != nullptr) @@ -119,120 +130,284 @@ EmbeddedAssemblies::install_preload_hooks () mono_install_assembly_refonly_preload_hook (open_from_bundles_refonly, nullptr); } -int -EmbeddedAssemblies::TypeMappingInfo_compare_key (const void *a, const void *b) +template +const Entry* +EmbeddedAssemblies::binary_search (const Key *key, const Entry *base, size_t nmemb, [[maybe_unused]] size_t extra_size) { - return strcmp (reinterpret_cast (a), reinterpret_cast (b)); -} + static_assert (compare != nullptr, "compare is a required template parameter"); -inline const char* -EmbeddedAssemblies::find_entry_in_type_map (const char *name, uint8_t map[], TypeMapHeader& header) -{ - const char *e = reinterpret_cast (bsearch (name, map, header.entry_count, header.entry_length, TypeMappingInfo_compare_key )); - if (e == nullptr) + // This comes from the user code, so let's be civil + if (key == nullptr) { + log_warn (LOG_ASSEMBLY, "Key passed to binary_search must not be nullptr"); return nullptr; - return e + header.value_offset; + } + + // This is a coding error on our part, crash! + if (base == nullptr) { + log_fatal (LOG_ASSEMBLY, "Map address not passed to binary_search"); + exit (FATAL_EXIT_MISSING_ASSEMBLY); + } + + constexpr size_t size = sizeof(Entry); + while (nmemb > 0) { + const Entry *ret; + if constexpr (use_extra_size) { + ret = reinterpret_cast(reinterpret_cast(base) + ((size + extra_size) * (nmemb / 2))); + } else { + ret = base + (nmemb / 2); + } + + int result = compare (key, ret); + if (result < 0) { + nmemb /= 2; + } else if (result > 0) { + if constexpr (use_extra_size) { + base = reinterpret_cast(reinterpret_cast(ret) + size + extra_size); + } else { + base = ret + 1; + } + nmemb -= nmemb / 2 + 1; + } else { + return ret; + } + } + + return nullptr; } -const char* -EmbeddedAssemblies::typemap_java_to_managed (const char *java) +MonoReflectionType* +EmbeddedAssemblies::typemap_java_to_managed (MonoString *java_type) { + timing_period total_time; + if (XA_UNLIKELY (utils.should_log (LOG_TIMING))) { + timing = new Timing (); + total_time.mark_start (); + } + + if (XA_UNLIKELY (java_type == nullptr)) { + log_warn (LOG_ASSEMBLY, "typemap: null 'java_type' passed to 'typemap_java_to_managed'"); + return nullptr; + } + + simple_pointer_guard java_type_name (mono_string_to_utf8 (java_type)); + if (XA_UNLIKELY (!java_type_name || *java_type_name == '\0')) { + log_warn (LOG_ASSEMBLY, "typemap: empty Java type name passed to 'typemap_java_to_managed'"); + return nullptr; + } + + int32_t type_token_id = -1; + TypeMapModule *module; +#if defined (DEBUG) || !defined (ANDROID) + if (application_config.instant_run_enabled) { + size_t idx = 0; + for (; idx < module_count; idx++) { + const uint8_t *java_entry = binary_search (java_type_name.get (), modules[idx].java_map, modules[idx].entry_count, modules[idx].java_name_width + 3); + if (java_entry == nullptr) + continue; + type_token_id = *reinterpret_cast(java_entry + modules[idx].java_name_width); + break; + } + + if (idx >= module_count) { + log_error (LOG_ASSEMBLY, "typemap: unable to find module with Java type '%s' mapping", java_type_name.get ()); + return nullptr; + } + + module = &modules[idx]; + } else { +#endif + const TypeMapJava *java_entry = binary_search (java_type_name.get (), map_java, java_type_count, java_name_width); + if (java_entry == nullptr) { + log_warn (LOG_ASSEMBLY, "typemap: unable to find mapping to a managed type from Java type '%s'", java_type_name.get ()); + return nullptr; + } + + if (java_entry->module_index >= map_module_count) { + log_warn (LOG_ASSEMBLY, "typemap: mapping from Java type '%s' to managed type has invalid module index", java_type_name.get ()); + return nullptr; + } + + module = const_cast(&map_modules[java_entry->module_index]); + const TypeMapModuleEntry *entry = binary_search (&java_entry->type_token_id, module->map, module->entry_count); + if (entry == nullptr) { + log_warn (LOG_ASSEMBLY, "typemap: unable to find mapping from Java type '%s' to managed type with token ID %u in module [%s]", java_type_name.get (), java_entry->type_token_id, MonoGuidString (module->module_uuid).get ()); + return nullptr; + } + type_token_id = java_entry->type_token_id; #if defined (DEBUG) || !defined (ANDROID) - for (TypeMappingInfo *info = java_to_managed_maps; info != nullptr; info = info->next) { - /* log_warn (LOG_DEFAULT, "# jonp: checking file: %s!%s for type '%s'", info->source_apk, info->source_entry, java); */ - const char *e = reinterpret_cast (bsearch (java, info->mapping, static_cast(info->num_entries), static_cast(info->entry_length), TypeMappingInfo_compare_key)); - if (e == nullptr) - continue; - return e + info->value_offset; } #endif - return find_entry_in_type_map (java, jm_typemap, jm_typemap_header); + + if (module->image == nullptr) { + module->image = mono_image_loaded (module->assembly_name); + if (module->image == nullptr) { + // TODO: load + log_error (LOG_ASSEMBLY, "typemap: assembly '%s' not loaded yet!", module->assembly_name); + } + + if (module->image == nullptr) { + log_error (LOG_ASSEMBLY, "typemap: unable to load assembly '%s' when looking up managed type corresponding to Java type '%s'", module->assembly_name, java_type_name.get ()); + return nullptr; + } + } + + log_debug (LOG_ASSEMBLY, "typemap: java type '%s' corresponds to managed token id %u (0x%x)", java_type_name.get (), type_token_id, type_token_id); + MonoClass *klass = mono_class_get (module->image, static_cast(type_token_id)); + if (klass == nullptr) { + log_error (LOG_ASSEMBLY, "typemap: unable to find managed type with token ID %u in assembly '%s', corresponding to Java type '%s'", type_token_id, module->assembly_name, java_type_name.get ()); + return nullptr; + } + + MonoReflectionType *ret = mono_type_get_object (mono_domain_get (), mono_class_get_type (klass)); + if (XA_UNLIKELY (utils.should_log (LOG_TIMING))) { + total_time.mark_end (); + + Timing::info (total_time, "Typemap.java_to_managed: end, total time"); + } + + return ret; } -const char* -EmbeddedAssemblies::typemap_managed_to_java (const char *managed) +int +EmbeddedAssemblies::compare_java_name (const char *java_name, const TypeMapJava *entry) { -#if defined (DEBUG) || !defined (ANDROID) - for (TypeMappingInfo *info = managed_to_java_maps; info != nullptr; info = info->next) { - /* log_warn (LOG_DEFAULT, "# jonp: checking file: %s!%s for type '%s'", info->source_apk, info->source_entry, managed); */ - const char *e = reinterpret_cast (bsearch (managed, info->mapping, static_cast(info->num_entries), static_cast(info->entry_length), TypeMappingInfo_compare_key)); - if (e == nullptr) - continue; - return e + info->value_offset; + if (entry == nullptr || entry->java_name[0] == '\0') { + return -1; } -#endif - return find_entry_in_type_map (managed, mj_typemap, mj_typemap_header); + + return strcmp (java_name, reinterpret_cast(entry->java_name)); } #if defined (DEBUG) || !defined (ANDROID) -void -EmbeddedAssemblies::extract_int (const char **header, const char *source_apk, const char *source_entry, const char *key_name, int *value) +int +EmbeddedAssemblies::compare_java_name (const char *java_name, const uint8_t *entry) { - int read = 0; - int consumed = 0; - size_t key_name_len = 0; - char scanf_format [20] = { 0, }; + if (entry == nullptr) + return 1; - if (header == nullptr || *header == nullptr) - return; + return strcmp (java_name, reinterpret_cast(entry)); +} +#endif // DEBUG || !ANDROID - key_name_len = strlen (key_name); - if (key_name_len >= (sizeof (scanf_format) - sizeof ("=%d%n"))) { - *header = nullptr; - return; +const char* +EmbeddedAssemblies::typemap_managed_to_java (const uint8_t *mvid, const int32_t token) +{ + timing_period total_time; + if (XA_UNLIKELY (utils.should_log (LOG_TIMING))) { + timing = new Timing (); + total_time.mark_start (); } - snprintf (scanf_format, sizeof (scanf_format), "%s=%%d%%n", key_name); + if (mvid == nullptr) { + log_warn (LOG_ASSEMBLY, "typemap: no mvid specified in call to typemap_managed_to_java"); + return nullptr; + } - read = sscanf (*header, scanf_format, value, &consumed); - if (read != 1) { - log_warn (LOG_DEFAULT, "Could not read header '%s' value from '%s!%s': read %i elements, expected 1 element. Contents: '%s'", - key_name, source_apk, source_entry, read, *header); - *header = nullptr; - return; + const TypeMapModule *map; + size_t map_entry_count; +#if defined (DEBUG) || !defined (ANDROID) + if (application_config.instant_run_enabled) { + map = modules; + map_entry_count = module_count; + } else { +#endif + map = map_modules; + map_entry_count = map_module_count; +#if defined (DEBUG) || !defined (ANDROID) + } +#endif + const TypeMapModule *match = binary_search (mvid, map, map_entry_count); + if (match == nullptr) { + log_warn (LOG_ASSEMBLY, "typemap: module matching MVID [%s] not found.", MonoGuidString (mvid).get ()); + return nullptr; } - *header = *header + consumed + 1; -} -bool -EmbeddedAssemblies::add_type_mapping (TypeMappingInfo **info, const char *source_apk, const char *source_entry, const char *addr) -{ - TypeMappingInfo *p = new TypeMappingInfo (); // calloc (1, sizeof (struct TypeMappingInfo)); - int version = 0; - const char *data = addr; - - extract_int (&data, source_apk, source_entry, "version", &version); - if (version != 1) { - delete p; - log_warn (LOG_DEFAULT, "Unsupported version '%i' within type mapping file '%s!%s'. Ignoring...", version, source_apk, source_entry); - return false; + if (match->map == nullptr) { + log_warn (LOG_ASSEMBLY, "typemap: module with MVID [%s] has no associated type map.", MonoGuidString (mvid).get ()); + return nullptr; } - extract_int (&data, source_apk, source_entry, "entry-count", &p->num_entries); - extract_int (&data, source_apk, source_entry, "entry-len", &p->entry_length); - extract_int (&data, source_apk, source_entry, "value-offset", &p->value_offset); - p->mapping = data; + log_debug (LOG_ASSEMBLY, "typemap: MVID [%s] maps to assembly %s, looking for token %d (0x%x), table index %d", MonoGuidString (mvid).get (), match->assembly_name, token, token, token & 0x00FFFFFF); + // Each map entry is a pair of 32-bit integers: [TypeTokenID][JavaMapArrayIndex] + const TypeMapModuleEntry *entry = binary_search (&token, match->map, match->entry_count); + if (entry == nullptr) { + if (match->duplicate_count > 0 && match->duplicate_map != nullptr) { + log_debug (LOG_ASSEMBLY, "typemap: searching module [%s] duplicate map for token %u (0x%x)", MonoGuidString (mvid).get (), token, token); + entry = binary_search (&token, match->duplicate_map, match->duplicate_count); + } - if ((p->mapping == 0) || - (p->num_entries <= 0) || - (p->entry_length <= 0) || - (p->value_offset >= p->entry_length) || - (p->mapping == nullptr)) { - log_warn (LOG_DEFAULT, "Could not read type mapping file '%s!%s'. Ignoring...", source_apk, source_entry); - delete p; - return false; + if (entry == nullptr) { + log_warn (LOG_ASSEMBLY, "typemap: type with token %d (0x%x) in module {%s} (%s) not found.", token, token, MonoGuidString (mvid).get (), match->assembly_name); + return nullptr; + } } - p->source_apk = strdup (source_apk); - p->source_entry = strdup (source_entry); - if (*info) { - (*info)->next = p; + uint32_t java_entry_count; +#if defined (DEBUG) || !defined (ANDROID) + if (application_config.instant_run_enabled) { + java_entry_count = match->entry_count; } else { - *info = p; +#endif + java_entry_count = java_type_count; +#if defined (DEBUG) || !defined (ANDROID) } - return true; +#endif + if (entry->java_map_index >= java_entry_count) { + log_warn (LOG_ASSEMBLY, "typemap: type with token %d (0x%x) in module {%s} (%s) has invalid Java type index %u", token, token, MonoGuidString (mvid).get (), match->assembly_name, entry->java_map_index); + return nullptr; + } + + const char *ret; +#if defined (DEBUG) || !defined (ANDROID) + if (application_config.instant_run_enabled) { + ret = reinterpret_cast(match->java_map + ((match->java_name_width + 4) * entry->java_map_index)); + } else { +#endif + const TypeMapJava *java_entry = reinterpret_cast (reinterpret_cast(map_java) + ((sizeof(TypeMapJava) + java_name_width) * entry->java_map_index)); + ret = reinterpret_cast(reinterpret_cast(java_entry) + 8); +#if defined (DEBUG) || !defined (ANDROID) + } +#endif + + if (XA_UNLIKELY (ret == nullptr)) { + log_warn (LOG_ASSEMBLY, "typemap: empty Java type name returned for entry at index %u", entry->java_map_index); + } + + if (XA_UNLIKELY (utils.should_log (LOG_TIMING))) { + total_time.mark_end (); + + Timing::info (total_time, "Typemap.managed_to_java: end, total time"); + } + + log_debug ( + LOG_ASSEMBLY, + "typemap: type with token %d (0x%x) in module {%s} (%s) corresponds to Java type '%s'", + token, + token, + MonoGuidString (mvid).get (), + match->assembly_name, + ret + ); + + return ret; +} + +int +EmbeddedAssemblies::compare_type_token (const int32_t *token, const TypeMapModuleEntry *entry) +{ + if (entry == nullptr) { + log_fatal (LOG_ASSEMBLY, "typemap: compare_type_token: entry is nullptr"); + exit (FATAL_EXIT_MISSING_ASSEMBLY); + } + + return *token - entry->type_token_id; +} + +int +EmbeddedAssemblies::compare_mvid (const uint8_t *mvid, const TypeMapModule *module) +{ + return memcmp (mvid, module->module_uuid, sizeof(module->module_uuid)); } -#endif // DEBUG || !ANDROID EmbeddedAssemblies::md_mmap_info EmbeddedAssemblies::md_mmap_apk_file (int fd, uint32_t offset, uint32_t size, const char* filename, const char* apk) @@ -299,50 +474,270 @@ EmbeddedAssemblies::gather_bundled_assemblies_from_apk (const char* apk, monodro } #if defined (DEBUG) || !defined (ANDROID) +ssize_t EmbeddedAssemblies::do_read (int fd, void *buf, size_t count) +{ + ssize_t ret; + do { + ret = ::read (fd, buf, count); + } while (ret < 0 && errno == EINTR); + + return ret; +} + +template +bool +EmbeddedAssemblies::typemap_read_header ([[maybe_unused]] int dir_fd, const char *file_type, const char *dir_path, const char *file_path, uint32_t expected_magic, H &header, size_t &file_size, int &fd) +{ + struct stat sbuf; + int res; + +#if __ANDROID_API__ < 21 + simple_pointer_guard full_file_path = utils.path_combine (dir_path, file_path); + res = stat (full_file_path, &sbuf); +#else + res = fstatat (dir_fd, file_path, &sbuf, 0); +#endif + if (res < 0) { + log_error (LOG_ASSEMBLY, "typemap: failed to stat %s file '%s/%s': %s", file_type, dir_path, file_path, strerror (errno)); + return false; + } + + file_size = static_cast(sbuf.st_size); + if (file_size < sizeof (header)) { + log_error (LOG_ASSEMBLY, "typemap: %s file '%s/%s' is too small (must be at least %u bytes)", file_type, dir_path, file_path, sizeof (header)); + return false; + } + +#if __ANDROID_API__ < 21 + fd = open (full_file_path, O_RDONLY); +#else + fd = openat (dir_fd, file_path, O_RDONLY); +#endif + if (fd < 0) { + log_error (LOG_ASSEMBLY, "typemap: failed to open %s file %s/%s for reading: %s", file_type, dir_path, file_path, strerror (errno)); + return false; + } + + ssize_t nread = do_read (fd, &header, sizeof (header)); + if (nread <= 0) { + if (nread < 0) { + log_error (LOG_ASSEMBLY, "typemap: failed to read %s file header from '%s/%s': %s", file_type, dir_path, file_path, strerror (errno)); + } else { + log_error (LOG_ASSEMBLY, "typemap: end of file while reading %s file header from '%s/%s'", file_type, dir_path, file_path); + } + + return false; + } + + if (header.magic != expected_magic) { + log_error (LOG_ASSEMBLY, "typemap: invalid magic value in the %s file header from '%s/%s': expected 0x%X, got 0x%X", file_type, dir_path, file_path, expected_magic, header.magic); + return false; + } + + if (header.version != MODULE_FORMAT_VERSION) { + log_error (LOG_ASSEMBLY, "typemap: incompatible %s format version. This build supports only version %u, file '%s/%s' uses version %u", file_type, MODULE_FORMAT_VERSION, dir_path, file_path, header.version); + return false; + } + + return true; +} + +uint8_t* +EmbeddedAssemblies::typemap_load_index (TypeMapIndexHeader &header, size_t file_size, int index_fd) +{ + constexpr size_t UUID_SIZE = 16; + + size_t entry_size = header.module_file_name_width + UUID_SIZE; + size_t data_size = entry_size * module_count; + if (sizeof(header) + data_size > file_size) { + log_error (LOG_ASSEMBLY, "typemap: index file is too small, expected %u, found %u bytes", data_size + sizeof(header), file_size); + return nullptr; + } + + auto data = new uint8_t [data_size]; + ssize_t nread = do_read (index_fd, data, data_size); + if (nread != static_cast(data_size)) { + log_error (LOG_ASSEMBLY, "typemap: failed to read %u bytes from index file. %s", data_size, strerror (errno)); + return nullptr; + } + + uint8_t *p = data; + for (size_t i = 0; i < module_count; i++) { + memcpy (modules[i].module_uuid, p, UUID_SIZE); + modules[i].assembly_name = reinterpret_cast(p + UUID_SIZE); + p += entry_size; + } + + return data; +} + +uint8_t* +EmbeddedAssemblies::typemap_load_index (int dir_fd, const char *dir_path, const char *index_path) +{ + log_debug (LOG_ASSEMBLY, "typemap: loading TypeMap index file '%s/%s'", dir_path, index_path); + + TypeMapIndexHeader header; + size_t file_size; + int fd = -1; + uint8_t *data = nullptr; + + if (!typemap_read_header (dir_fd, "TypeMap index", dir_path, index_path, MODULE_INDEX_MAGIC, header, file_size, fd)) { + goto cleanup; + } + + module_count = header.entry_count; + modules = new TypeMapModule[module_count](); + data = typemap_load_index (header, file_size, fd); + + cleanup: + if (fd >= 0) + close (fd); + + return data; +} + +bool +EmbeddedAssemblies::typemap_load_file (BinaryTypeMapHeader &header, const char *dir_path, const char *file_path, int file_fd, TypeMapModule &module) +{ + size_t alloc_size = ADD_WITH_OVERFLOW_CHECK (size_t, header.assembly_name_length, 1); + module.assembly_name = new char[alloc_size]; + + ssize_t nread = do_read (file_fd, module.assembly_name, header.assembly_name_length); + module.assembly_name [header.assembly_name_length] = 0; + module.entry_count = header.entry_count; + + log_debug ( + LOG_ASSEMBLY, + "typemap: '%s/%s':: entry count == %u; duplicate entry count == %u; Java type name field width == %u; MVID == %s; assembly name length == %u; assembly name == %s", + dir_path, file_path, header.entry_count, header.duplicate_count, header.java_name_width, + MonoGuidString (header.module_uuid).get (), header.assembly_name_length, module.assembly_name + ); + + alloc_size = MULTIPLY_WITH_OVERFLOW_CHECK (size_t, header.java_name_width + 4, header.entry_count); + module.java_name_width = header.java_name_width; + module.java_map = new uint8_t[alloc_size]; + nread = do_read (file_fd, module.java_map, alloc_size); + if (nread != static_cast(alloc_size)) { + log_error (LOG_ASSEMBLY, "typemap: failed to read %u bytes (java-to-managed) from module file %s/%s. %s", alloc_size, dir_path, file_path, strerror (errno)); + return false; + } + + module.map = new TypeMapModuleEntry[header.entry_count]; + alloc_size = MULTIPLY_WITH_OVERFLOW_CHECK (size_t, sizeof(TypeMapModuleEntry), header.entry_count); + nread = do_read (file_fd, module.map, alloc_size); + if (nread != static_cast(alloc_size)) { + log_error (LOG_ASSEMBLY, "typemap: failed to read %u bytes (managed-to-java) from module file %s/%s. %s", alloc_size, dir_path, file_path, strerror (errno)); + return false; + } + + // alloc_size = module.java_name_width + 1; + // auto chars = new char[alloc_size](); + // uint8_t *p = module.java_map; + // log_debug (LOG_ASSEMBLY, "Java entries in %s/%s", dir_path, file_path); + // for (size_t i = 0; i < module.entry_count; i++) { + // memcpy (chars, p, module.java_name_width); + // uint32_t token = *reinterpret_cast(p + module.java_name_width); + // log_debug (LOG_ASSEMBLY, " %04u: %s; %u (0x%x)", i, chars, token, token); + // p += module.java_name_width + 4; + // } + // delete[] chars; + + // log_debug (LOG_ASSEMBLY, "Managed entries in %s/%s", dir_path, file_path); + // for (size_t i = 0; i < module.entry_count; i++) { + // log_debug (LOG_ASSEMBLY, " %04u: token %u (0x%x); index %u", i, module.map[i].type_token_id, module.map[i].type_token_id, module.map[i].java_map_index); + // } + + if (header.duplicate_count == 0) + return true; + + module.duplicate_map = new TypeMapModuleEntry[header.duplicate_count]; + alloc_size = MULTIPLY_WITH_OVERFLOW_CHECK (size_t, sizeof(TypeMapModuleEntry), header.duplicate_count); + nread = do_read (file_fd, module.duplicate_map, alloc_size); + if (nread != static_cast(alloc_size)) { + log_error (LOG_ASSEMBLY, "typemap: failed to read %u bytes (managed-to-java duplicates) from module file %s/%s. %s", alloc_size, dir_path, file_path, strerror (errno)); + return false; + } + + return true; +} + +bool +EmbeddedAssemblies::typemap_load_file (int dir_fd, const char *dir_path, const char *file_path, TypeMapModule &module) +{ + log_debug (LOG_ASSEMBLY, "typemap: loading TypeMap file '%s/%s'", dir_path, file_path); + + bool ret = true; + BinaryTypeMapHeader header; + size_t file_size; + int fd = -1; + + module.java_map = nullptr; + module.map = nullptr; + module.duplicate_map = nullptr; + + if (!typemap_read_header (dir_fd, "TypeMap", dir_path, file_path, MODULE_MAGIC, header, file_size, fd)) { + ret = false; + goto cleanup; + } + + ret = typemap_load_file (header, dir_path, file_path, fd, module); + + cleanup: + if (fd >= 0) + close (fd); + + if (!ret) { + delete[] module.java_map; + module.java_map = nullptr; + delete[] module.map; + module.map = nullptr; + delete[] module.duplicate_map; + module.duplicate_map = nullptr; + } + + return ret; +} + void EmbeddedAssemblies::try_load_typemaps_from_directory (const char *path) { - // read the entire typemap file into a string - // process the string using the add_type_mapping - char *dir_path = utils.path_combine (path, "typemaps"); - if (dir_path == nullptr || !utils.directory_exists (dir_path)) { - log_warn (LOG_DEFAULT, "directory does not exist: `%s`", dir_path); - free (dir_path); + if (!application_config.instant_run_enabled) { + log_info (LOG_ASSEMBLY, "typemap: instant run disabled, not loading type maps from storage"); return; } + simple_pointer_guard dir_path (utils.path_combine (path, "typemaps")); monodroid_dir_t *dir; if ((dir = utils.monodroid_opendir (dir_path)) == nullptr) { - log_warn (LOG_DEFAULT, "could not open directory: `%s`", dir_path); - free (dir_path); + log_warn (LOG_DEFAULT, "typemap: could not open directory: `%s`", dir_path.get ()); return; } - monodroid_dirent_t *e; - while ((e = androidSystem.readdir (dir)) != nullptr) { -#if WINDOWS - char *file_name = utils.utf16_to_utf8 (e->d_name); -#else /* def WINDOWS */ - char *file_name = e->d_name; -#endif /* ndef WINDOWS */ - char *file_path = utils.path_combine (dir_path, file_name); - if (utils.monodroid_dirent_hasextension (e, ".mj") || utils.monodroid_dirent_hasextension (e, ".jm")) { - char *val = nullptr; - size_t len = androidSystem.monodroid_read_file_into_memory (file_path, val); - if (len > 0 && val != nullptr) { - if (utils.monodroid_dirent_hasextension (e, ".mj")) { - if (!add_type_mapping (&managed_to_java_maps, file_path, override_typemap_entry_name, ((const char*)val))) - delete[] val; - } else if (utils.monodroid_dirent_hasextension (e, ".jm")) { - if (!add_type_mapping (&java_to_managed_maps, file_path, override_typemap_entry_name, ((const char*)val))) - delete[] val; - } - } + int dir_fd; +#if __ANDROID_API__ < 21 + dir_fd = -1; +#else + dir_fd = dirfd (dir); +#endif + + constexpr char index_name[] = "typemap.index"; + + // The pointer must be stored here because, after index is loaded, module.assembly_name points into the index data + // and must be valid until after the actual module file is loaded. + simple_pointer_guard index_data = typemap_load_index (dir_fd, dir_path, index_name); + if (!index_data) { + log_fatal (LOG_ASSEMBLY, "typemap: unable to load TypeMap data index from '%s/%s'", dir_path.get (), index_name); + exit (FATAL_EXIT_NO_ASSEMBLIES); // TODO: use a new error code here + } + + for (size_t i = 0; i < module_count; i++) { + TypeMapModule &module = modules[i]; + if (!typemap_load_file (dir_fd, dir_path, module.assembly_name, module)) { + continue; } } + utils.monodroid_closedir (dir); - free (dir_path); - return; } #endif diff --git a/src/monodroid/jni/embedded-assemblies.hh b/src/monodroid/jni/embedded-assemblies.hh index 1f3a024c960..f000bab413b 100644 --- a/src/monodroid/jni/embedded-assemblies.hh +++ b/src/monodroid/jni/embedded-assemblies.hh @@ -3,8 +3,11 @@ #define INC_MONODROID_EMBEDDED_ASSEMBLIES_H #include +#include #include +#include "xamarin-app.hh" + struct TypeMapHeader; namespace xamarin::android::internal { @@ -39,8 +42,8 @@ namespace xamarin::android::internal { void try_load_typemaps_from_directory (const char *path); #endif void install_preload_hooks (); - const char* typemap_java_to_managed (const char *java); - const char* typemap_managed_to_java (const char *managed); + MonoReflectionType* typemap_java_to_managed (MonoString *java_type); + const char* typemap_managed_to_java (const uint8_t *mvid, const int32_t token); /* returns current number of *all* assemblies found from all invocations */ template @@ -66,9 +69,14 @@ namespace xamarin::android::internal { size_t register_from (const char *apk_file, monodroid_should_register should_register); void gather_bundled_assemblies_from_apk (const char* apk, monodroid_should_register should_register); MonoAssembly* open_from_bundles (MonoAssemblyName* aname, bool ref_only); - void extract_int (const char **header, const char *source_apk, const char *source_entry, const char *key_name, int *value); #if defined (DEBUG) || !defined (ANDROID) - bool add_type_mapping (TypeMappingInfo **info, const char *source_apk, const char *source_entry, const char *addr); + template + bool typemap_read_header (int dir_fd, const char *file_type, const char *dir_path, const char *file_path, uint32_t expected_magic, H &header, size_t &file_size, int &fd); + uint8_t* typemap_load_index (int dir_fd, const char *dir_path, const char *index_path); + uint8_t* typemap_load_index (TypeMapIndexHeader &header, size_t file_size, int index_fd); + bool typemap_load_file (int dir_fd, const char *dir_path, const char *file_path, TypeMapModule &module); + bool typemap_load_file (BinaryTypeMapHeader &header, const char *dir_path, const char *file_path, int file_fd, TypeMapModule &module); + static ssize_t do_read (int fd, void *buf, size_t count); #endif // DEBUG || !ANDROID bool register_debug_symbols_for_assembly (const char *entry_name, MonoBundledAssembly *assembly, const mono_byte *debug_contents, int debug_size); @@ -76,8 +84,6 @@ namespace xamarin::android::internal { static MonoAssembly* open_from_bundles_full (MonoAssemblyName *aname, char **assemblies_path, void *user_data); static MonoAssembly* open_from_bundles_refonly (MonoAssemblyName *aname, char **assemblies_path, void *user_data); - static int TypeMappingInfo_compare_key (const void *a, const void *b); - const char *find_entry_in_type_map (const char *name, uint8_t map[], TypeMapHeader& header); void zip_load_entries (int fd, const char *apk_name, monodroid_should_register should_register); bool zip_read_cd_info (int fd, uint32_t& cd_offset, uint32_t& cd_size, uint16_t& cd_entries); @@ -95,6 +101,16 @@ namespace xamarin::android::internal { return assemblies_prefix_override != nullptr ? assemblies_prefix_override : assemblies_prefix; } + template + const Entry* binary_search (const Key *key, const Entry *base, size_t nmemb, size_t extra_size = 0); + + static int compare_mvid (const uint8_t *mvid, const TypeMapModule *module); + static int compare_type_token (const int32_t *token, const TypeMapModuleEntry *entry); + static int compare_java_name (const char *java_name, const TypeMapJava *entry); +#if defined (DEBUG) || !defined (ANDROID) + static int compare_java_name (const char *java_name, const uint8_t *java_map); +#endif + private: bool register_debug_symbols; MonoBundledAssembly **bundled_assemblies = nullptr; @@ -102,6 +118,8 @@ namespace xamarin::android::internal { #if defined (DEBUG) || !defined (ANDROID) TypeMappingInfo *java_to_managed_maps; TypeMappingInfo *managed_to_java_maps; + TypeMapModule *modules; + size_t module_count; #endif // DEBUG || !ANDROID const char *assemblies_prefix_override = nullptr; }; diff --git a/src/monodroid/jni/external-api.cc b/src/monodroid/jni/external-api.cc index 49e24da3d79..c49a58308af 100644 --- a/src/monodroid/jni/external-api.cc +++ b/src/monodroid/jni/external-api.cc @@ -159,15 +159,9 @@ _monodroid_get_display_dpi (float *x_dpi, float *y_dpi) } MONO_API const char * -monodroid_typemap_java_to_managed (const char *java) +monodroid_typemap_managed_to_java (const uint8_t *mvid, const int32_t token) { - return embeddedAssemblies.typemap_java_to_managed (java); -} - -MONO_API const char * -monodroid_typemap_managed_to_java (const char *managed) -{ - return embeddedAssemblies.typemap_managed_to_java (managed); + return embeddedAssemblies.typemap_managed_to_java (mvid, token); } MONO_API int monodroid_embedded_assemblies_set_assemblies_prefix (const char *prefix) diff --git a/src/monodroid/jni/monodroid-glue-internal.hh b/src/monodroid/jni/monodroid-glue-internal.hh index 390b97d815a..c4afbfc8e1a 100644 --- a/src/monodroid/jni/monodroid-glue-internal.hh +++ b/src/monodroid/jni/monodroid-glue-internal.hh @@ -175,6 +175,7 @@ namespace xamarin::android::internal static void jit_done (MonoProfiler *prof, MonoMethod *method, MonoJitInfo* jinfo); static void thread_start (MonoProfiler *prof, uintptr_t tid); static void thread_end (MonoProfiler *prof, uintptr_t tid); + static MonoReflectionType* typemap_java_to_managed (MonoString *java_type_name); #if defined (DEBUG) void set_debug_env_vars (void); diff --git a/src/monodroid/jni/monodroid-glue.cc b/src/monodroid/jni/monodroid-glue.cc index 79f2fe950e1..294fbb99125 100644 --- a/src/monodroid/jni/monodroid-glue.cc +++ b/src/monodroid/jni/monodroid-glue.cc @@ -68,7 +68,7 @@ #include "mkbundle-api.h" #include "monodroid-glue-internal.hh" #include "globals.hh" -#include "xamarin-app.h" +#include "xamarin-app.hh" #include "timing.hh" #ifndef WINDOWS @@ -345,12 +345,14 @@ inline void MonodroidRuntime::gather_bundled_assemblies (jstring_array_wrapper &runtimeApks, size_t *out_user_assemblies_count) { #if defined(DEBUG) || !defined (ANDROID) - for (size_t i = 0; i < AndroidSystem::MAX_OVERRIDES; ++i) { - const char *p = androidSystem.get_override_dir (i); - if (!utils.directory_exists (p)) - continue; - log_info (LOG_ASSEMBLY, "Loading TypeMaps from %s", p); - embeddedAssemblies.try_load_typemaps_from_directory (p); + if (application_config.instant_run_enabled) { + for (size_t i = 0; i < AndroidSystem::MAX_OVERRIDES; ++i) { + const char *p = androidSystem.get_override_dir (i); + if (!utils.directory_exists (p)) + continue; + log_info (LOG_ASSEMBLY, "Loading TypeMaps from %s", p); + embeddedAssemblies.try_load_typemaps_from_directory (p); + } } #endif @@ -918,6 +920,8 @@ MonodroidRuntime::lookup_bridge_info (MonoDomain *domain, MonoImage *image, cons void MonodroidRuntime::init_android_runtime (MonoDomain *domain, JNIEnv *env, jclass runtimeClass, jobject loader) { + mono_add_internal_call ("Java.Interop.TypeManager::monodroid_typemap_java_to_managed", reinterpret_cast(typemap_java_to_managed)); + struct JnienvInitializeArgs init = {}; init.javaVm = osBridge.get_jvm (); init.env = env; @@ -1400,6 +1404,12 @@ MonodroidRuntime::create_and_initialize_domain (JNIEnv* env, jclass runtimeClass return domain; } +MonoReflectionType* +MonodroidRuntime::typemap_java_to_managed (MonoString *java_type_name) +{ + return embeddedAssemblies.typemap_java_to_managed (java_type_name); +} + inline void MonodroidRuntime::Java_mono_android_Runtime_initInternal (JNIEnv *env, jclass klass, jstring lang, jobjectArray runtimeApksJava, jstring runtimeNativeLibDir, jobjectArray appDirs, jobject loader, diff --git a/src/monodroid/jni/xamarin-app.h b/src/monodroid/jni/xamarin-app.h deleted file mode 100644 index 96812303a4e..00000000000 --- a/src/monodroid/jni/xamarin-app.h +++ /dev/null @@ -1,42 +0,0 @@ -// Dear Emacs, this is a -*- C++ -*- header -#ifndef __XAMARIN_ANDROID_TYPEMAP_H -#define __XAMARIN_ANDROID_TYPEMAP_H - -#include - -#include "monodroid.h" - -struct TypeMapHeader -{ - uint32_t version; - uint32_t entry_count; - uint32_t entry_length; - uint32_t value_offset; -}; - -struct ApplicationConfig -{ - bool uses_mono_llvm; - bool uses_mono_aot; - bool uses_assembly_preload; - bool is_a_bundled_app; - bool broken_exception_transitions; - uint8_t bound_exception_type; - uint32_t package_naming_policy; - uint32_t environment_variable_count; - uint32_t system_property_count; - const char *android_package_name; -}; - -MONO_API TypeMapHeader jm_typemap_header; -MONO_API uint8_t jm_typemap[]; - -MONO_API TypeMapHeader mj_typemap_header; -MONO_API uint8_t mj_typemap[]; - -MONO_API ApplicationConfig application_config; -MONO_API const char* app_environment_variables[]; -MONO_API const char* app_system_properties[]; - -MONO_API const char* mono_aot_mode_name; -#endif // __XAMARIN_ANDROID_TYPEMAP_H diff --git a/src/monodroid/jni/xamarin-app.hh b/src/monodroid/jni/xamarin-app.hh new file mode 100644 index 00000000000..7f71a0cf1b5 --- /dev/null +++ b/src/monodroid/jni/xamarin-app.hh @@ -0,0 +1,86 @@ +// Dear Emacs, this is a -*- C++ -*- header +#ifndef __XAMARIN_ANDROID_TYPEMAP_H +#define __XAMARIN_ANDROID_TYPEMAP_H + +#include + +#include + +#include "monodroid.h" + +static constexpr uint32_t MODULE_MAGIC = 0x4D544158; // 'XATM', little-endian +static constexpr uint32_t MODULE_INDEX_MAGIC = 0x49544158; // 'XATI', little-endian +static constexpr uint8_t MODULE_FORMAT_VERSION = 1; // Keep in sync with the value in src/Xamarin.Android.Build.Tasks/Utilities/TypeMapGenerator.cs + +struct BinaryTypeMapHeader +{ + uint32_t magic; + uint32_t version; + uint8_t module_uuid[16]; + uint32_t entry_count; + uint32_t duplicate_count; + uint32_t java_name_width; + uint32_t assembly_name_length; +}; + +struct TypeMapIndexHeader +{ + uint32_t magic; + uint32_t version; + uint32_t entry_count; + uint32_t module_file_name_width; +}; + +struct TypeMapModuleEntry +{ + int32_t type_token_id; + uint32_t java_map_index; +}; + +struct TypeMapModule +{ + uint8_t module_uuid[16]; + uint32_t entry_count; + uint32_t duplicate_count; + TypeMapModuleEntry *map; + TypeMapModuleEntry *duplicate_map; + char *assembly_name; + MonoImage *image; + uint32_t java_name_width; + uint8_t *java_map; +}; + +struct TypeMapJava +{ + uint32_t module_index; + int32_t type_token_id; + uint8_t java_name[]; +}; + +struct ApplicationConfig +{ + bool uses_mono_llvm; + bool uses_mono_aot; + bool uses_assembly_preload; + bool is_a_bundled_app; + bool broken_exception_transitions; + bool instant_run_enabled; + uint8_t bound_exception_type; + uint32_t package_naming_policy; + uint32_t environment_variable_count; + uint32_t system_property_count; + const char *android_package_name; +}; + +MONO_API const uint32_t map_module_count; +MONO_API const uint32_t java_type_count; +MONO_API const uint32_t java_name_width; +MONO_API const TypeMapModule map_modules[]; +MONO_API const TypeMapJava map_java[]; + +MONO_API ApplicationConfig application_config; +MONO_API const char* app_environment_variables[]; +MONO_API const char* app_system_properties[]; + +MONO_API const char* mono_aot_mode_name; +#endif // __XAMARIN_ANDROID_TYPEMAP_H diff --git a/tests/MSBuildDeviceIntegration/Tests/InstantRunTest.cs b/tests/MSBuildDeviceIntegration/Tests/InstantRunTest.cs index e62c706518b..6dea9d2dda6 100644 --- a/tests/MSBuildDeviceIntegration/Tests/InstantRunTest.cs +++ b/tests/MSBuildDeviceIntegration/Tests/InstantRunTest.cs @@ -285,15 +285,14 @@ public void InstantRunFastDevTypemaps ([Values ("dx", "d8")] string dexTool) Assert.IsTrue (b.Install (proj), "packaging should have succeeded. 0"); var apk = Path.Combine (Root, b.ProjectDirectory, proj.IntermediateOutputPath, "android", "bin", "UnnamedProject.UnnamedProject.apk"); - Assert.IsNull (ZipHelper.ReadFileFromZip (apk, "typemap.jm"), $"typemap.jm should NOT be in {apk}."); - Assert.IsNull (ZipHelper.ReadFileFromZip (apk, "typemap.mj"), $"typemap.mj should NOT be in {apk}."); + Assert.IsNull (ZipHelper.ReadFileFromZip (apk, "MonoAndroid.0.typemap"), $"MonoAndroid.0.typemap should NOT be in {apk}."); var logLines = b.LastBuildOutput; Assert.IsTrue (logLines.Any (l => l.Contains ("Building target \"_BuildApkFastDev\" completely.") || l.Contains ("Target _BuildApkFastDev needs to be built")), "Apk should have been built"); Assert.IsTrue (logLines.Any (l => l.Contains ("Building target \"_Upload\" completely")), "_Upload target should have run"); - Assert.IsTrue (logLines.Any (l => l.Contains ("NotifySync CopyFile") && l.Contains ("typemap.jm")), "typemap.jm should have been uploaded"); - Assert.IsTrue (logLines.Any (l => l.Contains ("NotifySync CopyFile") && l.Contains ("typemap.mj")), "typemap.mj should have been uploaded"); + Assert.IsTrue (logLines.Any (l => l.Contains ("NotifySync CopyFile") && l.Contains ("Mono.Android.0.typemap")), "Mono.Android.0.typemap should have been uploaded"); + Assert.IsTrue (logLines.Any (l => l.Contains ("NotifySync CopyFile") && l.Contains ("typemap.index")), "typemap.index should have been uploaded"); } } diff --git a/tests/Runtime-AppBundle/Mono.Android-TestsAppBundle.csproj b/tests/Runtime-AppBundle/Mono.Android-TestsAppBundle.csproj index 793fa8b7d00..f087d605d4e 100644 --- a/tests/Runtime-AppBundle/Mono.Android-TestsAppBundle.csproj +++ b/tests/Runtime-AppBundle/Mono.Android-TestsAppBundle.csproj @@ -21,6 +21,8 @@ d8 aab <_MonoAndroidTest>..\..\src\Mono.Android\Test\ + true + ..\..\product.snk diff --git a/tests/Runtime-MultiDex/Mono.Android-TestsMultiDex.csproj b/tests/Runtime-MultiDex/Mono.Android-TestsMultiDex.csproj index 39810986789..a8306f81f2e 100644 --- a/tests/Runtime-MultiDex/Mono.Android-TestsMultiDex.csproj +++ b/tests/Runtime-MultiDex/Mono.Android-TestsMultiDex.csproj @@ -20,6 +20,8 @@ False true d8 + true + ..\..\product.snk diff --git a/tests/locales/Xamarin.Android.Locale-Tests/Environment.txt b/tests/locales/Xamarin.Android.Locale-Tests/Environment.txt index 429f2b065cc..0ebab922566 100644 --- a/tests/locales/Xamarin.Android.Locale-Tests/Environment.txt +++ b/tests/locales/Xamarin.Android.Locale-Tests/Environment.txt @@ -1,4 +1,9 @@ +debug.mono.debug=1 +debug.mono.log=default,timing=bare +MONO_LOG_LEVEL=debug +MONO_LOG_MASK=asm +MONO_XDEBUG=1 + # Force runtime to use fr-FR locale instead of default locale. LANG=fr-FR __XA_USE_JAVA_DEFAULT_TIMEZONE_ID__=true - diff --git a/tests/msbuild-times-reference/MSBuildDeviceIntegration.csv b/tests/msbuild-times-reference/MSBuildDeviceIntegration.csv index 1bc3ab89ab5..4f19d360e8f 100644 --- a/tests/msbuild-times-reference/MSBuildDeviceIntegration.csv +++ b/tests/msbuild-times-reference/MSBuildDeviceIntegration.csv @@ -3,11 +3,11 @@ Test Name,Time in ms (int) # Data Build_No_Changes,3350 -Build_CSharp_Change,4100 +Build_CSharp_Change,4500 Build_AndroidResource_Change,4250 -Build_AndroidManifest_Change,4250 +Build_AndroidManifest_Change,4500 Build_Designer_Change,3750 -Build_JLO_Change,8150 +Build_JLO_Change,8700 Build_CSProj_Change,9800 Build_XAML_Change,9600 Build_XAML_Change_RefAssembly,6350