Make all assemblies RID-specific

[grendello]

This PR modifies our build in the way that all the assemblies which form the
application are divided into separate batches, each of them corresponding to
one enabled target architecture (ABI) of the application. This applies to
all the assemblies, including the satellite ones.

Previously, only a handful of assemblies (e.g. System.Private.CoreLib) were
treated as architecture-specific from the start, with a handful others made
architecture-specific by the managed linker/trimmer. The trouble with the latter
set is that neither the linker nor us really know whether or not an assembly actually
is architecture-specific. We began to suspect this known unknown might be a cause
of mysterious run-time crashes, as the MonoVM runtime cannot and will not check whether
a given assembly is specific to the current architecture on which the application is
running. Assemblies produced by the linker (and also those which are specific to
some architecture from the start) are identical to the architecture-agnostic ones on
the file level. The only way we were able to differentiate between them after linking,
was through comparison of the assembly MVID UUID hashes. Based on this difference,
we would pre-process the set of referenced assemblies and assign it to either the
architecture-agnostic group (when all the copies of the assembly had the same MVID), or
to the architecture-specific one (when any of the copies of the assembly had a different
MVID), thus de-duplicating the input assemblies while trying to keep the ones specific
to any architecture in separate copies.

With this PR, the pre-processing changes so that every assembly ends up in every target
architecture batch, regardless of whether its MVID differs to its berthern or not. This
is done very early in the build process on our side, where we make sure that each assembly
either has the Abi metadata or is given one, and is placed in the corresponding batch.
Further processing of those batches is "parallel", in that no code attempts to de-duplicate
the batches.

With the strict division of assemblies into architecture-specific groups, we also figured that
we can make the resulting application packages smaller, by placing all the assemblies (and their
associated files, like debug data or configs) in the lib/{ARCH} subdirectory of the APK/AAB
archive. This applies both to discrete assemblies (.dll, .pdb and .config files) as well
as to assembly stores. All of them used to be placed in a custom subdirectory of the APK archive,
called assemblies, which had the effect of placing architecture-specific assemblies in APKs
which wouldn't load them because of running on a different architecture, thus making them just
uselessly occupy storage space. This also applied to the runtime config binary blob file.

Assembly stores also underwent a format change to accommodate the changes, and in effect we also
now have just a single store per architecture, instead of two.

Putting all those files in lib is not without its challenges, however. Firstly, Android only
allows the .so (Linux ELF shared library) files to be placed in that directory. It will not
validate whether the file is actually a shared library, but it will insist on it following a
certain naming pattern: the file must start with lib, must not have any characters in its name
outside the alphanumeric, underscore, dash and dot and must be placed in the lib/{ARCH} directory,
without any subdirectories. On top of those restrictions, we also must take a lot of care not
to clash with the names of "external" shared libraries, including the ones produced by Mono AOT.

These restrictions require us to come up with a way to "mangle" our assembly etc entries, not only
to avoid the clashes, but also to be able to put the satellite assemblies in the lib/{ARCH}/culture/
subdirectories. The naming scheme implemented here is as follows:

• Regular assemblies: lib_ASSEMBLY_NAME.dll.so
• Satellite assemblies: lib-CULTURE-ASSEMBLY_NAME.{dll,config,pdb}.so
• Runtime config blob: libarc.bin.so (the a is prepended to try to put it in the front of
  the archive listing, so that at runtime we can finish scanning for files earlier)
• Assembly store: libassemblies.ARCH.blob.so

The _ and - assembly name markers were chosen so that they are compliant with the character set
restrictions, but also make it easy to differentiate at the run time between the two kinds and to
ignore "irrelevant" entries quickly.

The changes implemented here also required us to place all the assemblies in new locations on disk,
while building the application. They are now placed in subdirectories named after either the
target architecture/ABI or the .NET RID. (e.g. obj/Release/netX.Y-android/android-arm64). This, in
turn, affects e.g. FastDev as now the synchronized content is in the .../.__override__/{ARCH} directory
on device, instead of just in .../.__override__/

[jonpryor]

We want/need to create a dir in Release; where else should methods.txt and counters.txt go? I can't imagine that we'd want to remove support for methods.txt & counters.txt, not while MonoVM is still used.

[jonathanpeppers]

Yes, I think methods.txt would be used in Release mode by jit-times:

• https://github.com/xamarin/xamarin-android/blob/main/Documentation/guides/profiling.md#profiling-the-jit-compiler

[grendello]

The TODO should have the word "always" - it makes sense to create the dir when methods.txt or counters.txt are to be created, which depends on more strict conditions than currently used in the function - it checks only for log_categories == 0, which is very broad. That was the idea behind the TODO

[jonpryor]

Is it at all possible for strlen(entry_name.get()) to be shorter than strlen(state.prefix)? I'm not sure I like the use of memcmp() here. Slightly similar, is it at all possible for entry_name to ever be the empty string, thus causing entry_name.get()[0] to be an invalid memory access?

[jonpryor]

Isn't this just a pointer comparison?

I'm going to have to track what state.prefix is, aren't I…

[grendello]

They can be the same pointer, see here.

If they're the same location, we short circuit the checking and avoid some comparisons in the next if statement.

[jonpryor]

Should this be an assert of some form? Plus other asserts at e.g. startup (somehow)?

[grendello]

No, it's just a warning that things are becoming slower than necessary. We can recover from the condition, no need to abort.

[jonpryor]

".NET Android", not Xamarin.Android.

[jonpryor]

This would (presumably) fix: #8168

[jonpryor]

Draft commit message:

Fixes: https://github.com/xamarin/xamarin-android/issues/8168

Context: https://github.com/xamarin/xamarin-android/issues/4337
Context: https://github.com/xamarin/xamarin-android/issues/8155

Context: 55e5c349d2ba86d53a4006f73746b0ad29df14c6
Context: 68368189d67c46ddbfed4e90e622f635c4aff11e
Context: 929e7012410233e6814af369db582f238ba185ad
Context: c92702619f5fabcff0ed88e09160baf9edd70f41
Context: 2f192386e8072f8e0ecaf0de2fe48654f3ade423

Issue xamarin/xamarin-android#8155 noted a *fundamental* mismatch in
expectations between the Classic Xamarin.Android packaging worldview
and the .NET worldview: In Classic Xamarin.Android, all assemblies
are presumed to be architecture agnostic ("AnyCPU"), while in .NET:

 1. `System.Private.CoreLib.dll` was *always* an architecture-specific
    assembly (see xamarin/xamarin-android#4337), and

 2. The .NET Trimmer is extensible and can apply ABI-specific changes
    to IL which *effectively* results in an architecture-specific
    assembly (xamarin/xamarin-android#8155).  Meanwhile, there is no
    way of knowing that this is happening, and the trimmer doesn't
    mark the resulting assembly as architecture-specific.

We long tried to "paper over" this difference, by trying to find --
and preserve the "nature" of -- architecture-agnostic assemblies
(55e5c349, …).  Unfortunately, all attempts at trying to preserve the
concept of architecture-agnostic assemblies have failed; we're
fighting against .NET tooling in attempting to do so.

In commit 68368189 this came to a head: a long worked-on feature
LLVM Marshal Methods (8bc7a3e8) had to be disabled because of
hangs within MAUI+Blazor Hybrid+.NET Android apps, and we suspect
that treating an assembly as architecture-agnostic when it was
"actually" architecture-specific is a plausible culprit.

Bite the bullet: there is no longer such a thing as an architecture-
agnostic assembly.  Treat *all* assemblies as if they were
architecture-specific.

Additionally, alter assembly packaging so that instead of using
`assemblies/assemblies*.blob` files (c9270261), we instead store the
assemblies within `lib/ABI` of the `.apk`/`.aab`.

The Runtime config blob `rc.bin` is stored as `lib/ABI/libarc.bin.so`.

When `$(AndroidUseAssemblyStore)`=true, assemblies will be stored
within `lib/ABI/libassemblies.ABI.blob.so`, e.g.
`lib/arm64-v8a/libassemblies.arm64-v8a.blob.so`.

When `$(AndroidUseAssemblyStore)`=false and Fast Deployment is *not*
used, then assemblies are stored individually within `lib/ABI` as
compressed assembly data, with the following "name mangling"
convention:

  * Regular assemblies: `lib_` + Assembly File Name + `.so`
  * Satellite assemblies:
    `lib-` + culture + `-` + Assembly File Name + `.so`

For example, consider this selected `unzip -l` output:

	% unzip -l bin/Release/net9.0-android/*-Signed.apk | grep lib/arm64-v8a
	   723560  01-01-1981 01:01   lib/arm64-v8a/libSystem.IO.Compression.Native.so
	    70843  01-01-1981 01:01   lib/arm64-v8a/lib_Java.Interop.dll.so
	   157256  01-01-1981 01:01   lib/arm64-v8a/libaot-Java.Interop.dll.so
	     1512  01-01-1981 01:01   lib/arm64-v8a/libarc.bin.so

  * `libSystem.IO.Compression.Native.so` is a native shared library
    from .NET
  * `lib_Java.Interop.dll.so` is compressed assembly data for
    `Java.Interop.dll`
  * `libaot-Java.Interop.dll.so` contains Profiled AOT output for
    `Java.Interop.dll`
  * `libarc.bin.so` is the `rc.bin` file used by .NET runtime startup

Additionally, note that Android limits the characters that can be
used in native library filenames to the regex set `[-._A-Za-z0-9]`.

TODO: No error checking is done to ensure that "Assembly File Name"
stays within the limits of `[-.A-Za-z0-9]`, e.g. if you set
`$(AssemblyName)=Emoji😅` *and `$(AndroidUseAssemblyStore)`=false,
then we'll try to add `lib/arm64-v8a/lib_Emoji😅.dll.so`, which will
fail at runtime.  This works when `$(AndroidUseAssemblyStore)`=true,
which is the default.

Pros:

  * We're no longer fighting against .NET tooling features such as
    ILLink Substitutions.

  * While `.aab` files will get larger, we expect that the actual
    `.apk` files sent to Android devices from the Google Play
    Store will be *smaller*, as the Google Play Store would always
    preserve/transmit *all* `assemblies/assemblies*.blob` files,
    while now it will be able to remove `lib/ABI/*` for unsupported
    ABIs.
    
Cons:

  * `.apk` files containing more than one ABI ***will get larger***,
    as there will no longer be "de-duping" of architecture-agnostic
    assembly data.  We don't consider this a significant concern, as
    we believe `.aab` is the predominant packaging format.

~~ All assemblies are architecture-specific ~~

Assembly pre-processing changes so that every assembly ends up in
every target architecture batch, regardless of whether its MVID
differs from its brethren or not.  This is done very early in the
build process on our side, where we make sure that each assembly
either has the `%(Abi)` metadata or is given one, and is placed in
the corresponding batch.  Further processing of those batches is
"parallel", in that no code attempts to de-duplicate the batches.


~~ Impact on Fast Deployment, `$(IntermediateOutputPath)` ~~

The changes also required us to place all the assemblies in new
locations on disk within `$(IntermediateOutputPath)` when building
the application.  (Related: 2f192386.)  Assemblies are now placed in
subdirectories named after either the target architecture/ABI or the
.NET `$(RuntimeIdentifier)`, e.g.
`obj/Release/netX.Y-android/android-arm64`.  This, in turn, affects
e.g. Fast Deployment as now the synchronized content is in the
`…/.__override__/ABI` directory on device, instead of just in
`…/.__override__`.


~~ File Formats ~~

The assembly store format (c9270261) is updated to use the following
structures:

	struct AssemblyStoreHeader {
	    uint32_t magic;
	    uint32_t version;
	    uint32_t entry_count;               // Number of assemblies in the store
	    uint32_2 index_entry_count;
	    uint32_t index_size;
	};
	struct AssemblyStoreIndexEntry {
	    intptr_t name_hash;	                // xxhash of assembly filename
	    uint32_t descriptor_index;          // index into assemblies array
	};
	struct AssemblyStoreEntryDescriptor {
	    uint32_t mapping_index;             // index into runtime array
	    uint32_t data_offset;               // index into `data` for assembly `.dll`
	    uint32_t data_size;                 // size of assembly, in bytes
	    uint32_t debug_data_offset;         // index into `data` for assembly `.pdb`; 0 if not present
	    uint32_t debug_data_size;           // size of `.pdb`, in bytes; 0 if not present
	    uint32_t config_data_offset;        // index into `data` for assembly `.config`; 0 if not present
	    uint32_t config_data_size;          // size of `.config`, in bytes; 0 if not present
	};
	struct AssemblyStoreAssemblyInfo {
	    uint32_t length;                    // bytes
	    uint8_t  name[length];
	};

`libassemblies.ABI.blob.so` has the following format, and is *not* a
valid ELF file:

	AssemblyStoreHeader                 header {…};
	AssemblyStoreIndexEntry             index [header.index_entry_count];
	AssemblyStoreAssemblyDescriptor     descriptors [header.entry_count];
	AssemblyStoreAssemblyInfo           names [header.entry_count];
	uint8_t data[];

[jonpryor]

I don't understand the comment; put what in a "scope guard"? lib_dir? (Why would lib_dir need to be guarded from multiple threads when it's a local variable?)

[jonpryor]

…or is this about an RAII wrapper aorund DIR* so that closedir() is called at end of scope?

[grendello]

Yep, it'd be a wrapper to close the directory when the scope is no longer valid.

[jonpryor]

Space after ,.

[jonpryor]

The description of the binary file format in the comment block above AssemblyStoreHeader should be udpated to be consistent with the comment in AssemblyStoreGenerator.cs.

[dellis1972]

Is this method always required? Not kust for AssemblyStore ?

e.g do we need this in Debug/Fast Deployment modes?

[grendello]

We always put the runtime config blob (produced from the JSON input file) in the apk, yes. This is the MonoVM settings/feature flags/properties etc, unrelated to our assembly blobs.