Skip to content

Commit e1af958

Browse files
grendellogrendello
authored
[Xamarin.Android.Build.Tasks, monodroid] LLVM Marshal Methods Infra (#7004)
Context: https://github.com/xamarin/xamarin-android/wiki/Blueprint#java-type-registration Introduce low-level "plumbing" for future JNI marshal method work. A JNI marshal method is a [JNI-callable][0] function pointer provided to [`JNIEnv::RegisterNatives()`][1]. Currently, JNI marshal methods are provided via the interaction between `generator` and `JNINativeWrapper.CreateDelegate()`: * `generator` emits the "actual" JNI-callable method. * `JNINativeWrapper.CreateDelegate()` uses System.Reflection.Emit to *wrap* the `generator`-emitted for exception marshaling. (Though see also 32cff43.) JNI marshal methods are needed for all Java-to-C# transitions. Consider the virtual `Activity.OnCreate()` method: partial class Activity { static Delegate? cb_onCreate_Landroid_os_Bundle_; static Delegate GetOnCreate_Landroid_os_Bundle_Handler () { if (cb_onCreate_Landroid_os_Bundle_ == null) cb_onCreate_Landroid_os_Bundle_ = JNINativeWrapper.CreateDelegate ((_JniMarshal_PPL_V) n_OnCreate_Landroid_os_Bundle_); return cb_onCreate_Landroid_os_Bundle_; } static void n_OnCreate_Landroid_os_Bundle_ (IntPtr jnienv, IntPtr native__this, IntPtr native_savedInstanceState) { var __this = global::Java.Lang.Object.GetObject<Android.App.Activity> (jnienv, native__this, JniHandleOwnership.DoNotTransfer)!; var savedInstanceState = global::Java.Lang.Object.GetObject<Android.OS.Bundle> (native_savedInstanceState, JniHandleOwnership.DoNotTransfer); __this.OnCreate (savedInstanceState); } // Metadata.xml XPath method reference: path="/api/package[@name='android.app']/class[@name='Activity']/method[@name='onCreate' and count(parameter)=1 and parameter[1][@type='android.os.Bundle']]" [Register ("onCreate", "(Landroid/os/Bundle;)V", "GetOnCreate_Landroid_os_Bundle_Handler")] protected virtual unsafe void OnCreate (Android.OS.Bundle? savedInstanceState) => … } `Activity.n_OnCreate_Landroid_os_Bundle_()` is the JNI marshal method, responsible for marshaling parameters from JNI values into C# types, forwarding the method invocation to `Activity.OnCreate()`, and (if necessary) marshaling the return value back to JNI. `Activity.GetOnCreate_Landroid_os_Bundle_Handler()` is part of the type registration infrastructure, providing a `Delegate` instance to `RegisterNativeMembers .RegisterNativeMembers()`, which is eventually passed to `JNIEnv::RegisterNatives()`. While this works, it's not incredibly performant: unless using one of the optimized delegate types (32cff43 et. al), System.Reflection.Emit is used to create a wrapper around the marshal method, which is something we've wanted to avoid doing for years. Thus, the idea: since we're *already* bundling a native toolchain and using LLVM-IR to produce `libxamarin-app.so` (b21cbf9, 5271f3e), what if we emitted [Java Native Method Names][2] and *skipped* all the done as part of `Runtime.register()` and `JNIEnv.RegisterJniNatives()`? Given: class MyActivity : Activity { protected override void OnCreate(Bundle? state) => … } During the build, `libxamarin-app.so` would contain the function: JNIEXPORT void JNICALL Java_crc…_MyActivity_n_1onCreate (JNIEnv *env, jobject self, jobject state); During App runtime, the `Runtime.register()` invocation present in [Java Callable Wrappers][3] would either be omitted or would be a no-op, and Android/JNI would instead resolve `MyActivity.n_onCreate()` as `Java_crc…_MyActivity_n_1onCreate()`. Many of the specifics are still being investigated, and this feature will be spread across various areas. We call this effort "LLVM Marshal Methods". First, prepare the way. Update `Xamarin.Android.Build.Tasks.dll` and `src/monodroid` to introduce support for generating JNI marshal methods into `libxamarin-app.so`. Most of the added code is *disabled and hidden* behind `#if ENABLE_MARSHAL_METHODS`. ~~ TODO ~~ Other pieces, in no particular order: * Update [`Java.Interop.Tools.JavaCallableWrappers`][4] so that static constructors aren't needed when LLVM Marshal Methods are used. * Update [`generator`][5] so that *two* sets of marshal methods are emitted: the current set e.g. `Activity.n_OnCreate_Landroid_os_Bundle_()`, and an "overload" set which has [`UnmanagedCallersOnlyAttribute`][6]. LLVM Marshal Methods will be able to directly call these "unmanaged marshal methods" without the overhead of `mono_runtime_invoke()`; see also f48b97c. * Finish the LLVM code generator so that LLVM Marshal Methods are emitted into `libxamarin-app.so`. * Update the linker so that much of the earlier marshal method infrastructure is removed in Release apps. When LLVM Marshal Methods are used, there is no need for `Activity.cb_onCreate_Landroid_os_Bundle_`, `Actvitiy.GetOnCreate_Landroid_os_Bundle_Handler()`, or the `Activity.n_OnCreate_Landroid_os_Bundle_()` without `[UnmanagedCallersOnly]`. Meanwhile, we cannot remove the existing infrastructure, as the current System.Reflection.Emit-oriented code is needed for faster app builds, a desirable feature of Debug configuration builds. LLVM Marshal Methods will be a Release configuration-only feature. [0]: https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/design.html#native_method_arguments [1]: https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html#RegisterNatives [2]: https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/design.html#resolving_native_method_names [3]: https://github.com/xamarin/xamarin-android/wiki/Blueprint#java-type-registration [4]: https://github.com/xamarin/java.interop/tree/main/src/Java.Interop.Tools.JavaCallableWrappers [5]: https://github.com/xamarin/xamarin-android/wiki/Blueprint#generator [6]: https://docs.microsoft.com/dotnet/api/system.runtime.interopservices.unmanagedcallersonlyattribute?view=net-6.0
1 parent 9987069 commit e1af958

28 files changed

+760
-173
lines changed

‎Directory.Build.props

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,10 @@
2424
<RollForward>Major</RollForward>
2525
</PropertyGroup>
2626

27+
<PropertyGroup Condition=" '$(MSBuildRuntimeType)' == 'Core' ">
28+
<_EnableMarshalMethods>NoThanks</_EnableMarshalMethods> <!-- set to YesPlease to enable -->
29+
</PropertyGroup>
30+
2731
<PropertyGroup>
2832
<ProductVersion>12.3.99</ProductVersion>
2933
<!-- NuGet package version numbers. See Documentation/guides/OneDotNet.md.

‎src/Xamarin.Android.Build.Tasks/Tasks/GenerateJavaStubs.cs

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,8 @@ namespace Xamarin.Android.Tasks
2626

2727
public class GenerateJavaStubs : AndroidTask
2828
{
29+
public const string MarshalMethodsRegisterTaskKey = ".:!MarshalMethods!:.";
30+
2931
public override string TaskPrefix => "GJS";
3032

3133
[Required]
@@ -338,6 +340,9 @@ bool CreateJavaSources (IEnumerable<TypeDefinition> javaTypes, TypeDefinitionCac
338340
string monoInit = GetMonoInitSource (AndroidSdkPlatform);
339341
bool hasExportReference = ResolvedAssemblies.Any (assembly => Path.GetFileName (assembly.ItemSpec) == "Mono.Android.Export.dll");
340342
bool generateOnCreateOverrides = int.Parse (AndroidSdkPlatform) <= 10;
343+
#if ENABLE_MARSHAL_METHODS
344+
var overriddenMethodDescriptors = new List<OverriddenMethodDescriptor> ();
345+
#endif
341346

342347
bool ok = true;
343348
foreach (var t in javaTypes) {
@@ -355,6 +360,9 @@ bool CreateJavaSources (IEnumerable<TypeDefinition> javaTypes, TypeDefinitionCac
355360
};
356361

357362
jti.Generate (writer);
363+
#if ENABLE_MARSHAL_METHODS
364+
overriddenMethodDescriptors.AddRange (jti.OverriddenMethodDescriptors);
365+
#endif
358366
writer.Flush ();
359367

360368
var path = jti.GetDestinationPath (outputPath);
@@ -388,6 +396,9 @@ bool CreateJavaSources (IEnumerable<TypeDefinition> javaTypes, TypeDefinitionCac
388396
}
389397
}
390398
}
399+
#if ENABLE_MARSHAL_METHODS
400+
BuildEngine4.RegisterTaskObjectAssemblyLocal (MarshalMethodsRegisterTaskKey, overriddenMethodDescriptors, RegisteredTaskObjectLifetime.Build);
401+
#endif
391402
return ok;
392403
}
393404

‎src/Xamarin.Android.Build.Tasks/Tasks/GeneratePackageManagerJava.cs

Lines changed: 32 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -299,8 +299,16 @@ void AddEnvironment ()
299299

300300
int assemblyCount = 0;
301301
HashSet<string> archAssemblyNames = null;
302-
302+
#if ENABLE_MARSHAL_METHODS
303+
var uniqueAssemblyNames = new HashSet<string> (StringComparer.OrdinalIgnoreCase);
304+
#endif
303305
Action<ITaskItem> updateAssemblyCount = (ITaskItem assembly) => {
306+
string assemblyName = Path.GetFileName (assembly.ItemSpec);
307+
#if ENABLE_MARSHAL_METHODS
308+
if (!uniqueAssemblyNames.Contains (assemblyName)) {
309+
uniqueAssemblyNames.Add (assemblyName);
310+
}
311+
#endif
304312
if (!UseAssemblyStore) {
305313
assemblyCount++;
306314
return;
@@ -316,7 +324,6 @@ void AddEnvironment ()
316324
} else {
317325
archAssemblyNames ??= new HashSet<string> (StringComparer.OrdinalIgnoreCase);
318326

319-
string assemblyName = Path.GetFileName (assembly.ItemSpec);
320327
if (!archAssemblyNames.Contains (assemblyName)) {
321328
assemblyCount++;
322329
archAssemblyNames.Add (assemblyName);
@@ -431,16 +438,34 @@ void AddEnvironment ()
431438
JNIEnvRegisterJniNativesToken = jnienv_registerjninatives_method_token,
432439
};
433440
appConfigAsmGen.Init ();
434-
441+
#if ENABLE_MARSHAL_METHODS
442+
var marshalMethodsAsmGen = new MarshalMethodsNativeAssemblyGenerator () {
443+
NumberOfAssembliesInApk = assemblyCount,
444+
UniqueAssemblyNames = uniqueAssemblyNames,
445+
OverriddenMethodDescriptors = BuildEngine4.GetRegisteredTaskObjectAssemblyLocal<List<Java.Interop.Tools.JavaCallableWrappers.OverriddenMethodDescriptor>> (GenerateJavaStubs.MarshalMethodsRegisterTaskKey, RegisteredTaskObjectLifetime.Build)
446+
};
447+
marshalMethodsAsmGen.Init ();
448+
#endif
435449
foreach (string abi in SupportedAbis) {
436-
string baseAsmFilePath = Path.Combine (EnvironmentOutputDirectory, $"environment.{abi.ToLowerInvariant ()}");
437-
string llFilePath = $"{baseAsmFilePath}.ll";
450+
string targetAbi = abi.ToLowerInvariant ();
451+
string environmentBaseAsmFilePath = Path.Combine (EnvironmentOutputDirectory, $"environment.{targetAbi}");
452+
string marshalMethodsBaseAsmFilePath = Path.Combine (EnvironmentOutputDirectory, $"marshal_methods.{targetAbi}");
453+
string environmentLlFilePath = $"{environmentBaseAsmFilePath}.ll";
454+
string marshalMethodsLlFilePath = $"{marshalMethodsBaseAsmFilePath}.ll";
438455

456+
AndroidTargetArch targetArch = GetAndroidTargetArchForAbi (abi);
457+
using (var sw = MemoryStreamPool.Shared.CreateStreamWriter ()) {
458+
appConfigAsmGen.Write (targetArch, sw, environmentLlFilePath);
459+
sw.Flush ();
460+
Files.CopyIfStreamChanged (sw.BaseStream, environmentLlFilePath);
461+
}
462+
#if ENABLE_MARSHAL_METHODS
439463
using (var sw = MemoryStreamPool.Shared.CreateStreamWriter ()) {
440-
appConfigAsmGen.Write (GetAndroidTargetArchForAbi (abi), sw, llFilePath);
464+
marshalMethodsAsmGen.Write (targetArch, sw, marshalMethodsLlFilePath);
441465
sw.Flush ();
442-
Files.CopyIfStreamChanged (sw.BaseStream, llFilePath);
466+
Files.CopyIfStreamChanged (sw.BaseStream, marshalMethodsLlFilePath);
443467
}
468+
#endif
444469
}
445470

446471
void AddEnvironmentVariable (string name, string value)

‎src/Xamarin.Android.Build.Tasks/Tasks/PrepareAbiItems.cs

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,9 @@ public class PrepareAbiItems : AndroidTask
1414
const string TypeMapBase = "typemaps";
1515
const string EnvBase = "environment";
1616
const string CompressedAssembliesBase = "compressed_assemblies";
17-
17+
#if ENABLE_MARSHAL_METHODS
18+
const string MarshalMethodsBase = "marshal_methods";
19+
#endif
1820
public override string TaskPrefix => "PAI";
1921

2022
[Required]
@@ -50,6 +52,10 @@ public override bool RunTask ()
5052
baseName = EnvBase;
5153
} else if (String.Compare ("compressed", Mode, StringComparison.OrdinalIgnoreCase) == 0) {
5254
baseName = CompressedAssembliesBase;
55+
#if ENABLE_MARSHAL_METHODS
56+
} else if (String.Compare ("marshal_methods", Mode, StringComparison.OrdinalIgnoreCase) == 0) {
57+
baseName = MarshalMethodsBase;
58+
#endif
5359
} else {
5460
Log.LogError ($"Unknown mode: {Mode}");
5561
return false;

‎src/Xamarin.Android.Build.Tasks/Tests/Xamarin.ProjectTools/Resources/Base/BuildReleaseArm64SimpleDotNet.apkdesc

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
"Size": 60736
99
},
1010
"assemblies/Mono.Android.dll": {
11-
"Size": 152435
11+
"Size": 152344
1212
},
1313
"assemblies/rc.bin": {
1414
"Size": 1083
@@ -59,13 +59,13 @@
5959
"Size": 9591
6060
},
6161
"assemblies/UnnamedProject.dll": {
62-
"Size": 3560
62+
"Size": 3555
6363
},
6464
"classes.dex": {
6565
"Size": 348440
6666
},
6767
"lib/arm64-v8a/libmonodroid.so": {
68-
"Size": 484512
68+
"Size": 510920
6969
},
7070
"lib/arm64-v8a/libmonosgen-2.0.so": {
7171
"Size": 4667768
@@ -80,7 +80,7 @@
8080
"Size": 146816
8181
},
8282
"lib/arm64-v8a/libxamarin-app.so": {
83-
"Size": 15944
83+
"Size": 15904
8484
},
8585
"META-INF/BNDLTOOL.RSA": {
8686
"Size": 1213
@@ -92,7 +92,7 @@
9292
"Size": 3646
9393
},
9494
"res/drawable-hdpi-v4/icon.png": {
95-
"Size": 4762
95+
"Size": 4791
9696
},
9797
"res/drawable-mdpi-v4/icon.png": {
9898
"Size": 2200
@@ -116,5 +116,5 @@
116116
"Size": 1904
117117
}
118118
},
119-
"PackageSize": 3492724
119+
"PackageSize": 3500916
120120
}

‎src/Xamarin.Android.Build.Tasks/Tests/Xamarin.ProjectTools/Resources/Base/BuildReleaseArm64SimpleLegacy.apkdesc

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,10 @@
55
"Size": 2604
66
},
77
"assemblies/Java.Interop.dll": {
8-
"Size": 67948
8+
"Size": 67950
99
},
1010
"assemblies/Mono.Android.dll": {
11-
"Size": 257194
11+
"Size": 257187
1212
},
1313
"assemblies/mscorlib.dll": {
1414
"Size": 769016
@@ -23,7 +23,7 @@
2323
"Size": 2879
2424
},
2525
"classes.dex": {
26-
"Size": 349820
26+
"Size": 349736
2727
},
2828
"lib/arm64-v8a/libmono-btls-shared.so": {
2929
"Size": 1613872
@@ -32,7 +32,7 @@
3232
"Size": 750976
3333
},
3434
"lib/arm64-v8a/libmonodroid.so": {
35-
"Size": 392576
35+
"Size": 421872
3636
},
3737
"lib/arm64-v8a/libmonosgen-2.0.so": {
3838
"Size": 4030448
@@ -74,5 +74,5 @@
7474
"Size": 1724
7575
}
7676
},
77-
"PackageSize": 4044500
77+
"PackageSize": 4052692
7878
}

‎src/Xamarin.Android.Build.Tasks/Tests/Xamarin.ProjectTools/Resources/Base/BuildReleaseArm64XFormsDotNet.apkdesc

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
"Size": 66806
1212
},
1313
"assemblies/Mono.Android.dll": {
14-
"Size": 447633
14+
"Size": 447543
1515
},
1616
"assemblies/mscorlib.dll": {
1717
"Size": 3888
@@ -119,7 +119,7 @@
119119
"Size": 1906
120120
},
121121
"assemblies/UnnamedProject.dll": {
122-
"Size": 117250
122+
"Size": 117244
123123
},
124124
"assemblies/Xamarin.AndroidX.Activity.dll": {
125125
"Size": 5941
@@ -191,7 +191,7 @@
191191
"Size": 3460820
192192
},
193193
"lib/arm64-v8a/libmonodroid.so": {
194-
"Size": 484512
194+
"Size": 510920
195195
},
196196
"lib/arm64-v8a/libmonosgen-2.0.so": {
197197
"Size": 4667768
@@ -206,7 +206,7 @@
206206
"Size": 146816
207207
},
208208
"lib/arm64-v8a/libxamarin-app.so": {
209-
"Size": 98480
209+
"Size": 98440
210210
},
211211
"META-INF/android.support.design_material.version": {
212212
"Size": 12
@@ -773,7 +773,7 @@
773773
"Size": 470
774774
},
775775
"res/drawable-hdpi-v4/icon.png": {
776-
"Size": 4762
776+
"Size": 4791
777777
},
778778
"res/drawable-hdpi-v4/notification_bg_low_normal.9.png": {
779779
"Size": 212
@@ -1961,5 +1961,5 @@
19611961
"Size": 341228
19621962
}
19631963
},
1964-
"PackageSize": 8347981
1964+
"PackageSize": 8356173
19651965
}

‎src/Xamarin.Android.Build.Tasks/Tests/Xamarin.ProjectTools/Resources/Base/BuildReleaseArm64XFormsLegacy.apkdesc

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,10 @@
88
"Size": 7215
99
},
1010
"assemblies/Java.Interop.dll": {
11-
"Size": 68913
11+
"Size": 68916
1212
},
1313
"assemblies/Mono.Android.dll": {
14-
"Size": 567718
14+
"Size": 567711
1515
},
1616
"assemblies/Mono.Security.dll": {
1717
"Size": 68432
@@ -65,22 +65,22 @@
6565
"Size": 131930
6666
},
6767
"assemblies/Xamarin.AndroidX.DrawerLayout.dll": {
68-
"Size": 15426
68+
"Size": 15425
6969
},
7070
"assemblies/Xamarin.AndroidX.Fragment.dll": {
71-
"Size": 43135
71+
"Size": 43134
7272
},
7373
"assemblies/Xamarin.AndroidX.Legacy.Support.Core.UI.dll": {
74-
"Size": 6715
74+
"Size": 6714
7575
},
7676
"assemblies/Xamarin.AndroidX.Lifecycle.Common.dll": {
7777
"Size": 7062
7878
},
7979
"assemblies/Xamarin.AndroidX.Lifecycle.LiveData.Core.dll": {
80-
"Size": 7193
80+
"Size": 7194
8181
},
8282
"assemblies/Xamarin.AndroidX.Lifecycle.ViewModel.dll": {
83-
"Size": 4872
83+
"Size": 4873
8484
},
8585
"assemblies/Xamarin.AndroidX.Loader.dll": {
8686
"Size": 13585
@@ -113,7 +113,7 @@
113113
"Size": 43497
114114
},
115115
"classes.dex": {
116-
"Size": 3462252
116+
"Size": 3462080
117117
},
118118
"lib/arm64-v8a/libmono-btls-shared.so": {
119119
"Size": 1613872
@@ -122,7 +122,7 @@
122122
"Size": 750976
123123
},
124124
"lib/arm64-v8a/libmonodroid.so": {
125-
"Size": 392576
125+
"Size": 421872
126126
},
127127
"lib/arm64-v8a/libmonosgen-2.0.so": {
128128
"Size": 4030448
@@ -1883,5 +1883,5 @@
18831883
"Size": 341040
18841884
}
18851885
},
1886-
"PackageSize": 9562270
1886+
"PackageSize": 9570462
18871887
}

‎src/Xamarin.Android.Build.Tasks/Utilities/ApplicationConfigNativeAssemblyGenerator.cs

Lines changed: 1 addition & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,8 @@
11
using System;
22
using System.Collections.Generic;
33
using System.IO;
4-
using System.Text;
54

65
using Java.Interop.Tools.TypeNameMappings;
7-
using K4os.Hash.xxHash;
86
using Microsoft.Build.Framework;
97
using Microsoft.Build.Utilities;
108

@@ -342,21 +340,11 @@ void WriteDSOCache (LlvmIrGenerator generator)
342340

343341
// We need to hash here, because the hash is architecture-specific
344342
foreach (StructureInstance<DSOCacheEntry> entry in dsoCache) {
345-
entry.Obj.hash = HashName (entry.Obj.HashedName);
343+
entry.Obj.hash = HashName (entry.Obj.HashedName, is64Bit);
346344
}
347345
dsoCache.Sort ((StructureInstance<DSOCacheEntry> a, StructureInstance<DSOCacheEntry> b) => a.Obj.hash.CompareTo (b.Obj.hash));
348346

349347
generator.WriteStructureArray (dsoCacheEntryStructureInfo, dsoCache, "dso_cache");
350-
351-
ulong HashName (string name)
352-
{
353-
byte[] nameBytes = Encoding.UTF8.GetBytes (name);
354-
if (is64Bit) {
355-
return XXH64.DigestOf (nameBytes, 0, nameBytes.Length);
356-
}
357-
358-
return (ulong)XXH32.DigestOf (nameBytes, 0, nameBytes.Length);
359-
}
360348
}
361349
}
362350
}

0 commit comments

Comments
 (0)