build-apks fails in conjunction with coreLibraryDesugaringEnabled

[mtotschnig]

Describe the bug
The build-apks command fails on an app with coreLibraryDesugaringEnabled and minSdk = 16

Bundletool version(s) affected
1.2.0

Stacktrace

 bundletool build-apks --bundle app/build/outputs/bundle/reflectRelease/app-reflect-release.aab  --output ~/tmp/my_app.apks
INFO: The APKs will be signed with the debug keystore found at '/Volumes/Transcend/Android/.android/debug.keystore'.
Error: Merging dex file containing classes with prefix 'j$.' with classes with any other prefixes is not allowed.
[BT:1.2.0] Error: Dex merging failed.
com.android.tools.build.bundletool.model.exceptions.CommandExecutionException: Dex merging failed.
	at com.android.tools.build.bundletool.model.exceptions.InternalExceptionBuilder.build(InternalExceptionBuilder.java:57)
	at com.android.tools.build.bundletool.mergers.D8DexMerger.translateD8Exception(D8DexMerger.java:107)
	at com.android.tools.build.bundletool.mergers.D8DexMerger.merge(D8DexMerger.java:81)
	at com.android.tools.build.bundletool.mergers.ModuleSplitsToShardMerger.mergeDexFiles(ModuleSplitsToShardMerger.java:343)
	at com.android.tools.build.bundletool.mergers.ModuleSplitsToShardMerger.lambda$mergeDexFilesAndCache$1(ModuleSplitsToShardMerger.java:282)
	at java.util.HashMap.computeIfAbsent(HashMap.java:1126)
	at com.android.tools.build.bundletool.mergers.ModuleSplitsToShardMerger.mergeDexFilesAndCache(ModuleSplitsToShardMerger.java:280)
	at com.android.tools.build.bundletool.mergers.ModuleSplitsToShardMerger.mergeSingleShard(ModuleSplitsToShardMerger.java:163)
	at com.android.tools.build.bundletool.mergers.ModuleSplitsToShardMerger.mergeSingleShard(ModuleSplitsToShardMerger.java:99)
	at com.android.tools.build.bundletool.shards.StandaloneApksGenerator.lambda$generateStandaloneApks$1(StandaloneApksGenerator.java:93)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
	at com.android.tools.build.bundletool.shards.StandaloneApksGenerator.generateStandaloneApks(StandaloneApksGenerator.java:96)
	at com.android.tools.build.bundletool.shards.ShardedApksFacade.generateSplits(ShardedApksFacade.java:63)
	at com.android.tools.build.bundletool.commands.BuildApksManager.generateStandaloneApks(BuildApksManager.java:211)
	at com.android.tools.build.bundletool.commands.BuildApksManager.execute(BuildApksManager.java:147)
	at com.android.tools.build.bundletool.commands.BuildApksCommand.execute(BuildApksCommand.java:624)
	at com.android.tools.build.bundletool.BundleToolMain.main(BundleToolMain.java:75)
	at com.android.tools.build.bundletool.BundleToolMain.main(BundleToolMain.java:47)
Caused by: com.android.tools.r8.CompilationFailedException: Compilation failed to complete
	at com.android.tools.r8.utils.W.a(:87)
	at com.android.tools.r8.D8.run(:6)
	at com.android.tools.build.bundletool.mergers.D8DexMerger.merge(D8DexMerger.java:74)
	... 21 more
Caused by: com.android.tools.r8.utils.b: Error: Merging dex file containing classes with prefix 'j$.' with classes with any other prefixes is not allowed.
	at com.android.tools.r8.utils.M0.a(:21)
	at com.android.tools.r8.utils.W.a(:73)
	... 23 more

To Reproduce
I have patched android/app-bundle-samples in order to showcase the problem with the DynamicCodeLoadingKotlin sample: mtotschnig/app-bundle-samples@af6cec9

Expected behavior
build-apks should generate APKs to allow local testing.

Known workaround
None.

Environment:
JVM: 1.8.0_144 (Oracle Corporation 25.144-b01)
OS: Mac OS X 10.15.6 x86_64

[ymakhno]

desugar_jdk_libs is not currently fully supported in bundletool for applications with minSdk less than 21. This decision is made because even if we implement such support it will be very limited as on devices with sdk below 21 legacy multidex support is required for such apps: https://developer.android.com/studio/build/multidex#mdex-gradle.

And legacy multidex support + desugar_jdk_libs has a limitation: desugar_jdk_libs requires to have its own dex file that on devices with sdk <= 21 will be loaded after the main dex file with all activities and fragments. This means that a developer won't be able to use any Java 8 classes/interfaces as field types, method parameters, or method return types in subclasses of Activity, Fragment. Java 8 classes will work correctly only as local variables and in helper classes.

[mtotschnig]

Is this only a limitation of bundletool or would an Android App Bundle submitted to Play Store suffer from the same limitations?

[ymakhno]

The same limitations.

Would you still find desugar lib useful even if you won’t be able to use Java 8 types as field types, method parameters, or method return types in subclasses of Activity, Fragment as mentioned above?

[mtotschnig]

If I know these limitations, I would still find desugar lib useful, and restrict the use of Java 8 types to helper classes. Are these limitations officially documented? Do they also apply if I build a traditional APK? I have been deploying an APK to an API 16 device, and was able to make use of java.time.LocalDate as field type of an Activity class without problem.

[ymakhno]

They are still applicable to regular APKs as this is platform limitation.

Information on how desugaring works as well as why it produces second dex file is explained in a good way here: https://jakewharton.com/d8-library-desugaring/#second-dex.

Legacy multidex (multiple dex files on platforms below 21) is explained here: https://developer.android.com/studio/build/multidex#mdex-pre-l. What is primary dex file, what should be included inside primary dex file and why application may produce NoClassDefFoundError.

[ymakhno]

I think bundletool currently supports desugaring for bundles without feature modules with limitations described above. And I will keep this issue open to have it on our radar for bundles with coreLibraryDesugaringEnabled and feature modules.

[Biftor]

Still no solution?
I am dealing with same issue
https://github.com/Biftor/DynamicModule

[mochadwi]

both enabling/disabling desugar in dynamic feature this same error occurred to me:

following issue

1. https://issuetracker.google.com/issues/174166020
2. (as shown also by @Biftor here)
3. D8 dexing error using authn_android okta/okta-oidc-android#180

as explained above DEX merging failed, any workaround for this issue?

[ymakhno]

Implemented in bundletool 1.7.0.