Android Prebuilts on Maven Central 🚀

[cortinico]

tl;dr: From React Native 0.71, you will consume prebuilt artifacts on Android (and iOS to some degree) coming from Maven Central. This brings huge improvements in terms of build time for the New Architecture (and not only, keep on reading!)

Hey all,
As the release of 0.71 is (really) coming closer, we'd like to share another post on some of the improvements for the New Architecture that we'll be shipped in 0.71, similary to this other two posts.

TL;DR of what this means practically for you

• 📉 The NPM package react-native size is reduced by 72%.
• 📉 Android clean build time on New Architecture is reduced by 94%
• 📉 Disk space consumption of the node_modules/react-native folder after the first New Architecture build is reduced by 99%.

Sample Android New Architecture clean build timing comparison between 0.70 and 0.71

What has changed

Starting from React Native 0.71, we're publishing pre-built artifacts to Maven Central that will be consumed once you build your app.

We're specifically distributing:

• React Native Android (the Android library that contains the React Native SDK for Android, both Old & New Architecture).
• Hermes Engine Android (the Android library that contains the JS Engine to be used with React Native Android)
• Hermes Engine iOS (a prebuilt tarball for the Hermes engine to be used on iOS apps).

This is the implementation of RFC #508 - out-of-NPM package solution for React Native Artifacts. Please refer to the RFC discussion if you're curious on alternative solutions we evaluated.

Published artifacts can be viewed via this barebone HTTP UI here or via the Maven Central Search Engine

Why was this needed

Previously we were publishing everything inside a single monolithic NPM package (react-native). This simply doesn't scale as we hit the limit of the NPM package size. The New Architecture relying on C++ and the need to ship debug symbols made our artifacts grew quite significantly.

On top of this, using New Architecture on Android meant building from source the whole React Native framework + Hermes engine as we could not distribute prebuilts for them (due to the aformentioned limits).

We had reports of users turning on the New Architecture flag, waiting 1 hour and essentially turning New Architecture immediately back to off as their device was not powerful enough (like here and here). We believe that Developer Experience is crucial for React Native, and bringing build time from seconds to minutes/hours was a major blocker.

Moreover, users reported how the New Architecture was essentially consuming all their public CI credits making a build from source unfeasible for most users.

Why is this important

Practically this change has the following impact:

• The NPM package react-native is reduced in compressed size from 74.8Mb to 20.5Mb (that's a 72% reduction).
• The Android clean build Time for New Architecture is reduced by 94% (from 10m12s to 32sec).
• Android artifacts are cached per version. So you will download artifacts only for 0.71.0 once, regardless of how many different project you'll have on the same machine. This, together with the removal of the build from source results in reduction of diskspace footprint of the node_modules/react-native folder of 99% (from ~10Gb to 69Mb). This results in a overall reduction of the project folder size from from ~12.8Gb to 1.8Gb.
• Android/iOS/Hermes artifacts are lazily downloaded on demand. This means that if you don't develop Android apps, you won't be downloading anything Android related.
• Artifacts are variant-aware and again lazily downloaded. So you will be downloading only the -debug version and not downloading the -release one unless you're doing a release build.
• We removed the need for Windows Users to install Visual Studio Community edition, the C++ build toolchain, CMake, libicu and other complicated build tools (see below)
• We moved to use Prefab. This allows library to leverage our native artifacts (.so/headers) resulting in:
  • Expected disk space saved is ~900Mb for each library that was re-downloading native sources.
  • Build code logic reduction for library maintainer that don't need to re-download native sources. For instance react-native-mmkv mentioned that this reduced their build.gradle logic from 371 to 131 LOC. Smaller build logic => less things that can go wrong between updates.

Disk space consumption comparison between 0.70 and 0.71 of an empty React Native project with New Architecture enabled after the first build.

Other wins & opportunities

The list of opportunities and wins is quite long, let me walk through them one by one:

Prefab Support

In 0.71, we ship prebuilts that are Prefab capable. Prefab is a Industry Standard used to exchange .so and header files between projects. This allowed libraries like reanimated or react-native-mmkv to easily integrate with React Native.

Some reports from folks integrating their libraries with React Native via Prefab:

After some initial testing, we are impressed how easy they are to integrate and how many lines of code we can remove. 🥳
Source @tomekzaw - Software Mansion - Maintainer of Reanimated

My build.gradle file now got significantly smaller, from 371 lines to just 131. Since I don't do any downloading, unzipping, extracting, including headers, and adding source code, this feels much more robust! 🎉
Source @mrousavy - Margelo - Maintainer of react-native-mmkv

Historically, libraries that wanted to provide custom C++ code had to re-download Folly, Glog, Boost, Double Conversion, Fmt and Libevent. Each 3rd party library would have to unzip those and include it in their build.

In 0.70, this was happening for both React Native and for every library that was using those sources.

So you would end up with:

• A re-download of each of those native (unnecessary traffic). Only Boost weight 100Mb+ when zipped
• Multiple copies of Boost inside your node_modules/<react-native-library>, one for each library. Unzipping all those sources ended up consuming ~900Mb for each library.

This ended up devouring a lot of space and bandwidth.

Windows Support

Historically we asked Windows user on New Architecture to do the following:

• Install Cmake standalone
• Install either Build Tools for Visual Studio 2022 or Visual Studio 22 Community Edition - with the C++ desktop development.
• Install libicu
• Run their build inside the "Visual Studio Command Prompt" (using cmd prompt or PowerShell was not supported).

This is sadly a bumpy and lenghty experience, as the Old Architecture did not require those extra steps. 0.71 lifts those requirements bringing New Architecture aligned to the old experience.

Nightlies iOS Support

This change effectively fixed nightlies for iOS.

Every night, we publish a set of Artifact to Sonatype Snapshot repository (a repository similar to Maven Central but reserved for more frequent releases such as nightlies/commitlies).

For iOS specifically, nightlies were broken before we implemented this change as the Hermes Enginer iOS artifact was uploaded to Github Releases. As we created a Github Release only for point/rc/minor releases, nightlies were just failing to download Hermes and were not working.
Similarly, nightilies for Android will consume artifacts from the snapshot repository.

Using nightlies is crucial for us to have a faster feedback loop on our changes. You can read more on how Expo leveraged this change in this blogpost by @Kudo: Testing React Native from the nightly builds

Debug Symbols

The Android artifacts (both React Native and Hermes) contain debug symbols that can be used to symbolicate native stacktraces with the ndk-stack command.

As more and more users are adopting the New Architecture, native crashes are more frequent and it's crucial that we provide valuable stacktraces for them.

Hash/Signature Validation of artifacts

All the artifacts are hashed (MD5, SHA1, SHA256 and SHA512) and GPG signed. This allows users that are concerned about security (i.e. crypto wallet authors) to verify that our artifacts haven't been tampered. Gradle for instance offer the Gradle Verification feature to automate this flow.

Gradle Configuration Cache

This change allow use to make our build compatible with Gradle Configuration caching. While this is still an 'incubating' feature of Gralde, we can expect it to be enabled by default in Gradle 8.x (or any future version).

Configuration caching allows to reduce even further the build time. On my machine this allow me to reduce an incremental build time from 17 seconds to less than a second using a combination of Configuration Cache, ccache and the --active-arch-only.

Maven File layout

Having our artifacts stored in a Maven file layout allows user to potentially proxy/re-host our artifacts inside their corp infra. Solutions like Nexus OSS allows companies to mirror Maven Central and speed-up internal builds while protecting them from external downtimes.

Benchmarks Methodology

For future reference, I'm sharing my benchmarks setup. All the metrics have been collected on a Apple MacBook Pro M1 Max - 64Gb, which is a high-end machines. Users on lower-end devices like MacBook Air will see way more benefits from this change.

Disk space metrics have been collected using finder using the "size on disk" metric.

Build time metrics have been collected with the following commands:

# For 0.70 project
$ npx react-native init RN070 --skip-install && cd RN070 && yarn && cd android && ./gradlew :app:assembleDebug -PnewArchEnabled=true

# For 0.71 project
$ npx react-native init RN071 --version next --skip-install && cd RN071 && yarn && cd android && ./gradlew :app:assembleDebug -PnewArchEnabled=true

To recreate a scenario that simulates the "first user experience" we're not using niether the --active-arch-only nor ccache described in the Speeding up your Build phase guide.

Similarly, the download time (yarn or Gradle download time) has been excluded. All the tests for 0.71 are executed under the assumption that a version of react-android is already downloaded previously on the machine. This is done to exclude variability of different network connections.

Here the results.

• Clean Builds (Old Arch)
  • 0.70: 11sec
  • 0.71: 11sec
• Incremental Builds (Old Arch)
  • 0.70: 3sec (avg of 3 builds)
  • 0.71: 2sec (avg of 3 builds)
• Clean Builds (New Arch)
  • 0.70: 10min 12sec
  • 0.71: 32sec
• Incremental Builds (New Arch)
  • 0.70: 26sec (avg of 3 builds)
  • 0.71: 17sec (avg of 3 builds)
• Folder size of node_modules/react-native (New Arch)
  • 0.70 (before first build): 123.7Mb
  • 0.70 (after first build): 10Gb
  • 0.71 (before first build): 69.2Mb
  • 0.71 (after first build): 69.2Mb

---

We're happy to collect feedback on this specific change here and answer questions/follow-up if there are any.

[thymikee]

Great update, thank you!
In:

from to less than a second

the "from" value is missing. is it 17 sec? Not that it makes a big difference for the whole post, just for correctness :)

[cortinico]

the "from" value is missing. is it 17 sec? Not that it makes a big difference for the whole post, just for correctness :)

Yes is "from 17 sec". Corrected 👍

[TheRogue76]

I remember running the new arch when it first become more easily accessible (0.68 I think) and watching an M1 Max's fans spin up while it chugged along. And almost run out of disk space. Huge W. Really appreciate some of the latest things that the team is doing to fix some of the DX problems of the community!🚀

[mrousavy]

This is insane, awesome work!

Maybe also a quick detail; further down the road this will also make it possible to depend on third-party prefabs - as an example, VisionCamera currently depends on Reanimated and has to extract and recompile a bunch of stuff.
If third party libraries also ship prefabs, we can re-use those in the native code, making the integration between VisionCamera and Reanimated (or react-native-worklets) way simpler and less error-prone - same for VisionCamera Frame Processor Plugins which can now depend on the VisionCamera prefab instead of the sources.

Lots of exciting stuff! :)

[cortinico]

Thanks for the feedback @mrousavy

If third party libraries also ship prefabs, we can re-use those in the native code, making the integration between VisionCamera and Reanimated (or react-native-worklets) way simpler and less error-prone - same for VisionCamera Frame Processor Plugins which can now depend on the VisionCamera prefab instead of the sources.

Yup this is true. Prefab can be used to shared headers/.so between libraries as well.
The only heads up is that we're not suggesting libraries to publish prebuilds (as we did for React Native). The reason being potential ABI incompatibilities.

As long as the React Native CI is responsible for prebuilding and publishing a set of artifacts (Hermes, React Native, etc.) we're fine. If 3rd party libraries (say Reanimated) also start to publish prebuilts which might have built with different compilers or compilation flags, things can break badly.

So yes, you could use prefab to connect libs as long as those libs are not publishing prebuilds on Maven Central (or other repo).

[tomekzaw]

Yup this is true. Prefab can be used to shared headers/.so between libraries as well. The only heads up is that we're not suggesting libraries to publish prebuilds (as we did for React Native). The reason being potential ABI incompatibilities.

I totally agree with @cortinico here. Reanimated needs to support a number of different configurations, so we would need different prebuilds (.so files) for each minor version of React Native, debug or release mode, JSC or Hermes etc. We kind of did this before with AARs for Reanimated 2 but at some point it became totally unmaintainable so with Reanimated 3 we fully went towards building from sources and honestly we are happy with this decision. Obviously, it takes a few additional seconds to build (which nobody has complained about so far btw) but the flexibility and here is more important (at least for now).

Maybe also a quick detail; further down the road this will also make it possible to depend on third-party prefabs - as an example, VisionCamera currently depends on Reanimated and has to extract and recompile a bunch of stuff.
If third party libraries also ship prefabs, we can re-use those in the native code, making the integration between VisionCamera and Reanimated (or react-native-worklets) way simpler and less error-prone - same for VisionCamera Frame Processor Plugins which can now depend on the VisionCamera prefab instead of the sources.

I'm not convinced that it would be less error-prone. In fact, if both Reanimated and VisionCamera were distributed as prefabs, VisionCamera would have to be bound to a specific version of Reanimated - otherwise you would have to publish a number of prebuilts, each one for different version of Reanimated (similarly like Reanimated AARs did for React Native versions) due to aforementioned ABI incompatibilities. Also, it doesn't solve the problem with moving headers - i.e. you would still need to adapt your code to support newer versions of Reanimated even if it was distributed as a prefab.

Finally, we feel like 2022 was some kind of "transition time" for React Native (new architecture, prefab adoption, build scripts improvements, new app template). Most of the changes in Reanimated that could potentially break things for you were actually dictated by React Native itself and our will to support Fabric or new versions of RN (also note that it didn't break the integration with react-native-gesture-handler or expo-gl). We also took our time to significantly clean up our build scripts, remove legacy code, refactor critical parts, improve CI, release more often etc.

@mrousavy As always, thanks for your feedback. Someone wise said: first make it work, then make it great, then make it fast. I think right now we might be somewhere between great and fast.

[Kudo]

it is possible to publish prefab but still build from source. this is an example from react-native-v8: Kudo/react-native-v8#152. that may reduce the complexity for reanimated + v8 integration. i have been waiting the new react-native-v8 commonly adopted and will create a pr for reanimated. then reanimated don't have to take care backward compatibility of both prefab v8 and non prefab v8.

[cortinico]

it is possible to publish prefab but still build from source.

Yup that would be the recommended approach for 3rd party libraries 👍

[matinzd]

I am thrilled with the new 0.71 template and prebuilt binaries, particularly for Android. The removal of unnecessary code in the gradle script is a major improvement.

[douglowder]

Thanks for this extremely detailed writeup @cortinico ! We will start thinking about the best mechanism for providing prebuilt libraries in React Native forks (like the TV fork) so that they can enjoy the same developer experience.

[matinzd]

We have encountered an issue when attempting to upgrade examples in the lottie-react-native library. It appears that older libraries that have not been updated in a long time may have conflicts with new react native pre-built versions. I think it is worth noting here:
lottie-react-native/lottie-react-native#962

@cortinico

[matinzd]

UPDATE:
It seems react-native-windows was causing the conflicts. Removing it solved the issue for now. If anyone is wondering and facing this issue you can follow up here microsoft/react-native-windows#11093.