Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split large runtime manifest into separate bundles #8837

Merged
merged 14 commits into from
Feb 22, 2023

Conversation

mattcompiles
Copy link
Contributor

↪️ Pull Request

To prevent unnecessary cache invalidation, Parcel currently inlines manifests into entry bundles. However, in projects where the entries are very large this still leads to large amounts of unnecessary cache invalidation. To address this, this PR adds a new configurable feature to Parcel that can split runtime manifest assets out into their own bundles.

Basic implementation is as follows:

  • Allow runtimes to return a new value (moveToSeparateBundle) that signifies a runtime asset should have it's own bundle
  • When processing the runtime assets, if moveToSeparateBundle is set then create a new bundle inside the current bundle group and add the asset to that bundle instead.

To allow for this change, createBundle needed to be moved to the internal bundle graph.

Improvement

After making a small change to a commonly used dynamic import in a very large project here's the different in outputs.

No manifest splitting
35 / 3290 (1.1%) bundles changed
32490 / 217051 (15%) bytes changed

With manifest splitting
35 / 3322 (1.1%) bundles changed
2376 / 217062 (1.1%) bytes changed

Questions/Discussions

  • Currently the config is called splitManifestThreshold. Not sure this is a great name. We could also just hard code a value if we don't think the config is useful.
  • I've set a pretty high/safe value as the default (100,000 bytes). Are we happy wtih this? We could get a bit more scientific about it.
  • I think it'd be useful to name these bundles so we know what they are (e.g. runtime-manifest.HASH.js). Maybe we should add a new property to Bundle to let namer plugins know this is a runtime/manifest bundle?

🚨 Test instructions

I've added a couple of new tests but happy to add more if we feel there's anything missing.

✔️ PR Todo

  • Added/updated unit tests for this change
  • Filled out test instructions (In case there aren't any unit tests)
  • Included links to related issues/PRs

@mattcompiles
Copy link
Contributor Author

Will fix up the broken tests next week. 👍

@parcel-benchmark
Copy link

Benchmark Results

Kitchen Sink ✅

Timings

Description Time Difference
Cold 1.95s -86.00ms
Cached 392.00ms -28.00ms 🚀

Cold Bundles

Bundle Size Difference Time Difference
dist/legacy/parcel.7cdb0fad.webp 102.94kb +0.00b 106.00ms -242.00ms 🚀
dist/legacy/parcel.7cdb0fad.webp 102.94kb +0.00b 107.00ms -242.00ms 🚀
dist/modern/parcel.7cdb0fad.webp 102.94kb +0.00b 108.00ms -241.00ms 🚀
dist/legacy/index.2c76ad23.js 1.66kb +0.00b 541.00ms -46.00ms 🚀
dist/legacy/index.8aaa89c9.js 1.20kb +0.00b 541.00ms -45.00ms 🚀
dist/modern/index.6be20f01.js 1.13kb +0.00b 541.00ms -45.00ms 🚀
dist/legacy/index.html 826.00b +0.00b 678.00ms -46.00ms 🚀
dist/modern/index.html 749.00b +0.00b 677.00ms -46.00ms 🚀
dist/legacy/index.b8ae99ba.css 94.00b +0.00b 327.00ms -38.00ms 🚀
dist/modern/index.31cedca9.css 94.00b +0.00b 328.00ms -36.00ms 🚀

Cached Bundles

Bundle Size Difference Time Difference
dist/legacy/parcel.7cdb0fad.webp 102.94kb +0.00b 356.00ms +254.00ms ⚠️
dist/legacy/parcel.7cdb0fad.webp 102.94kb +0.00b 356.00ms +32.00ms ⚠️
dist/modern/parcel.7cdb0fad.webp 102.94kb +0.00b 357.00ms +33.00ms ⚠️
dist/legacy/index.b8ae99ba.css 94.00b +0.00b 366.00ms +35.00ms ⚠️
dist/modern/index.31cedca9.css 94.00b +0.00b 365.00ms +34.00ms ⚠️

React HackerNews ✅

Timings

Description Time Difference
Cold 12.24s -27.00ms
Cached 528.00ms +26.00ms

Cold Bundles

No bundle changes detected.

Cached Bundles

No bundle changes detected.

AtlasKit Editor ✅

Timings

Description Time Difference
Cold 2.09m +2.09m ⚠️
Cached 2.40s +2.40s ⚠️

Cold Bundles

No bundles found, this is probably a failed build...

Cached Bundles

No bundles found, this is probably a failed build...

Three.js ✅

Timings

Description Time Difference
Cold 8.13s -68.00ms
Cached 315.00ms +5.00ms

Cold Bundles

No bundle changes detected.

Cached Bundles

No bundle changes detected.

Click here to view a detailed benchmark overview.

|};

let defaultConfig: JSRuntimeConfig = {
splitManifestThreshold: 100000,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How did we determine 100kb as the threshold?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed, not a lot of thought went into this and it's more of a "finger in the air" guess. We'll merge and potentially refine it later after some tests.

@mattcompiles mattcompiles merged commit 359f6bb into v2 Feb 22, 2023
@mattcompiles mattcompiles deleted the mjones/external-bundle-manifest branch February 22, 2023 02:20
@jtojnar
Copy link
Contributor

jtojnar commented Jul 2, 2023

This appears to have broken HTML pages that contain the <base> tag, see #9092.

marcins pushed a commit to marcins/parcel that referenced this pull request Jul 14, 2023
* upstream/v2:
  Missing edge for multiple targets (parcel-bundler#8854)
  Split large runtime manifest into separate bundles (parcel-bundler#8837)
  Improvements to new resolver (parcel-bundler#8844)
  Fix published files for resolver
  New resolver implementation in Rust (parcel-bundler#8807)
  Update yarn.lock (parcel-bundler#8843)
  Bump napi-rs to latest (parcel-bundler#8838)
lettertwo added a commit that referenced this pull request Nov 6, 2023
* upstream/v2: (128 commits)
  [webextension] Add support for `chrome_style` (#8867)
  Switch to SWC minifier by default (#8860)
  Use BitSet for bundler intersections (#8862)
  best key logic truncating package names (#8865)
  Add support for loadConfig to resolver plugins (#8847)
  Missing edge for multiple targets (#8854)
  Split large runtime manifest into separate bundles (#8837)
  Improvements to new resolver (#8844)
  Fix published files for resolver
  New resolver implementation in Rust (#8807)
  Update yarn.lock (#8843)
  Bump napi-rs to latest (#8838)
  Support .proxyrc.cjs  (#8833)
  Sort global deps before injecting imports (#8818)
  Sort CSS module exports (#8817)
  fix: add extra information to unique bundles (#8784)
  Don't blow up HMR when <link />s don't have hrefs (#8800)
  v2.8.3
  Changelog for v2.8.3
  Address bug by updating an asset reference and merge conditions (#8762)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants