fix node externals resolving

[sokra]

What?

Various fixes in handling of externals in node.js

• add extensions to builtin externals to allow node.js ESM to work
• improve the auto-externals logic to detect more edge cases
• prepare for esmExternals support, but that's blocked by client manifest missing the async flag
• currently only esmExternals: false is supported

Why?

Turbopack Changes

• fix(turbopack): postcss should be applied to @imported CSS files turborepo#6531
• fix node.js externals turborepo#6530

[ijjk]

Stats from current PR

Default Build

General Overall increase ⚠️

	vercel/next.js canary	vercel/next.js sokra/fix-node-externals	Change
buildDuration	10.5s	10.5s	✓
buildDurationCached	6s	6s	N/A
nodeModulesSize	199 MB	199 MB	⚠️ +1.11 kB
nextStartRea..uration (ms)	418ms	423ms	N/A

Client Bundles (main, webpack)

	vercel/next.js canary	vercel/next.js sokra/fix-node-externals	Change
199-HASH.js gzip	28.7 kB	28.7 kB	N/A
3f784ff6-HASH.js gzip	53.3 kB	53.3 kB	✓
494.HASH.js gzip	180 B	181 B	N/A
framework-HASH.js gzip	45.2 kB	45.2 kB	✓
main-app-HASH.js gzip	241 B	238 B	N/A
main-HASH.js gzip	31.7 kB	31.7 kB	N/A
webpack-HASH.js gzip	1.7 kB	1.7 kB	✓
Overall change	100 kB	100 kB	✓

Legacy Client Bundles (polyfills)

	vercel/next.js canary	vercel/next.js sokra/fix-node-externals	Change
polyfills-HASH.js gzip	31 kB	31 kB	✓
Overall change	31 kB	31 kB	✓

Client Pages

	vercel/next.js canary	vercel/next.js sokra/fix-node-externals	Change
_app-HASH.js gzip	194 B	195 B	N/A
_error-HASH.js gzip	182 B	181 B	N/A
amp-HASH.js gzip	501 B	503 B	N/A
css-HASH.js gzip	322 B	323 B	N/A
dynamic-HASH.js gzip	2.5 kB	2.5 kB	✓
edge-ssr-HASH.js gzip	253 B	255 B	N/A
head-HASH.js gzip	348 B	347 B	N/A
hooks-HASH.js gzip	369 B	368 B	N/A
image-HASH.js gzip	4.28 kB	4.27 kB	N/A
index-HASH.js gzip	256 B	256 B	✓
link-HASH.js gzip	2.61 kB	2.6 kB	N/A
routerDirect..HASH.js gzip	311 B	311 B	✓
script-HASH.js gzip	384 B	383 B	N/A
withRouter-HASH.js gzip	307 B	308 B	N/A
1afbb74e6ecf..834.css gzip	106 B	106 B	✓
Overall change	3.17 kB	3.17 kB	✓

Client Build Manifests

	vercel/next.js canary	vercel/next.js sokra/fix-node-externals	Change
_buildManifest.js gzip	484 B	483 B	N/A
Overall change	0 B	0 B	✓

Rendered Page Sizes

	vercel/next.js canary	vercel/next.js sokra/fix-node-externals	Change
index.html gzip	527 B	528 B	N/A
link.html gzip	538 B	542 B	N/A
withRouter.html gzip	524 B	523 B	N/A
Overall change	0 B	0 B	✓

Edge SSR bundle Size

	vercel/next.js canary	vercel/next.js sokra/fix-node-externals	Change
edge-ssr.js gzip	92.4 kB	92.4 kB	N/A
page.js gzip	145 kB	145 kB	N/A
Overall change	0 B	0 B	✓

Middleware size

	vercel/next.js canary	vercel/next.js sokra/fix-node-externals	Change
middleware-b..fest.js gzip	626 B	625 B	N/A
middleware-r..fest.js gzip	150 B	151 B	N/A
middleware.js gzip	24.8 kB	24.8 kB	N/A
edge-runtime..pack.js gzip	1.92 kB	1.92 kB	✓
Overall change	1.92 kB	1.92 kB	✓

Next Runtimes

	vercel/next.js canary	vercel/next.js sokra/fix-node-externals	Change
app-page-exp...dev.js gzip	167 kB	167 kB	N/A
app-page-exp..prod.js gzip	93.2 kB	93.2 kB	N/A
app-page-tur..prod.js gzip	94 kB	94 kB	N/A
app-page-tur..prod.js gzip	88.5 kB	88.5 kB	N/A
app-page.run...dev.js gzip	137 kB	137 kB	N/A
app-page.run..prod.js gzip	87.9 kB	87.9 kB	N/A
app-route-ex...dev.js gzip	23.8 kB	23.8 kB	✓
app-route-ex..prod.js gzip	16.4 kB	16.4 kB	✓
app-route-tu..prod.js gzip	16.5 kB	16.5 kB	✓
app-route-tu..prod.js gzip	16 kB	16 kB	✓
app-route.ru...dev.js gzip	23.2 kB	23.2 kB	✓
app-route.ru..prod.js gzip	16 kB	16 kB	✓
pages-api-tu..prod.js gzip	9.37 kB	9.37 kB	✓
pages-api.ru...dev.js gzip	9.64 kB	9.64 kB	✓
pages-api.ru..prod.js gzip	9.37 kB	9.37 kB	✓
pages-turbo...prod.js gzip	21.8 kB	21.8 kB	✓
pages.runtim...dev.js gzip	22.5 kB	22.5 kB	✓
pages.runtim..prod.js gzip	21.8 kB	21.8 kB	✓
server.runti..prod.js gzip	49.1 kB	49.1 kB	N/A
Overall change	206 kB	206 kB	✓

Diff details

Diff for page.js

Diff too large to display

Diff for app-page-exp..ntime.dev.js

Diff too large to display

Diff for app-page-exp..time.prod.js

Diff too large to display

Diff for app-page-tur..time.prod.js

Diff too large to display

Diff for app-page-tur..time.prod.js

Diff too large to display

Diff for app-page.runtime.dev.js

Diff too large to display

Diff for app-page.runtime.prod.js

Diff too large to display

Diff for server.runtime.prod.js

Diff too large to display

Commit: c0e51d6

[ijjk]

Failing test suites

Commit: 274bfa1

pnpm test-dev test/development/basic/gssp-ssr-change-reloading/test/index.test.ts

• GS(S)P Server-Side Change Reloading > should not reload page when client-side is changed too GSP

Expand output

● GS(S)P Server-Side Change Reloading › should not reload page when client-side is changed too GSP

expect(received).toEqual(expected) // deep equality

- Expected  - 1
+ Received  + 1

  Object {
    "count": 1,
    "params": Object {
      "post": "first",
    },
-   "random": 0.1095494597606339,
+   "random": 0.3004102436629015,
  }

  48 |
  49 |     const props2 = JSON.parse(await browser.elementByCss('#props').text())
> 50 |     expect(props).toEqual(props2)
     |                   ^
  51 |
  52 |     await next.patchFile(page, originalContent)
  53 |     await check(() => browser.elementByCss('#change').text(), 'change me')

  at Object.toEqual (development/basic/gssp-ssr-change-reloading/test/index.test.ts:50:19)

Read more about building and testing Next.js in contributing.md.

pnpm test-dev test/e2e/app-dir/app-compilation/index.test.ts

• app dir > HMR > should not cause error when removing loading.js

Expand output

● app dir › HMR › should not cause error when removing loading.js

TIMED OUT: hello from new page

Server Error

undefined

  626 |
  627 |   if (hardError) {
> 628 |     throw new Error('TIMED OUT: ' + regex + '\n\n' + content + '\n\n' + lastErr)
      |           ^
  629 |   }
  630 |   return false
  631 | }

  at check (lib/next-test-utils.ts:628:11)
  at Object.<anonymous> (e2e/app-dir/app-compilation/index.test.ts:44:11)

Read more about building and testing Next.js in contributing.md.

[github-actions]

All broken links are now fixed, thank you!

[HofmannZ]

@sokra - What's blocking this PR from being merged? 🙂

[sokra]

What's blocking this PR from being merged? 🙂

The CI doesn't like me...