Fix chunk buffering for server components

[shuding]

When testing vercel/server-components-notes-demo#11, we've observed some weird issues that the initial chunk (the "shell") is sometimes broken, and mixed with the data stream. After some testing I confirmed that our buffering (createBufferedTransformStream) is working as we expected: the first chunk it queues is the correct and complete shell. However it's impossible to inject extra things inside a single data chunk. With some deeper investigation it turned out to be something in the underlying Web Stream implementation in the platform — it has a fixed queue size, probably using ByteLengthQueuingStrategy, which makes larger chunks (~8,192B) being splitted into smaller ones. Since we pass everything through two TransformStreams, our buffered shell is chunkenized again.

As described in the code comment, the safest and easiest way to address this is to put the buffering strategy just before merging the data stream.

The issue cannot be reproduce with this fix applied: https://server-components-notes-demo-shuding1.vercel.app. However, it seems not possible to reproduce in a local environment.

Bug

• Related issues linked using fixes #number
• Integration tests added
• Errors have helpful link attached, see contributing.md

Feature

• Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
• Related issues linked using fixes #number
• Integration tests added
• Documentation added
• Telemetry added. In case of a feature if it's used or not.
• Errors have helpful link attached, see contributing.md

Documentation / Examples

• Make sure the linting passes by running yarn lint

[huozhi]

👍

[ijjk]

Stats from current PR

Default Build

General Overall increase ⚠️

	vercel/next.js canary	shuding/next.js shu/f3f5	Change
buildDuration	20.1s	20s	-61ms
buildDurationCached	7.7s	7.9s	⚠️ +197ms
nodeModulesSize	359 MB	359 MB	⚠️ +692 B

Page Load Tests Overall increase ✓

	vercel/next.js canary	shuding/next.js shu/f3f5	Change
/ failed reqs	0	0	✓
/ total time (seconds)	4.285	4.373	⚠️ +0.09
/ avg req/sec	583.38	571.7	⚠️ -11.68
/error-in-render failed reqs	0	0	✓
/error-in-render total time (seconds)	2.153	2.126	-0.03
/error-in-render avg req/sec	1161.15	1176.1	+14.95

Client Bundles (main, webpack, commons)

	vercel/next.js canary	shuding/next.js shu/f3f5	Change
450.HASH.js gzip	179 B	179 B	✓
framework-HASH.js gzip	42 kB	42 kB	✓
main-HASH.js gzip	27.9 kB	27.9 kB	✓
webpack-HASH.js gzip	1.44 kB	1.44 kB	✓
Overall change	71.6 kB	71.6 kB	✓

Legacy Client Bundles (polyfills)

	vercel/next.js canary	shuding/next.js shu/f3f5	Change
polyfills-HASH.js gzip	31 kB	31 kB	✓
Overall change	31 kB	31 kB	✓

Client Pages

	vercel/next.js canary	shuding/next.js shu/f3f5	Change
_app-HASH.js gzip	1.36 kB	1.36 kB	✓
_error-HASH.js gzip	194 B	194 B	✓
amp-HASH.js gzip	312 B	312 B	✓
css-HASH.js gzip	326 B	326 B	✓
dynamic-HASH.js gzip	2.57 kB	2.57 kB	✓
head-HASH.js gzip	350 B	350 B	✓
hooks-HASH.js gzip	919 B	919 B	✓
image-HASH.js gzip	5.05 kB	5.05 kB	✓
index-HASH.js gzip	263 B	263 B	✓
link-HASH.js gzip	2.26 kB	2.26 kB	✓
routerDirect..HASH.js gzip	321 B	321 B	✓
script-HASH.js gzip	383 B	383 B	✓
withRouter-HASH.js gzip	318 B	318 B	✓
85e02e95b279..7e3.css gzip	107 B	107 B	✓
Overall change	14.7 kB	14.7 kB	✓

Client Build Manifests

	vercel/next.js canary	shuding/next.js shu/f3f5	Change
_buildManifest.js gzip	460 B	460 B	✓
Overall change	460 B	460 B	✓

Rendered Page Sizes

	vercel/next.js canary	shuding/next.js shu/f3f5	Change
index.html gzip	531 B	531 B	✓
link.html gzip	544 B	544 B	✓
withRouter.html gzip	526 B	526 B	✓
Overall change	1.6 kB	1.6 kB	✓

---

Default Build with SWC (Decrease detected ✓)

General Overall increase ⚠️

	vercel/next.js canary	shuding/next.js shu/f3f5	Change
buildDuration	24.8s	25.1s	⚠️ +375ms
buildDurationCached	7.7s	7.7s	⚠️ +46ms
nodeModulesSize	359 MB	359 MB	⚠️ +692 B

Page Load Tests Overall decrease ⚠️

	vercel/next.js canary	shuding/next.js shu/f3f5	Change
/ failed reqs	0	0	✓
/ total time (seconds)	4.373	4.507	⚠️ +0.13
/ avg req/sec	571.73	554.7	⚠️ -17.03
/error-in-render failed reqs	0	0	✓
/error-in-render total time (seconds)	2.169	2.218	⚠️ +0.05
/error-in-render avg req/sec	1152.5	1126.96	⚠️ -25.54

Client Bundles (main, webpack, commons)

	vercel/next.js canary	shuding/next.js shu/f3f5	Change
450.HASH.js gzip	179 B	179 B	✓
framework-HASH.js gzip	42.1 kB	42.1 kB	✓
main-HASH.js gzip	28 kB	28 kB	✓
webpack-HASH.js gzip	1.44 kB	1.44 kB	✓
Overall change	71.7 kB	71.7 kB	✓

Legacy Client Bundles (polyfills)

	vercel/next.js canary	shuding/next.js shu/f3f5	Change
polyfills-HASH.js gzip	31 kB	31 kB	✓
Overall change	31 kB	31 kB	✓

Client Pages

	vercel/next.js canary	shuding/next.js shu/f3f5	Change
_app-HASH.js gzip	1.35 kB	1.35 kB	✓
_error-HASH.js gzip	180 B	180 B	✓
amp-HASH.js gzip	305 B	305 B	✓
css-HASH.js gzip	321 B	321 B	✓
dynamic-HASH.js gzip	2.56 kB	2.56 kB	✓
head-HASH.js gzip	342 B	342 B	✓
hooks-HASH.js gzip	911 B	911 B	✓
image-HASH.js gzip	5.08 kB	5.08 kB	✓
index-HASH.js gzip	256 B	256 B	✓
link-HASH.js gzip	2.28 kB	2.28 kB	✓
routerDirect..HASH.js gzip	314 B	314 B	✓
script-HASH.js gzip	375 B	375 B	✓
withRouter-HASH.js gzip	309 B	309 B	✓
85e02e95b279..7e3.css gzip	107 B	107 B	✓
Overall change	14.7 kB	14.7 kB	✓

Client Build Manifests

	vercel/next.js canary	shuding/next.js shu/f3f5	Change
_buildManifest.js gzip	459 B	459 B	✓
Overall change	459 B	459 B	✓

Rendered Page Sizes

	vercel/next.js canary	shuding/next.js shu/f3f5	Change
index.html gzip	534 B	534 B	✓
link.html gzip	547 B	547 B	✓
withRouter.html gzip	528 B	528 B	✓
Overall change	1.61 kB	1.61 kB	✓

Commit: 717636d