chore(test): run related E2E deploy tests on PRs

[balazsorban44]

What?

This PR builds on #63757.

It adds a new workflow, that runs E2E Deploy tests related to file changes in a PR.

How?

The workflow collects all tests related to file changes in a given PR based on the test/related-tests-manifest.json and run those deployed.

Closes NEXT-2951

[balazsorban44]

• chore(test): run related E2E deploy tests on PRs #63763 👈
• chore: add test mapping script #63757
• canary

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @balazsorban44 and the rest of your teammates on Graphite

[ijjk]

Failing test suites

Commit: 4c24768

__NEXT_EXPERIMENTAL_PPR=true pnpm test-dev test/e2e/app-dir/metadata/metadata.test.ts (PPR)

• app dir - metadata > react cache > should have same title and page value when navigating

Expand output

● app dir - metadata › react cache › should have same title and page value when navigating

expect(received).toBe(expected) // Object.is equality

Expected: "0.30467623224719564"
Received: "0.7926862089797004"

  1006 |         const obj = JSON.parse(title)
  1007 |         // Check `cache()`
> 1008 |         expect(obj.val.toString()).toBe(value)
       |                                    ^
  1009 |         // Check `fetch()`
  1010 |         // TODO-APP: Investigate why fetch deduping doesn't apply but cache() does.
  1011 |         if (!isNextDev) {

  at Object.toBe (e2e/app-dir/metadata/metadata.test.ts:1008:36)

Read more about building and testing Next.js in contributing.md.

TURBOPACK=1 pnpm test-start test/e2e/prerender.test.ts (turbopack)

• Prerender > should on-demand revalidate for fallback: blocking with onlyGenerated if generated

Expand output

● Prerender › should on-demand revalidate for fallback: blocking with onlyGenerated if generated

expect(received).toMatch(expected)

Expected pattern: /(HIT|STALE)/
Received string:  "MISS"

  2296 |
  2297 |         expect(initialTime).toBe($2('#time').text())
> 2298 |         expect(res2.headers.get('x-nextjs-cache')).toMatch(/(HIT|STALE)/)
       |                                                    ^
  2299 |
  2300 |         const res3 = await fetchViaHTTP(
  2301 |           next.url,

  at Object.toMatch (e2e/prerender.test.ts:2298:52)

Read more about building and testing Next.js in contributing.md.

[ijjk]

Stats from current PR

Default Build

General Overall increase ⚠️

	vercel/next.js canary	vercel/next.js chore/run-e2e-deploy-on-prs	Change
buildDuration	17.5s	17.5s	N/A
buildDurationCached	9.5s	8.2s	N/A
nodeModulesSize	199 MB	199 MB	⚠️ +2.89 kB
nextStartRea..uration (ms)	451ms	470ms	N/A

Client Bundles (main, webpack)

	vercel/next.js canary	vercel/next.js chore/run-e2e-deploy-on-prs	Change
2453-HASH.js gzip	31.6 kB	31.5 kB	N/A
3304.HASH.js gzip	181 B	181 B	✓
3f784ff6-HASH.js gzip	53.7 kB	53.7 kB	✓
8299-HASH.js gzip	5.1 kB	5.1 kB	N/A
framework-HASH.js gzip	45.2 kB	45.2 kB	✓
main-app-HASH.js gzip	242 B	242 B	✓
main-HASH.js gzip	29.6 kB	29.7 kB	N/A
webpack-HASH.js gzip	1.68 kB	1.68 kB	N/A
Overall change	99.3 kB	99.3 kB	✓

Legacy Client Bundles (polyfills)

	vercel/next.js canary	vercel/next.js chore/run-e2e-deploy-on-prs	Change
polyfills-HASH.js gzip	31 kB	31 kB	✓
Overall change	31 kB	31 kB	✓

Client Pages

	vercel/next.js canary	vercel/next.js chore/run-e2e-deploy-on-prs	Change
_app-HASH.js gzip	196 B	197 B	N/A
_error-HASH.js gzip	184 B	184 B	✓
amp-HASH.js gzip	505 B	505 B	✓
css-HASH.js gzip	324 B	325 B	N/A
dynamic-HASH.js gzip	2.5 kB	2.5 kB	N/A
edge-ssr-HASH.js gzip	258 B	258 B	✓
head-HASH.js gzip	352 B	352 B	✓
hooks-HASH.js gzip	370 B	371 B	N/A
image-HASH.js gzip	4.27 kB	4.27 kB	✓
index-HASH.js gzip	259 B	259 B	✓
link-HASH.js gzip	2.67 kB	2.67 kB	N/A
routerDirect..HASH.js gzip	314 B	312 B	N/A
script-HASH.js gzip	386 B	386 B	✓
withRouter-HASH.js gzip	309 B	309 B	✓
1afbb74e6ecf..834.css gzip	106 B	106 B	✓
Overall change	6.63 kB	6.63 kB	✓

Client Build Manifests

	vercel/next.js canary	vercel/next.js chore/run-e2e-deploy-on-prs	Change
_buildManifest.js gzip	483 B	485 B	N/A
Overall change	0 B	0 B	✓

Rendered Page Sizes

	vercel/next.js canary	vercel/next.js chore/run-e2e-deploy-on-prs	Change
index.html gzip	528 B	527 B	N/A
link.html gzip	541 B	540 B	N/A
withRouter.html gzip	524 B	522 B	N/A
Overall change	0 B	0 B	✓

Edge SSR bundle Size

	vercel/next.js canary	vercel/next.js chore/run-e2e-deploy-on-prs	Change
edge-ssr.js gzip	94.4 kB	94.5 kB	N/A
page.js gzip	3.05 kB	3.05 kB	N/A
Overall change	0 B	0 B	✓

Middleware size

	vercel/next.js canary	vercel/next.js chore/run-e2e-deploy-on-prs	Change
middleware-b..fest.js gzip	623 B	625 B	N/A
middleware-r..fest.js gzip	155 B	156 B	N/A
middleware.js gzip	25.5 kB	25.6 kB	N/A
edge-runtime..pack.js gzip	839 B	839 B	✓
Overall change	839 B	839 B	✓

Next Runtimes

	vercel/next.js canary	vercel/next.js chore/run-e2e-deploy-on-prs	Change
app-page-exp...dev.js gzip	171 kB	171 kB	✓
app-page-exp..prod.js gzip	97.5 kB	97.5 kB	✓
app-page-tur..prod.js gzip	99.2 kB	99.2 kB	✓
app-page-tur..prod.js gzip	93.5 kB	93.5 kB	✓
app-page.run...dev.js gzip	145 kB	145 kB	✓
app-page.run..prod.js gzip	92 kB	92 kB	✓
app-route-ex...dev.js gzip	21.5 kB	21.5 kB	✓
app-route-ex..prod.js gzip	15.2 kB	15.2 kB	✓
app-route-tu..prod.js gzip	15.2 kB	15.2 kB	✓
app-route-tu..prod.js gzip	14.9 kB	14.9 kB	✓
app-route.ru...dev.js gzip	21.1 kB	21.1 kB	✓
app-route.ru..prod.js gzip	14.9 kB	14.9 kB	✓
pages-api-tu..prod.js gzip	9.55 kB	9.55 kB	✓
pages-api.ru...dev.js gzip	9.82 kB	9.82 kB	✓
pages-api.ru..prod.js gzip	9.55 kB	9.55 kB	✓
pages-turbo...prod.js gzip	21.4 kB	21.4 kB	✓
pages.runtim...dev.js gzip	22.1 kB	22.1 kB	✓
pages.runtim..prod.js gzip	21.4 kB	21.4 kB	✓
server.runti..prod.js gzip	51.5 kB	51.5 kB	N/A
Overall change	894 kB	894 kB	✓

build cache

	vercel/next.js canary	vercel/next.js chore/run-e2e-deploy-on-prs	Change
0.pack gzip	1.59 MB	1.59 MB	N/A
index.pack gzip	107 kB	107 kB	N/A
Overall change	0 B	0 B	✓

Diff details

Diff for middleware.js

Diff too large to display

Diff for edge-ssr.js

Diff too large to display

Diff for server.runtime.prod.js

Diff too large to display

Commit: 4c24768