Skip to content

Commit

Permalink
Improve compile time on large application (#50792)
Browse files Browse the repository at this point in the history
## What?

While investigating slow compilation for a page on vercel.com in
development I found that there was close to 10 seconds of time
unaccounted for in `.next/trace`. Ran a profile and found that time was
spent in watchpack `batch`, specifically calling `close` many times.
When I tried to debug this further by running unbundled webpack I
noticed the same issue didn't exists.

### Before

<img width="1329" alt="Before"
src="https://github.com/vercel/next.js/assets/6324199/9ace4628-db04-4de7-993f-65aef9dffc55">

### After

<img width="1278" alt="After"
src="https://github.com/vercel/next.js/assets/6324199/55d5e58b-4a27-4235-8dea-723a7a78c117">

## Raw numbers

<table>
<tr>
 <td>Before</td>
 <td>After</td>
 <td>Delta</td>
 <td>Delta (percent)</td>
</tr>
<tr>
 <td>13840 ms</td>
 <td>3580 ms</td>
 <td>-10260 ms</td>
  <td>-74.13%</td>
</tr>
</table>

## How?

Investigated further and found that specifically not minifying watchpack
solved the issue.

<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:

## For Contributors

### Improving Documentation or adding/fixing Examples

- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md

### Fixing a bug

- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md

### Adding a feature

- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md



## For Maintainers

- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change

### What?

### Why?

### How?

Closes NEXT-
Fixes #

-->
  • Loading branch information
timneutkens authored Jun 5, 2023
1 parent 886b389 commit 8ab38ce
Show file tree
Hide file tree
Showing 9 changed files with 18 additions and 20 deletions.
1 change: 1 addition & 0 deletions packages/next/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@
"caniuse-lite": "^1.0.30001406",
"postcss": "8.4.14",
"styled-jsx": "5.1.1",
"watchpack": "2.4.0",
"zod": "3.21.4"
},
"peerDependencies": {
Expand Down
6 changes: 4 additions & 2 deletions packages/next/src/cli/next-dev.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import { findPagesDir } from '../lib/find-pages-dir'
import { findRootDir } from '../lib/find-root'
import { fileExists, FileType } from '../lib/file-exists'
import { getNpxCommand } from '../lib/helpers/get-npx-command'
import Watchpack from 'next/dist/compiled/watchpack'
import Watchpack from 'watchpack'
import stripAnsi from 'next/dist/compiled/strip-ansi'
import { getPossibleInstrumentationHookFilenames } from '../build/worker'

Expand Down Expand Up @@ -479,7 +479,9 @@ const nextDev: CliCommand = async (argv) => {
}

previousInstrumentationFiles.clear()
knownFiles.forEach((_, key) => previousInstrumentationFiles.add(key))
knownFiles.forEach((_: any, key: any) =>
previousInstrumentationFiles.add(key)
)
})

const projectFolderWatcher = new Watchpack({
Expand Down
2 changes: 1 addition & 1 deletion packages/next/src/compiled/sass-loader/cjs.js

Large diffs are not rendered by default.

1 change: 0 additions & 1 deletion packages/next/src/compiled/watchpack/watchpack.js

This file was deleted.

2 changes: 1 addition & 1 deletion packages/next/src/compiled/webpack/bundle5.js

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions packages/next/src/server/dev/next-dev-server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ import fs from 'fs'
import { Worker } from 'next/dist/compiled/jest-worker'
import findUp from 'next/dist/compiled/find-up'
import { join as pathJoin, relative, resolve as pathResolve, sep } from 'path'
import Watchpack from 'next/dist/compiled/watchpack'
import Watchpack from 'watchpack'
import { ampValidation } from '../../build/output'
import {
INSTRUMENTATION_HOOK_FILENAME,
Expand Down Expand Up @@ -128,7 +128,7 @@ export interface Options extends ServerOptions {
export default class DevServer extends Server {
private devReady: Promise<void>
private setDevReady?: Function
private webpackWatcher?: Watchpack | null
private webpackWatcher?: any | null
private hotReloader?: HotReloader
private isCustomServer: boolean
protected sortedRoutes?: string[]
Expand Down
9 changes: 1 addition & 8 deletions packages/next/taskfile.js
Original file line number Diff line number Diff line change
Expand Up @@ -591,13 +591,7 @@ export async function ncc_next_font(task, opts) {
}

// eslint-disable-next-line camelcase
externals['watchpack'] = 'next/dist/compiled/watchpack'
export async function ncc_watchpack(task, opts) {
await task
.source(relative(__dirname, require.resolve('watchpack')))
.ncc({ packageName: 'watchpack', externals })
.target('src/compiled/watchpack')
}
externals['watchpack'] = 'watchpack'

// eslint-disable-next-line camelcase
externals['jest-worker'] = 'next/dist/compiled/jest-worker'
Expand Down Expand Up @@ -2195,7 +2189,6 @@ export async function ncc(task, opts) {
.parallel(
[
'ncc_node_html_parser',
'ncc_watchpack',
'ncc_chalk',
'ncc_napirs_triples',
'ncc_p_limit',
Expand Down
4 changes: 4 additions & 0 deletions packages/next/types/misc.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,10 @@ declare module 'next/dist/compiled/babel/preset-env' {
const anyType: any
export default anyType
}
declare module 'watchpack' {
const anyType: any
export default anyType
}
declare module 'next/dist/compiled/babel/core' {
export * from '@babel/core'
}
Expand Down
9 changes: 4 additions & 5 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit 8ab38ce

Please sign in to comment.