Next.js development high memory usage

[timneutkens]

Before posting a comment on this issue please read this entire post.

Previous work

The past few weeks we've been investigating / optimizing various memory usage issues. Specifically geared towards production memory usage. In investigating these we were able to find there was one memory leak in Node.js itself when using fetch() in Node.js versions before 18.17.0 (you'll want to use 18.17.1 for security patches though).

Most of the reports related to memory usage turned out to be reports of "it's higher than the previous version" rather than a memory leak. This was expected because in order to run App Router and Pages Router at the same time with different React versions two separate processes were needed. This has been resolved by reducing the amount of processes to two, one for Routing and App Router rendering, and one for Pages Router rendering. So far we haven't received new reports since the latest release.

In some issues there were reports related to Image Optimization in production, however no reproduction was provided so it could not be investigated adequately, if you have a reproduction for that please refer to this issue: #54482

New

With the memory usage in production resolved we've started investigating reports of development memory usage spikes. Unfortunately these reports suffer from the same problem as the production memory usage issues people raised before, they're full of comments saying same issue or posting screenshots of monitoring tools saying "Look, same issue".

Unfortunately, as you can image, these replies are not enough to investigate / narrow down what causes the memory usage, for example in multiple cases that we did get a reproduction and could investigate the reason for the high memory usage was:

• A bug in the application code, causing infinite looping components
• Accidental import of ~11.000 modules through icon libraries -- Yes, many icon libraries ship massive amounts of re-exports and those have to be bundled before it can be tree-shaken. We've been working on an automated way to split these up in optimize_barrel SWC transform and new optimizePackageImports config #54572 that should help a bit to reduce the size (and compilation speed too).
• Webpack customization, e.g. adding external libraries that change webpack settings, for example to write additional sourcemaps

So far I've been able to make one small change to webpack's memory caching to make it garbage collect a bit more aggressively in #54397. I'm not expecting that change to have a big impact on the reported issues though.

We'd like to investigate these reports further, however we're unable to narrow these down if there is no code to run to collect heap snapshots and profiles for, hence this issue. If you are able to please provide runnable code of what you're experiencing.

Comments that don't include runnable code will be automatically hidden in order to keep this issue productive. This includes comments that only have a screenshot and applications that can't run.

I'm going to combine the other reports into this issue as separate comments.

I've made sure that we have 2-3 engineers on our team available to investigate when we get runnable reproductions to investigate.

Thanks in advance!

_NEXT-1569

[timneutkens]

@noetix Disagree that this had anything to do with bullying. All I did was explain that my posts are continuously being ignored, including in this issue. Then I explained what to do, which is to provide a runnable application. There is nothing we can do without a runnable example, which was already shared in the initial issue, I even made it bold to highlight that further.

Happy to explain it again, the reason we can't do anything without runnable code is that in order to narrow down the memory usage we need to change the Next.js core code in the application, for example to disable client components compilation and such in order to narrow down where the memory usage comes from. There is no way to do that based on screenshots / messages / information you can provide as it would require countless hours of your time and our time (think 2 weeks full time at least) in order to investigate this.

The emoji reactions not being shown for off-topic marked posts is a bug in GitHub. As mentioned in the initial issue any posts that don't include a reproduction will be automatically hidden.

Since you didn't like the earlier explanation I'll just remove it, don't feel strongly about keeping the comment. Definitely wasn't bullying, you were reading into that. Bullying would be the threats I've received recently from anonymous developers on Twitter that they'll come visit my house soon...

@weyert we haven't made changes to development memory usage besides the PR linked in the issue so really all I need is a reproduction, luckily @AhmedChabayta posted one, hopefully that is enough, fingers crossed.

@Thinkscape please open a separate issue, that bug would be separate from this issue 👍

[limeburst]

I've posted a reproduction here: https://github.com/limeburst/vercel-nextjs-54708

Start the development server, navigate from

• http://localhost:3000/1
• http://localhost:3000/2
• ...
• http://localhost:3000/20

And watch the memory usage grow, until the server restarts.

[rajat1saxena]

You can use this code to see the issue. The server is getting aborted silently without any errors.

https://github.com/codelitdev/courselit/tree/tailwindcss-2

Logs

rajat@rajat-laptop:~/projects/courselit$ yarn dev
- info Loaded env from /home/rajat/projects/courselit/apps/web/.env.local
- info Loaded env from /home/rajat/projects/courselit/apps/web/.env
- ready started server on [::]:3000, url: http://localhost:3000
- event compiled client and server successfully in 545 ms (18 modules)
- wait compiling...
- event compiled client and server successfully in 263 ms (18 modules)
- info Loaded env from /home/rajat/projects/courselit/apps/web/.env.local
- info Loaded env from /home/rajat/projects/courselit/apps/web/.env
- info Loaded env from /home/rajat/projects/courselit/apps/web/.env.local
- info Loaded env from /home/rajat/projects/courselit/apps/web/.env
- wait compiling /404 (client and server)...
- wait compiling / (client and server)...
rajat@rajat-laptop:~/projects/courselit$

[stigmartins1]

I'm getting 'server out of memory' after a while by letting the server run and writing/saving code that calls the following functions a few times.

import { google } from "googleapis";

export async function authSheets() {
  //Function for authentication object
  const auth = new google.auth.GoogleAuth({
    keyFile: "./auth/auth-sa-sptk.json",
    scopes: ["https://www.googleapis.com/auth/spreadsheets"],
  });

  //Create client instance for auth
  const authClient = await auth.getClient();

  //Instance of the Sheets API
  const sheets = google.sheets({ version: "v4", auth: authClient });

  return {
    auth,
    authClient,
    sheets,
  };
}

import { authSheets } from "./authSheets";

export async function clearSheetContents(sheetName) {
  console.log("sheet =", sheetName);
  const SHEET_ID = "123";
  const sheetId = SHEET_ID;
  const { sheets } = await authSheets();

  try {
    const result = await sheets.spreadsheets.values.clear({
      spreadsheetId: sheetId,
      range: sheetName,
    });
    console.log("result.data =", result.data);
  } catch (err) {
    // TODO (developer) - Handle exception
    throw err;
  }
}

import { authSheets } from "./authSheets";

// https://developers.google.com/sheets/api/guides/values
export async function setSheetValues(sheetName, input) {
  const SHEET_ID = "123";
  const sheetId = SHEET_ID;
  const values = [input];
  const resource = { values };
  // Updates require a valid ValueInputOption parameter
  const valueInputOption = "RAW"; // The input is not parsed and is inserted as a string.
  const { sheets } = await authSheets();

  try {
    const result = await sheets.spreadsheets.values.append({
      spreadsheetId: sheetId,
      range: sheetName,
      valueInputOption: valueInputOption,
      resource,
    });
    console.log("result.data =", result.data);
  } catch (err) {
    // TODO (developer) - Handle exception
    throw err;
  }
}

{
  "name": "test",
  "version": "0.1.0",
  "private": true,
  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "lint": "next lint"
  },
  "dependencies": {
    "autoprefixer": "10.4.15",
    "axios": "^1.5.0",
    "encoding": "^0.1.13",
    "eslint": "8.48.0",
    "eslint-config-next": "13.4.19",
    "googleapis": "^126.0.1",
    "next": "13.4.19",
    "postcss": "8.4.29",
    "react": "18.2.0",
    "react-dom": "18.2.0",
    "tailwindcss": "3.3.3"
  }
}

Node v. v18.17.1

[yogithesymbian]

Potential memory leak in Nextjs Tylerastro/NCU_TOM#12

we have same issue my engine mbpm1 . i think i have wrong choosing nextjs 14 now its complicated . still looking for other solutions . the apps its not very big and its only one page already crap . antd nextjs 14 tailwind .

[wintercounter]

For those who are having issues with react-icons.

I currently have to work on a laptop with only 8G memory. I couldn't even render the first page before Next dying. I created this tool that you can use to purge all unused icons from the source files. Only for development for those who are completely blocked by this.

https://github.com/wintercounter/purge-react-icons

[elie222]

For those who are having issues with react-icons.

I currently have to work on a laptop with only 8G memory. I couldn't even render the first page before Next dying. I created this tool that you can use to purge all unused icons from the source files. Only for development for those who are completely blocked by this.

wintercounter/purge-react-icons

Does lucide react have similar issues?

[wintercounter]

Shouldn't, same for goes react-icons if you only use one icon pack. The issue is when projects are using multiple packs, then too many barrel files will get loaded. The project I'm working on currently is mainly using Lucide, but it also loads some icons from 12 other packs xD

[Royal-lobster]

next-server taking around 4-7gb for our codebase, attaching traces, hope it helps to solve this issue

• https://gist.github.com/Royal-lobster/c86c7c5dd887f4a06daa19800c229375
• https://gist.github.com/Royal-lobster/ecf87144f4d64ac304cb75276208801e
• https://gist.github.com/Royal-lobster/539c41c4e1035619810d13794da8a002

[yes-oo]

for now is there any workaround to this issue , i have a 12gb ram laptop and i can't continue programming my website , it's always 90% to 100% full after navigating to multiple web pages in dev mode , i hope this issue got solved soon , my entire system ram usage is 1.7 the remaining is eaten by dev mode

[mrgoonie]

for now is there any workaround to this issue , i have a 12gb ram laptop and i can't continue programming my website , it's always 90% to 100% full after navigating to multiple web pages in dev mode , i hope this issue got solved soon , my entire system ram usage is 1.7 the remaining is eaten by dev mode

These worked to me so far:

• Don't use fetch, switch to axios
• Use exactly Node v20.15.1 (not lower, not higher)

Read more here: #64212

[yes-oo]

• v20.15.1

Thank you for your suggestion. I appreciate your time. I tried your solution, but it seems to be the same, if not worse, than the latest version of Node.js. As for fetch and Axios, I don't use fetch anywhere in my project, so that's not the issue. I'm also using the latest version of Next.js to avoid any bugs that have already been resolved

[yogithesymbian]

for now is there any workaround to this issue , i have a 12gb ram laptop and i can't continue programming my website , it's always 90% to 100% full after navigating to multiple web pages in dev mode , i hope this issue got solved soon , my entire system ram usage is 1.7 the remaining is eaten by dev mode

These worked to me so far:

• Don't use fetch, switch to axios
• Use exactly Node v20.15.1 (not lower, not higher)

Read more here: #64212

I tested with a new project and haven’t used Axios, Fetch, or any other libraries yet—it's a fresh project, but I'm still encountering the issue. nextjs antd tailwind .

my terminal consume 7GB memory .
engine : mbpm1 ( macOS sequoia 15.0.1 )

edit : "I think it’s consuming my disk space (5GB - 7GB) because it's overloaded. My friend, working on the same project, is using 13GB of memory."

[hunghuy201280]

mine consumed ~13GB of memory which is insane

[delphinic-owl]

mine consumed ~13GB of memory which is insane

marginally beat you !

[shahargl]

is there any workaround for it?

[alex-statsig]

One thing I've uncovered recently is a stateful server can easily leak memory in development only. While nextjs does encourage a stateless server (I suppose you need to use a separate server otherwise), its easy to write code that subtly causes the same issue. These are pretty painful to fix today, but with module.hot support it could be much easier (although discussions on that seem stagnant)

Some examples:

• Using a setInterval and never clearing it (ex. to clear some global cache periodically, or flush logs, etc.)
• Adding a global hook (that captures module-scoped variables in its closure, which is easy to do accidentally). This includes configuring axios-retry, process.on lifecycle hooks, adding mongoose schemas, etc.
• Using packages that add global intervals or hooks, such as a database client without properly cleaning up any singletons

These can all lead to that version of the module (and its downstream dependencies) never getting garbage collected (since there's a dangling reference to it). This means every time you save a server module, a new copy of all your server modules is added to memory. If you re-save server files a lot, and have a large backend, this can lead to OOMs quickly.

The only workaround I've found today is to pollute the global namespace with cleanup functions that get invoked when the module is re-imported (so you can shutdown the old database client, or clear the old interval, etc.)

[yuriiholskiy]

Hey,
any updates on this?
We have a simple app with some requests with fetch, but It takes so much memory on server that it is insane?

Any workarounds on this?

[tocteman]

10gb RAM usage on a rather fresh project. I think this is a very important issue.
How can we diagnose/profile what's causing this massive RAM usage ourselves?

[jpmnteiro]

Same boat here.

Running nextjs (v14.x) takes 9.2GB~ RAM. Switching on --turbo brings the memory usage to 13.2GB~ RAM.

Is there any way to investigate what's causing the increase in memory usage?

[broscr]

The cause of the RAM overflow is using fetch. I had the same issue; I replaced all fetch requests with axios, and the project never went over 200MB of RAM after that.

[KillerCodeMonkey]

@broscr did you tried nextjs v15 before? i think fetch caching is deactivated per default, so there should no such strange memory usages, now

[broscr]

@broscr did you tried nextjs v15 before? i think fetch caching is deactivated per default, so there should no such strange memory usages, now

Yes I tried nextjs v15.0.1

[yogithesymbian]

 webpack: (config, { dev }) => {
    if (config.cache && !dev) {
      config.cache = Object.freeze({
        type: "memory",
      });
    }
    // Important: Return the modified config
    return config;
  },

doesnt work still in case consume high memory
here is my fullnext.config.js

// /* uncomment for analyze */
const withBundleAnalyzer = require("@next/bundle-analyzer")({
  enabled: process.env.ANALYZE === "true",
});

module.exports = withBundleAnalyzer({
  reactStrictMode: false, //Disable strict mode
  webpack: (config, { dev }) => {
    if (config.cache && !dev) {
      config.cache = Object.freeze({
        type: "memory",
      });
    }
    // Important: Return the modified config
    return config;
  },
  productionBrowserSourceMaps: false, // Disable source maps in development
  optimizeFonts: true, // Disable font optimization
  compiler:
    (process.env.NODE_ENV === "production") == true
      ? {
          styledComponents: true,
          // removeConsole: true,
          // removeConsole: {
          //   exclude: ["error"],
          // },
        }
      : {
          styledComponents: true,
          // removeConsole: { // laters exclude by --turbo
          //   exclude: ["error"],
          // },
        },
  swcMinify: true,
  transpilePackages: ["lodash-es"],
  experimental: {
    turbo: {
      rules: {
        "*.svg": {
          loaders: ["@svgr/webpack"],
          as: "*.js",
        },
      },
      resolveAlias: {
        underscore: "lodash",
        mocha: { browser: "mocha/browser-entry.js" },
      },
      resolveExtensions: [
        ".mdx",
        ".tsx",
        ".ts",
        ".jsx",
        ".js",
        ".mjs",
        ".json",
      ],
      moduleIdStrategy: "deterministic",
    },
    staleTimes: {
      dynamic: 30,
      static: 180,
    },
    optimizePackageImports: [
      "antd",
      "@ant-design/icons",
      "@ant-design/pro-components",
      "@emotion/css",
      "date-fns",
      "lodash-es",
      "ahooks",
      "crypto-js",
      "dayjs",
    ],
  },
  env: {
    use_otp: "1",
    use_otp_simulation: "1",
  },
});

[tnfssc]

Workaround for Linux users.

You can use the following command to limit the memory usage of the dev server. The compilations get slower when your memory limit is too small.

systemd-run --user --scope -p MemoryMax=4000M pnpm dev

Source: https://unix.stackexchange.com/a/536046

EDIT: This command limits the physical memory usage only. If your system doesn't have swap memory allocated, this method will crash the process.

[wintercounter]

How is this a workaround? Next dev server will keep restarting itself if it reaches the memory threshold when using Webpack. When using Turbopack it'll just silently kill itself.

…

On Tue, 26 Nov 2024, 06:21 Sharath, ***@***.***> wrote: Workaround for Linux users. You can use the following command to limit the memory usage of the dev server. The compilations get slower when your memory limit too small. systemd-run --user --scope -p MemoryMax=4000M pnpm dev Source: https://unix.stackexchange.com/a/536046 — Reply to this email directly, view it on GitHub <#54708 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHLJQGYBBXJ5KDH5GADIFD2CQANBAVCNFSM6AAAAAA4CX4LH6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOJZGY4DKMBWGM> . You are receiving this because you commented.Message ID: <vercel/next. ***@***.***>

[tnfssc]

How is this a workaround? Next dev server will keep restarting itself if it reaches the memory threshold when using Webpack. When using Turbopack it'll just silently kill itself.
…
On Tue, 26 Nov 2024, 06:21 Sharath, @.> wrote: Workaround for Linux users. You can use the following command to limit the memory usage of the dev server. The compilations get slower when your memory limit too small. systemd-run --user --scope -p MemoryMax=4000M pnpm dev Source: https://unix.stackexchange.com/a/536046 — Reply to this email directly, view it on GitHub <#54708 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHLJQGYBBXJ5KDH5GADIFD2CQANBAVCNFSM6AAAAAA4CX4LH6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOJZGY4DKMBWGM . You are receiving this because you commented.Message ID: <vercel/next. @.>

@wintercounter I tried it on my machine with 2000MB limit with turbo. The process instantly hit 1950+MB as expected. But it stayed there. The compilation times were a tad slower, but the program never crashed. I tried 200MB just for fun. Compilation took forever, but the program never crashed.

I asked my teammate to try the same after posting the workaround here, he faced the exact same issue as you described. Could it be related to my hardware configuration? I use Ubuntu 24.04 on a Ryzen 9. He didn't have swap memory allocated.

EDIT: Confirmed that it increases my swap usage. Most linux installs don't allocate swap memory, so it's crashing on them.

[TylerNRobertson]

This is still very much an issue. Currently running into 6-7 GB's used just for the ideal server, I'm not even navigating around the app.

Any tips or insights would be awesome, seems like this issue has been open for a while

[schnubor]

This is insane, it seems like it got worse with 15.1?

Same app, cold start on a 32GB machine:

With Turbopack:

With Webpack (much better):

[guillaume-g]

Same here ! I would love to help or give feedbacks 😊

[gho1b]

This is insane, it seems like it got worse with 15.1?

Same app, cold start on a 32GB machine:

With Turbopack:

With Webpack (much better):

So, is this problem coming from turbopack and not nextjs, isn't?