Fetch Cache misses on Build. API being called for every page generation twice on nextjs 13.5.4 #56355

MasteringBOX · 2023-10-03T10:13:55Z

Link to the code that reproduces this issue

https://github.com/MasteringBOX/fetch-bug

To Reproduce

Clone repository
Change call to your API to be able to check server logs or use another method to check how many times the API is being called.
Build and check API calls.

Current vs. Expected behavior

This project uses the app router.

Expected: I would expect the API to be called once when I call getData in generateStaticParams.

Current: I get 1 API call in generateStaticParams and then 2 API calls per page in a SSG page ([slug]/page). In a 5 SSG project I get 11 API calls instead of 1. In my current project this escalates to 1000 API calls instead of 10 in build.

I also get this error in some of the API calls, the error does not appear consistently through different versions of node and nextjs.

Failed to set fetch cache https://www.example.com/api/wp/v2/posts?per_page=5&page=1 TypeError: terminated
    at Fetch.onAborted (node:internal/deps/undici/undici:11000:53)
    at Fetch.emit (node:events:513:28)
    at Fetch.terminate (node:internal/deps/undici/undici:10272:14)
    at Object.onError (node:internal/deps/undici/undici:11095:36)
    at Request.onError (node:internal/deps/undici/undici:6477:31)
    at errorRequest (node:internal/deps/undici/undici:8440:17)
    at Socket.onSocketClose (node:internal/deps/undici/undici:7895:9)
    at Socket.emit (node:events:513:28)
    at TCP.<anonymous> (node:net:313:12)
    at TCP.callbackTrampoline (node:internal/async_hooks:130:17) {
  [cause]: SocketError: closed
      at Socket.onSocketClose (node:internal/deps/undici/undici:7883:35)
      at Socket.emit (node:events:513:28)
      at TCP.<anonymous> (node:net:313:12)
      at TCP.callbackTrampoline (node:internal/async_hooks:130:17) {
    code: 'UND_ERR_SOCKET',
    socket: {
      localAddress: undefined,
      localPort: undefined,
      remoteAddress: undefined,
      remotePort: undefined,
      remoteFamily: undefined,
      timeout: undefined,
      bytesWritten: 164152,
      bytesRead: 313
    }
  }

This happens in next 13.5.4 and 13.5.4-canary.11. It also fails in a lot of previous versions. 13.4.3 works ok in local in Node 20 and 18. In vercel I still get extra API calls in next 13.4.3 but not as many.

Verify canary release

I verified that the issue exists in the latest Next.js canary release

Provide environment information

Operating System:
  Platform: darwin
  Arch: arm64
  Version: Darwin Kernel Version 22.3.0: Mon Jan 30 20:39:46 PST 2023; root:xnu-8792.81.3~2/RELEASE_ARM64_T6020
Binaries:
  Node: 18.12.1
  npm: 8.19.2
  Yarn: N/A
  pnpm: N/A
Relevant Packages:
  next: 13.5.4
  eslint-config-next: 13.5.3
  react: 18.2.0
  react-dom: 18.2.0
  typescript: 5.2.2
Next.js Config:
  output: N/A

Which area(s) are affected? (Select all that apply)

App Router, Data fetching (gS(S)P, getInitialProps)

Additional context

I tested this in many nextjs versions. The latest I could find that works is 13.4.3. But even there the Vercel build will make a lot of extra calls, increasing build time and consuming more server resources.

Apart from the minimal reproducible example, I created another project with more fetch calls and pages that really showcases the severity of the issue. This one is not commented.
https://github.com/MasteringBOX/fetch-cache-bug

The text was updated successfully, but these errors were encountered:

domosedov · 2023-10-03T10:47:26Z

Have same problem
Local build works fine, but production build via Docker(using official with-docker example) has error

Failed to set fetch cache [my-content-api] TypeError: terminated
    at Fetch.onAborted (node:internal/deps/undici/undici:11000:53)
    at Fetch.emit (node:events:513:28)
    at Fetch.terminate (node:internal/deps/undici/undici:10272:14)
    at Object.onError (node:internal/deps/undici/undici:11095:36)
    at Request.onError (node:internal/deps/undici/undici:6477:31)
    at errorRequest (node:internal/deps/undici/undici:8440:17)
    at Socket.onSocketClose (node:internal/deps/undici/undici:7895:9)
    at Socket.emit (node:events:513:28)
    at TCP.<anonymous> (node:net:313:12)
    at TCP.callbackTrampoline (node:internal/async_hooks:130:17) {
  [cause]: SocketError: closed
      at Socket.onSocketClose (node:internal/deps/undici/undici:7883:35)
      at Socket.emit (node:events:513:28)
      at TCP.<anonymous> (node:net:313:12)
      at TCP.callbackTrampoline (node:internal/async_hooks:130:17) {
    code: 'UND_ERR_SOCKET',
    socket: {
      localAddress: undefined,
      localPort: undefined,
      remoteAddress: undefined,
      remotePort: undefined,
      remoteFamily: undefined,
      timeout: undefined,
      bytesWritten: 734984,
      bytesRead: 313
    }
  }
}

MasteringBOX · 2023-10-03T10:50:33Z

Please note that even if you don't get the error you may get duplicated API requests. Can you check your server access logs and confirm if you have more API requests than expected? Check this both in local and deployment because you may not get the error in local but the issue is there. I think the error happens when there's 2 requests being called simultanously and one fails to set Cache.

domosedov · 2023-10-03T11:59:26Z

Have same problem Local build works fine, but production build via Docker(using official with-docker example) has error

Failed to set fetch cache [my-content-api] TypeError: terminated
    at Fetch.onAborted (node:internal/deps/undici/undici:11000:53)
    at Fetch.emit (node:events:513:28)
    at Fetch.terminate (node:internal/deps/undici/undici:10272:14)
    at Object.onError (node:internal/deps/undici/undici:11095:36)
    at Request.onError (node:internal/deps/undici/undici:6477:31)
    at errorRequest (node:internal/deps/undici/undici:8440:17)
    at Socket.onSocketClose (node:internal/deps/undici/undici:7895:9)
    at Socket.emit (node:events:513:28)
    at TCP.<anonymous> (node:net:313:12)
    at TCP.callbackTrampoline (node:internal/async_hooks:130:17) {
  [cause]: SocketError: closed
      at Socket.onSocketClose (node:internal/deps/undici/undici:7883:35)
      at Socket.emit (node:events:513:28)
      at TCP.<anonymous> (node:net:313:12)
      at TCP.callbackTrampoline (node:internal/async_hooks:130:17) {
    code: 'UND_ERR_SOCKET',
    socket: {
      localAddress: undefined,
      localPort: undefined,
      remoteAddress: undefined,
      remotePort: undefined,
      remoteFamily: undefined,
      timeout: undefined,
      bytesWritten: 734984,
      bytesRead: 313
    }
  }
}

Replace FROM node:18-alpine AS base to FROM node:20-alpine AS base fix my issue

MasteringBOX · 2023-10-03T13:45:51Z

@domosedov what version of nextjs are you using? Have you checked in the server logs that you don't get multiple API calls for the same fetch?

diaz-hfc · 2023-10-03T15:36:58Z

This has been plaguing next js in the latest versions and the problem is that the response size is too big, so the cache just fails. You can find like 3 issues explaining the same issue.

domosedov · 2023-10-03T17:13:57Z

Have same problem Local build works fine, but production build via Docker(using official with-docker example) has error

Failed to set fetch cache [my-content-api] TypeError: terminated
    at Fetch.onAborted (node:internal/deps/undici/undici:11000:53)
    at Fetch.emit (node:events:513:28)
    at Fetch.terminate (node:internal/deps/undici/undici:10272:14)
    at Object.onError (node:internal/deps/undici/undici:11095:36)
    at Request.onError (node:internal/deps/undici/undici:6477:31)
    at errorRequest (node:internal/deps/undici/undici:8440:17)
    at Socket.onSocketClose (node:internal/deps/undici/undici:7895:9)
    at Socket.emit (node:events:513:28)
    at TCP.<anonymous> (node:net:313:12)
    at TCP.callbackTrampoline (node:internal/async_hooks:130:17) {
  [cause]: SocketError: closed
      at Socket.onSocketClose (node:internal/deps/undici/undici:7883:35)
      at Socket.emit (node:events:513:28)
      at TCP.<anonymous> (node:net:313:12)
      at TCP.callbackTrampoline (node:internal/async_hooks:130:17) {
    code: 'UND_ERR_SOCKET',
    socket: {
      localAddress: undefined,
      localPort: undefined,
      remoteAddress: undefined,
      remotePort: undefined,
      remoteFamily: undefined,
      timeout: undefined,
      bytesWritten: 734984,
      bytesRead: 313
    }
  }
}

Replace FROM node:18-alpine AS base to FROM node:20-alpine AS base fix my issue

new bugs

downgrade next to 13.4.12

MasteringBOX · 2023-10-04T23:29:40Z

@diaz-hfc I don't think the problem has to do with the response size. I've tried with very small sized response and I have the same issue. In my example project I fetch 5 blog posts and that is 28kb.

MasteringBOX · 2023-10-13T08:45:55Z

I'm not sure why this is not being paid attention to, its a big issue in my opinion.

nat-gaurav-saini · 2023-10-20T10:30:23Z

Facing same issue. @shuding @leerob. Sorry for tagging but could you please take a look.

samuelhorn · 2023-11-03T09:45:56Z

Also having this issue on every other build. Every time it's the same document, which has a large size. That makes me think it has to do with response size as well.

Weird thing is it works sometimes, and sometimes not. Locally it always works.

F7b5 · 2023-11-03T13:40:00Z

I've faced the same issue. After looking at the incremental cache, it seems to be related to this commit which added an IPC Server. By bypassing it manually the filesystem cache seems to be working again.

Just try and remove this from packages/next/src/server/lib/incremental-cache/index.ts, and then it works :

if ( process.env.__NEXT_INCREMENTAL_CACHE_IPC_PORT && process.env.__NEXT_INCREMENTAL_CACHE_IPC_KEY && process.env.NEXT_RUNTIME !== 'edge' ) { const invokeIpcMethod = require('../server-ipc/request-utils').invokeIpcMethod as typeof import('../server-ipc/request-utils').invokeIpcMethod return invokeIpcMethod({ method: 'set', ipcPort: process.env.__NEXT_INCREMENTAL_CACHE_IPC_PORT, ipcKey: process.env.__NEXT_INCREMENTAL_CACHE_IPC_KEY, args: [...arguments], }) }

This uses an IPC server (if available) for incremental cache methods to help prevent race conditions when reading/writing from cache and also to dedupe requests in cases where multiple cache reads are in flight. This cuts down on data fetching across the different build-time workers. Co-authored-by: JJ Kasper <22380829+ijjk@users.noreply.github.com>

MasteringBOX added the bug Issue was opened via the bug report template. label Oct 3, 2023

github-actions bot added the Pages Router Related to Pages Router. label Oct 3, 2023

MasteringBOX changed the title ~~Fetch Cache misses on Build. API being called for every page generation twice on nextjs 13.5.3~~ Fetch Cache misses on Build. API being called for every page generation twice on nextjs 13.5.4 Oct 3, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fetch Cache misses on Build. API being called for every page generation twice on nextjs 13.5.4 #56355

Fetch Cache misses on Build. API being called for every page generation twice on nextjs 13.5.4 #56355

MasteringBOX commented Oct 3, 2023 •

edited

Loading

domosedov commented Oct 3, 2023

MasteringBOX commented Oct 3, 2023

domosedov commented Oct 3, 2023

MasteringBOX commented Oct 3, 2023

diaz-hfc commented Oct 3, 2023 •

edited

Loading

domosedov commented Oct 3, 2023

MasteringBOX commented Oct 4, 2023

MasteringBOX commented Oct 13, 2023

nat-gaurav-saini commented Oct 20, 2023

samuelhorn commented Nov 3, 2023

F7b5 commented Nov 3, 2023

Fetch Cache misses on Build. API being called for every page generation twice on nextjs 13.5.4 #56355

Fetch Cache misses on Build. API being called for every page generation twice on nextjs 13.5.4 #56355

Comments

MasteringBOX commented Oct 3, 2023 • edited Loading

Link to the code that reproduces this issue

To Reproduce

Current vs. Expected behavior

Verify canary release

Provide environment information

Which area(s) are affected? (Select all that apply)

Additional context

domosedov commented Oct 3, 2023

MasteringBOX commented Oct 3, 2023

domosedov commented Oct 3, 2023

MasteringBOX commented Oct 3, 2023

diaz-hfc commented Oct 3, 2023 • edited Loading

domosedov commented Oct 3, 2023

MasteringBOX commented Oct 4, 2023

MasteringBOX commented Oct 13, 2023

nat-gaurav-saini commented Oct 20, 2023

samuelhorn commented Nov 3, 2023

F7b5 commented Nov 3, 2023

MasteringBOX commented Oct 3, 2023 •

edited

Loading

diaz-hfc commented Oct 3, 2023 •

edited

Loading