[NEXT-1314] High memory usage in deployed Next.js project

[ProchaLu]

Verify canary release

• I verified that the issue exists in the latest Next.js canary release

Provide environment information

Operating System:
      Platform: darwin
      Arch: arm64
      Version: Darwin Kernel Version 22.1.0: Sun Oct  9 20:14:30 PDT 2022; root:xnu-8792.41.9~2/RELEASE_ARM64_T8103
    Binaries:
      Node: 18.15.0
      npm: 9.5.0
      Yarn: 1.22.19
      pnpm: 8.5.0
    Relevant packages:
      next: 13.4.3-canary.1
      eslint-config-next: N/A
      react: 18.2.0
      react-dom: 18.2.0
      typescript: 5.0.4

Which area(s) of Next.js are affected? (leave empty if unsure)

No response

Link to the code that reproduces this issue

https://codesandbox.io/p/github/ProchaLu/next-js-ram-example/

To Reproduce

• Clone the repository: https://github.com/ProchaLu/next-js-ram-example
• Deploy the project on Fly.io

Describe the Bug

I have been working on a small project to reproduce an issue related to memory usage in Next.js. The project is built using the Next.js canary version 13.4.3-canary.1. It utilizes Next.js with App Router and Server Actions and does not use a database.

The problem arises when deploying the project on different platforms and observing the memory usage behavior. I have deployed the project on multiple platforms for testing purposes, including Vercel and Fly.io.

• On Vercel: https://next-js-ram-example.vercel.app/
  When interacting with the deployed version on Vercel, the project responds as expected. The memory usage remains stable and does not show any significant increase or latency

• On Fly.io: https://memory-test.fly.dev/
  However, when deploying the project on Fly.io, I noticed that the memory usage constantly remains around 220 MB, even during normal usage scenarios

Expected Behavior

I expect the small project to run smoothly without encountering any memory-related issues when deployed on various platforms, including Fly.io. Considering the previous successful deployment on Fly.io, which involved additional resource usage and utilized Next.js 13 with App Router and Server Actions, my anticipation is that the memory usage will remain stable and within acceptable limits.

Fly.io discussion: https://community.fly.io/t/high-memory-usage-in-deployed-next-js-project/12954?u=upleveled

Which browser are you using? (if relevant)

Chrome

How are you deploying your application? (if relevant)

Vercel, fly.io

_NEXT-1314

[thexpand]

Is this related to Server Actions, have you isolated the case?

[TheBit]

@thexpand This is not related to Server Actions. It is a severe memory leak starting from v13.3.5-canary.9. I was going to open a bug but found this one.

@shuding I suspect your PR #49116 as others in mentioned canary are not likely to cause this. Can you please take a look? This blocks us from upgrading to the latest Next.js.

Operating System:
      Platform: darwin
      Arch: arm64
      Version: Darwin Kernel Version 22.4.0: Mon Mar  6 20:59:58 PST 2023; root:xnu-8796.101.5~3/RELEASE_ARM64_T6020
    Binaries:
      Node: 18.15.0
      npm: 9.5.0
      Yarn: 1.22.19
      pnpm: 7.9.0
    Relevant packages:
      next: 13.4.4-canary.0
      eslint-config-next: N/A
      react: 18.2.0
      react-dom: 18.2.0
      typescript: 4.6.2

Tech Stack:

• Rush.js monorepo (on pnpm)
• +several Next.js apps in SSG (no SSR, no app dir but with next/image (not legacy one) and with middleware.js and few redirects and rewrites in next.config.js)
• +all of which uses latest (v7) common/storybook as 90% of theirs code base (so transpileModules is used for it)
• +external configuration and MaterialUI styles comes from Apollo Client used in getStaticProps pointing towards Strapi GraphQL (so on-demand revalidation is used)
• and everything deployed to our Kubernetes (build and run-time both are using latest Bullseye Debian so that sharp for next/image is working correctly) and monitored in Grafana
• BTW, Next.js is in standalone mode

Proofs:

13.3.5-canary.8 vs 13.3.5-canary.9

13.3.5-canary.8 vs 13.3.5-canary.9 with all images unoptimized

13.3.5-canary.8 vs 13.4.4-canary.0 (to test latest canary) with all images unoptimized + middleware removed

So, as you can see, the leak comes not from next/image or middleware, and the only PR which theoretically could cause this from canary.9 is this as for me: #49116

P.S. I also checked13.3.4 and found no leakage there. But on this version, we get Internal Server Error from middleware so can't use it, so I had to find a minimum canary version where this problem has been fixed - and this version is https://github.com/vercel/next.js/releases/tag/v13.3.5-canary.2, so we lock on this version for now (probably this PR #48723 fixed middleware problem)

[Josehower]

@shuding or @ijjk any thoughts on this issue? Can you confirm a current memory leak in Next.js?

[Josehower]

I have being constantly receiving this warning in the latest version of Next.js (13.4.4) when visiting pages in my app. I am preparing a reproduction repo but for now i just want to add some info that i have gathered:

1. The error is more prominent in environments with 256MB of ram (very common in free allowances apps like fly.io). locally i have trigger this error by using NODE_OPTIONS=--max-old-space-size=200 pnpm dev
2. The error happens when I navigate relatively fast in between pages. I can only guess here but maybe there is a high amount of memory during the caching process.
3. The error seems to be more frequent when visiting pages that perform server side code like a dabase query

- wait compiling /_error (client and server)...
(node:35717) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 server@app@/page listeners added to [EventEmitter]. Use emitter.setMaxListeners() to increase limit
(node:35717) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 server@app@/page3/page listeners added to [EventEmitter]. Use emitter.setMaxListeners() to increase limit

[Josehower]

I created this reproduction repo using the latest canary version of Next.js for the error documented before. In the repo i am using auto-cannon to request multiple pages very fast, simulating traffic to the website.

i documented this in a new issue since seems a different error #50909

[Josehower]

I created a different reproduction repo using the latest canary version of Next.js. The error is crashing the dev server when an import is missing.

https://nextjs.org/docs/messages/module-not-found

<--- Last few GCs --->

[2218:0x5eb9a70]    40167 ms: Mark-sweep 252.1 (263.9) -> 250.1 (263.7) MB, 206.0 / 0.0 ms  (average mu = 0.174, current mu = 0.125) allocation failure scavenge might not succeed
[2218:0x5eb9a70]    40404 ms: Mark-sweep 252.4 (263.9) -> 250.6 (264.2) MB, 216.7 / 0.0 ms  (average mu = 0.135, current mu = 0.086) allocation failure scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0xb02930 node::Abort() [/usr/local/bin/node]
 2: 0xa18149 node::FatalError(char const*, char const*) [/usr/local/bin/node]
 3: 0xcdd16e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 4: 0xcdd4e7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 5: 0xe94b55  [/usr/local/bin/node]
 6: 0xe95636  [/usr/local/bin/node]
 7: 0xea3b5e  [/usr/local/bin/node]
 8: 0xea45a0 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/bin/node]
 9: 0xea751e v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/usr/local/bin/node]
10: 0xe68a5a v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/usr/local/bin/node]
11: 0x11e17c6 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/usr/local/bin/node]
12: 0x15d5439  [/usr/local/bin/node]

I documented this in a new issue since it seems a different error #51025

[Josehower]

I created a reproduction using Docker to showcase how a Simple project using Next.js crashes when being used in environtments with ~225MB.

Steps to reproduce:

1. run docker pull josehower/next-js-memory-leak-reproduction-example:latest
2. run docker run -p 3000:3000 --memory=225m josehower/next-js-memory-leak-reproduction-example:latest
   • NOTE: in some environments the app is not even running with this memory restriction in this case add more memory --memory=256m
3. visit http://localhost:3000/
4. click fire
5. confirm the app is turning unresponsive and throwing the following error

Error: socket hang up
    at connResetException (node:internal/errors:717:14)
    at Socket.socketOnEnd (node:_http_client:526:23)
    at Socket.emit (node:events:525:35)
    at endReadableNT (node:internal/streams/readable:1359:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  code: 'ECONNRESET'
}

6. you can confirm the app is running when removing the --memory=225m option from the command

The way I recreated this Reproduction was by creating a Docker image from a simple Next.js app using autocannon to fake traffic to the website from a button.

• Created the reproduction repo
• Adding a Dockerfile and setup the image
• building with docker build . -t <username>/<image-name>:<version>
• publishing to Docker with docker push docker push <username>/<image-name>:<version>

[Starefossen]

Is this related to #49677 maybe?

[TheBit]

@Starefossen thanx! I've read that issue and found one connection with my setup: #49677 (comment) I also run my Next.js with NODE_OPTIONS='--dns-result-order=ipv4first' due to this, which was done here: nodejs/node#39987, and our migration from Node 16 to 18.

I will try to find time and repeat my upgrade without this option or some other change related to this. I can see that it already fixes the problem for this developer.