Unable to build and deploy - not enough memory

[ryanchristo]

Description

The build process is currently broken due to a lack of memory. The size of this project has grown considerably and running gatsby build now results in errors both locally and remotely.

The latest commit on the generator branch (d80911c) produced the following error on CircleCI:

Build timed out after 5h0m0s

Recent pull requests (#121 and #131) produced the following error on CircleCI:

Error: ENOMEM: not enough memory, read

Attempting to build locally produced the following error:

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

The issue was reported in gatbsy#13980 with potential solutions related to versions, changing the CPU count, and increasing max-old-space-size but no final solution was provided.

[ryanchristo]

I was able to successfully build locally after removing ipfs as a dependency but this is obviously not a solution. We need to optimize, increase the memory limit (locally and on CircleCI), and/or start thinking about splitting the website and developer portal into separate repositories.

[ryanchristo]

Replacing gatsby build with node --max_old_space_size=7168 node_modules/gatsby-cli build seems to resolve the issue locally, which was a solution mentioned in gatsby#175. I imagine we still should do some optimization but this seems like a viable workaround for now.

[ryanchristo]

Still an issue remotely on CircleCI 🤔

[ceolson01]

I'm pretty sure our CI instance is 2cpus & 4096mb

[ryanchristo]

#133 was a temporary solution to unblock the build and deploy process. This issue will remain open until we find a solution that will allow us to use ipfs in the developer portal. Finding a solution that will allow us to keep the current setup would be ideal, otherwise, a few alternatives might include the following:

• Switch CIs or upgrade CircleCI so that we can increase CPUs
• Move ipfs to the api so that we can store and retrieve ipfs hashes from there
• Switch to a monorepo so that we can bundle core, website, and portal separately ❤️

[ceolson01]

No longer an issue, after removal of some large dependencies, most notably ipfs. Closing