Split build HTML step in to batches

[Bouncey]

This is a feature request/question

At freeCodeCamp/guides we have ~2400 pages, which takes a --max_old_space_size of 4GB to build successfully. We are looking to expand the page count somewhere north of 10000 pages eventually, but we are bound by RAM limits due to the way webpack works.

Is it feasible to split the HTML build in to batches of say 500 pages each and build each batch separately? Would this help webpack release its hold on memory?

I have found the static-entry, but I would appreciate a pointer as to how this could be accomplished. I am more than willing to PR it.

[Bouncey]

Is #445 related?

[KyleAMathews]

@Bouncey sorry for missing this issue! Yeah #445 is definitely related and would greatly ease the memory pressure larger sites are seeing.

[KyleAMathews]

I'll add a bit more detail on #445 — would love a PR for it!

[benjaminhoffman]

thx @Bouncey for the issue! Doing issue cleanup so closing this for now in favor of 445. Feel free to re-open if I'm mistaken or if you think your problem is different enough to justify its own issue.. Thanks again!