Lazy compilation during development

[rauchg]

In order to make Next.js very performant, we need to address the following problems:

• When you run next, everything gets compiled eagerly
• Every time you save a file, whether it's relevant to the rendered routes in your browser or not, it gets eagerly compiled

The beautiful thing about our new compilation output is that it makes these problems very obvious and clear :)

[rauchg]

Added bonus: pre-fetching would trigger the lazy compilation of additional routes, so that's a very neat way of "doing work in the background".

Side note: a priority queue in place to always put direct routing work ahead of prefetching would be nice (but not a must)

[arunoda]

I didn't get this 100%. Is this what do you mean.

• When run next, we don't compile any pages. (No webpack build for pages)
• Then a user hit a page on the browser, we compile that page.

[rauchg]

Correct. Right now it builds everything. If anything, I'd rather bind to the port immediately, and then we could start compiling / since that's what the user is likely to hit next

[arunoda]

Okay. That's interesting.
We need to chat with @nkzawa with a bit on how we could do it.
And we need to figure our whether it actually improves the build time.

[nkzawa]

We used to have most of this feature on the old unpublished version of next.js. So actually, we can say it definitely improves your development experience a lot. The only issue I feel on this feature is it makes response time of your app totally different compared to production.

I have no concrete idea how we could do it but maybe should wait for webpack/webpack#3634 to be merged, which should make things a lot easier.

[arunoda]

@nkzawa I thought dynamic entries are something already supported by webpack.
Okay, I think waiting for that seems like a pretty good idea.

[rauchg]

@nkzawa it's true that it introduces a difference with production. We should have a flag next --eager perhaps to achieve an accurate simulation

[arunoda]

@rauchg @nkzawa I think we need to find a better solution than this one.
See this repo: https://github.com/tgoldenberg/next-sample (##918)

Basically it only have a single page, but with source maps enabled it took around ~10 seconds. Even without webpack it took around ~2 seconds for changes. Mostly this is because of the use of material-ui.

[arunoda]

I'm going to see whether adding tree-shaking support gives us something better.

[arunoda]

There's a way to do this properly. We could pick all the common modules and serve them as a DDL. So, nothing inside pages directory deals with Webpack at all. So, that's where we could get the speed.

I think we need to work hard to do this automatically, but we don't wanna do it like that. I'll try to work on this. If we can land this before 2.0, that would be huge.

[timneutkens]

This was implemented in #1111. Please do re-open if needed 😄