Serverless Next: make `next` dev-only dependency, introduce `next-server` for smaller builds and faster bootup

The problem: when optimizing for a production build, invoking next start or using require('next') within a custom server.js involves bringing the entire set of next dependencies, including the ones related exclusively to development, such as webpack.

Not only is this problematic from a build image standpoint and download time performance when generating production builds, but it also likely hurts bootup time. Note: This is lessened by the fact that we carefully lazily load heavy dependencies such as webpack in dev mode.

For the performance conscious and those sensitive to cold start times (see for example: https://twitter.com/rauchg/status/990667331205447680), we can introduce a next-server package.

It would have the same capabilities as require('next') minus all development-time settings, plus a very small next-server CLI that can open a port and perform graceful shutdown.

What we want to optimize for:

• The total dependency set of next-server has to be as small as possible
• We must heavily optimize for bootup time to start as quickly as possible

Furthermore, we should provide an example in examples/ of how to use next-server in combination with pkg to export your Next.js application as a self-contained ELF binary.

[ElvenMonky]

@timneutkens serverless target doesn’t and can not solve problems with custom server, which uses dynamic routes. #5927 Is not a solution for a lot of real world business applications like in my case, where we have to use dynamically generated pages, assets prefix, custom _app, _document and _err: basically everything stated in TODO list. next-server gives us partial solution to deploy to production without weird development only dependencies, like webpack and babel. This can be done however with some hacks and woodoo dancing, that we are discussing here.

I was under impression that you understand this difference and was hoping to see more robust solution someday to the initial issue as it is described by @rauchg

[rauchg]

The cold start times we see on Now 2.0 for our frontend are 1.5s, for an image size of 80mb IIRC

It should be possible to make it a lot closer to 1s without any changes to Node or V8 or any of the dependencies whose cold evaluation take a good chunk of time (like react and react-dom)