Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add full monorepo support for npm, yarn, and pnpm #23

Merged
merged 1 commit into from
Jan 4, 2023
Merged

Add full monorepo support for npm, yarn, and pnpm #23

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 102 additions & 9 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,12 @@ OpenNext aims to support all Next.js 13 features. Some features are work in prog

If your Next.js app does not use [middleware](https://nextjs.org/docs/advanced-features/middleware), `middleware-function` will not be generated.

3. Add `.open-next` to your `.gitignore` file
```
# OpenNext
/.open-next/
```

## How does OpenNext work?

When calling `open-next build`, OpenNext **builds the Next.js app** using the `@vercel/next` package. And then it **transforms the build output** to a format that can be deployed to AWS.
Expand Down Expand Up @@ -111,17 +117,55 @@ public,max-age=0,s-maxage=31536000,must-revalidate

#### Image optimization function

Create a Lambda function with the code from `.open-next/image-optimization-function`.
Create a Lambda function with the code from `.open-next/image-optimization-function`, and the handler is `index.mjs`.

This function handles image optimization requests when the Next.js `<Image>` component is used. The [sharp](https://www.npmjs.com/package/sharp) library is bundled with the function. And it is used to convert the image.

Note that image optimization function responds with the `Cache-Control` header, and the image will be cached both at the CDN level and at the browser level.

#### Server Lambda function

Create a Lambda function with the code from `.open-next/server-function`.
Create a Lambda function with the code from `.open-next/server-function`, and the handler is `index.mjs`.

This function handles all the other types of requests from the Next.js app, including Server-side Rendering (SSR) requests and API requests. OpenNext builds the Next.js app in the **standalone** mode. The standalone mode generates a `.next` folder containing the **NextServer** class that does the request handling. It also generates a `node_modules` folder with **all the dependencies** required to run the `NextServer`.

```
.next/ -> NextServer
node_modules/ -> dependencies
```

The server function adapter wraps around `NextServer` and exports a handler function that supports the Lambda request and response. The `server-function` bundle looks like:

```diff
.next/ -> NextServer
node_modules/ -> dependencies
+ index.mjs -> server function adapter
```

**Monorepo**
The build output looks slightly different when the Next.js app is part of a monorepo. Imagine the app sits inside `packages/web`, the build output looks like:

```
packages/
web/
.next/ -> NextServer
node_modules/ -> dependencies from root node_modules (optional)
node_modules/ -> dependencies from package node_modules
```

This function handles all the other types of requests from the Next.js app, including Server-side Rendering (SSR) requests and API requests. OpenNext builds the Next.js app in the **standalone** mode. The standalone mode generates a **NextServer** class that does the request handling. And the server function wraps around the NextServer.
In this case, the server function adapter needs to be created inside `packages/web` next to `.next/`. This is to ensure the adapter can import dependencies from both `node_modules` folders. We could set the Lambda handler to point to `packages/web/index.mjs`, but it is a bad practice to have the Lambda configuration coupled with the project structure. Instead, we will add a wrapper `index.mjs` at the `server-function` bundle root that re-exports the adapter.

```diff
packages/
web/
.next/ -> NextServer
node_modules/ -> dependencies from root node_modules (optional)
+ index.mjs -> server function adapter
node_modules/ -> dependencies from package node_modules
+ index.mjs -> adapter wrapper
```

This ensure the Lambda handler remains at `index.mjs`.

#### CloudFront distribution

Expand All @@ -137,7 +181,7 @@ Create a CloudFront distribution, and dispatch requests to their cooresponding h

#### Middleware Lambda@Edge function (optional)

Create a Lambda function with the code from `.open-next/middleware-function`, and attach it to the `/_next/data/*` and `/*` behaviors as `viewer request` edge function. This allows the function to run your [Middleware](https://nextjs.org/docs/advanced-features/middleware) code before the request hits your server function, and also before cached content.
Create a Lambda function with the code from `.open-next/middleware-function`, and the handler is `index.mjs`. Attach it to the `/_next/data/*` and `/*` behaviors as `viewer request` edge function. This allows the function to run your [Middleware](https://nextjs.org/docs/advanced-features/middleware) code before the request hits your server function, and also before cached content.

The middleware function uses the Node.js 18 [global fetch API](https://nodejs.org/de/blog/announcements/v18-release-announce/#new-globally-available-browser-compatible-apis). It requires to run on Node.js 18 runtime. [See why Node.js 18 runtime is required.](#workaround-add-headersgetall-extension-to-the-middleware-function)

Expand Down Expand Up @@ -169,6 +213,55 @@ To workaround the issue, the server function checks if the request is to an HTML
public, max-age=0, s-maxage=31536000, must-revalidate
```

#### WORKAROUND: Set `NextServer` working directory (AWS specific)

Next.js recommends using `process.cwd()` instead of `__dirname` to get the app directory. Imagine you have a `posts` folder in your app with markdown files:

```
pages/
posts/
my-post.md
public/
next.config.js
package.json
```

And you can build the file path like this:

```ts
path.join(process.cwd(), "posts", "my-post.md");
```

Recall in the [Server function](#server-lambda-function) section. In a non-monorepo setup, the `server-function` bundle looks like:

```
.next/
node_modules/
posts/
my-post.md <- path is "posts/my-post.md"
index.mjs
```

And `path.join(process.cwd(), "posts", "my-post.md")` resolves to the correct path.

However, when the user's app is inside a monorepo (ie. at `/packages/web`), the `server-function` bundle looks like:

```
packages/
web/
.next/
node_modules/
posts/
my-post.md <- path is "packages/web/posts/my-post.md"
index.mjs
node_modules/
index.mjs
```

And `path.join(process.cwd(), "posts", "my-post.md")` cannot be resolved.

To workaround the issue, we change the working directory for the server function to where `.next/` is located, ie. `packages/web`.

#### WORKAROUND: Pass headers from middleware function to server function (AWS specific)

[Middleware](https://nextjs.org/docs/advanced-features/middleware) allows you to modify the request and response headers. This requires the middleware function to be able to pass custom headers defined in your Next.js app's middleware code to the server function.
Expand Down Expand Up @@ -210,22 +303,22 @@ To run `OpenNext` locally:
1. Build `open-next`
```bash
cd open-next
yarn build
pnpm build
```
1. Run `open-next` in watch mode
```bash
yarn dev
pnpm dev
```
1. Make `open-next` linkable from your Next.js app
```bash
yarn link
pnpm link --global
```
1. Link `open-next` in your Next.js app
```bash
cd path/to/my/nextjs/app
yarn link open-next
pnpm link --global open-next
```
Now you can make changes in `open-next`, and run `yarn open-next build` in your Next.js app to test the changes.
Now you can make changes in `open-next`, and run `pnpm open-next build` in your Next.js app to test the changes.

## FAQ

Expand Down
76 changes: 7 additions & 69 deletions src/adapters/server-adapter.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import type {
import NextServer from "next/dist/server/next-server.js";
import { loadConfig } from "./util.js"

setNextjsServerWorkingDirectory();
const nextDir = path.join(__dirname, ".next");
const config = loadConfig(nextDir);
const htmlPages = loadHtmlPages();
Expand Down Expand Up @@ -49,8 +50,6 @@ const server = slsHttp(
{
binary: true,
provider: "aws",
// TODO: add support for basePath
//basePath: process.env.NEXTJS_LAMBDA_BASE_PATH,
},
);

Expand Down Expand Up @@ -90,76 +89,15 @@ export async function handler(event: APIGatewayProxyEventV2, context: Context):
// Helper functions //
//////////////////////

function setNextjsServerWorkingDirectory() {
// WORKAROUND: Set `NextServer` working directory (AWS specific) — https://github.com/serverless-stack/open-next#workaround-set-nextserver-working-directory-aws-specific
process.chdir(__dirname);
}

function loadHtmlPages() {
const filePath = path.join(nextDir, "server", "pages-manifest.json");
const json = fs.readFileSync(filePath, "utf-8");
return Object.entries(JSON.parse(json))
.filter(([_, value]) => (value as string).endsWith(".html"))
.map(([key]) => key);
}

//const createApigHandler = () => {
// const config = loadConfig();
// const requestHandler = new NextServer(config).getRequestHandler();
//
// return async (event) => {
// const request = convertApigRequestToNext(event);
// const response = await requestHandler(request);
// return convertNextResponseToApig(response);
// };
//};
//
//export const handler = createApigHandler();

//function convertApigRequestToNext(event) {
// let host = event.headers["x-forwarded-host"] || event.headers.host;
// let search = event.rawQueryString.length ? `?${event.rawQueryString}` : "";
// let scheme = "https";
// let url = new URL(event.rawPath + search, `${scheme}://${host}`);
// let isFormData = event.headers["content-type"]?.includes(
// "multipart/form-data"
// );
//
// // Build headers
// const headers = new Headers();
// for (let [header, value] of Object.entries(event.headers)) {
// if (value) {
// headers.append(header, value);
// }
// }
//
// return new Request(url.href, {
// method: event.requestContext.http.method,
// headers,
// body:
// event.body && event.isBase64Encoded
// ? isFormData
// ? Buffer.from(event.body, "base64")
// : Buffer.from(event.body, "base64").toString()
// : event.body,
// });
//}
//
//async function convertNextResponseToApig(response) {
// // Build cookies
// // note: AWS API Gateway will send back set-cookies outside of response headers.
// const cookies = [];
// for (let [key, values] of Object.entries(response.headers.raw())) {
// if (key.toLowerCase() === "set-cookie") {
// for (let value of values) {
// cookies.push(value);
// }
// }
// }
//
// if (cookies.length) {
// response.headers.delete("Set-Cookie");
// }
//
// return {
// statusCode: response.status,
// headers: Object.fromEntries(response.headers.entries()),
// cookies,
// body: await response.text(),
// };
//}
}
38 changes: 21 additions & 17 deletions src/build.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,8 @@ const outputDir = ".open-next";
const tempDir = path.join(outputDir, ".build");

export async function build() {
printVersion();

// Pre-build validation
printHeader("Validating Next.js app");
printVersion();
checkRunningInsideNextjsApp();
setStandaloneBuildMode();
const monorepoRoot = findMonorepoRoot();
Expand Down Expand Up @@ -46,7 +44,7 @@ function findMonorepoRoot() {
|| fs.existsSync(path.join(currentPath, "yarn.lock"))
|| fs.existsSync(path.join(currentPath, "pnpm-lock.yaml"))) {
if (currentPath !== appPath) {
console.info("Monorepo root detected at", currentPath);
console.info("Monorepo detected at", currentPath);
}
return currentPath;
}
Expand Down Expand Up @@ -78,9 +76,11 @@ function buildNextjsApp(monorepoRoot: string) {
function printHeader(header: string) {
header = `OpenNext — ${header}`;
console.info([
"",
"┌" + "─".repeat(header.length + 2) + "┐",
`│ ${header} │`,
"└" + "─".repeat(header.length + 2) + "┘",
"",
].join("\n"));
}

Expand Down Expand Up @@ -116,25 +116,19 @@ function createServerBundle(monorepoRoot: string) {
path.join(outputPath),
{ recursive: true, verbatimSymlinks: true }
);

// Resolve path to the Next.js app if inside the monorepo
// note: if user's app is inside a monorepo, standalone mode places
// `node_modules` inside `.next/standalone`, and others inside
// `.next/standalone/package/path` (ie. `.next`, `server.js`).
// We need to move them to the root of the output folder.
if (monorepoRoot) {
const packagePath = path.relative(monorepoRoot, appPath);
fs.readdirSync(path.join(outputPath, packagePath))
.forEach(file => {
fs.renameSync(
path.join(outputPath, packagePath, file),
path.join(outputPath, file)
);
});
}
// We need to output the handler file inside the package path.
const isMonorepo = monorepoRoot !== appPath;
const packagePath = path.relative(monorepoRoot, appPath);

// Standalone output already has a Node server "server.js", remove it.
// It will be replaced with the Lambda handler.
fs.rmSync(
path.join(outputPath, "server.js"),
path.join(outputPath, packagePath, "server.js"),
{ force: true }
);

Expand All @@ -145,7 +139,7 @@ function createServerBundle(monorepoRoot: string) {
esbuildSync({
entryPoints: [path.join(__dirname, "adapters", "server-adapter.js")],
external: ["next"],
outfile: path.join(outputPath, "index.mjs"),
outfile: path.join(outputPath, packagePath, "index.mjs"),
banner: {
js: [
"import { createRequire as topLevelCreateRequire } from 'module';",
Expand All @@ -155,6 +149,16 @@ function createServerBundle(monorepoRoot: string) {
].join(""),
},
});
// note: in the monorepo case, the handler file is output to
// `.next/standalone/package/path/index.mjs`, but we want
// the Lambda function to be able to find the handler at
// the root of the bundle. We will create a dummy `index.mjs`
// that re-exports the real handler.
if (isMonorepo) {
fs.writeFileSync(path.join(outputPath, "index.mjs"), [
`export * from "./${packagePath}/index.mjs";`,
].join(""))
};
}

function createImageOptimizationBundle() {
Expand Down