Skip to content

Commit

Permalink
feat: add support for browser rendering (#271)
Browse files Browse the repository at this point in the history
  • Loading branch information
atinux authored Sep 9, 2024
1 parent bc269e5 commit 0f928aa
Show file tree
Hide file tree
Showing 22 changed files with 1,480 additions and 573 deletions.
4 changes: 2 additions & 2 deletions docs/content/0.index.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ hero:
light: '/images/landing/hero-light.svg'
dark: '/images/landing/hero-dark.svg'
headline:
label: Using AI for User Experience
to: /blog/cloudflare-ai-for-user-experience
label: Browser rendering is available
to: /changelog/hub-browser
icon: i-ph-arrow-right
links:
- label: Start reading docs
Expand Down
4 changes: 2 additions & 2 deletions docs/content/1.docs/2.features/ai.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,11 +169,11 @@ NuxtHub AI is compatible with some functions of the [Vercel AI SDK](https://sdk
Make sure to install the Vercel AI SDK in your project.

```[Terminal]
npx nypm add ai @ai-sdk/vue
npx ni ai @ai-sdk/vue
```

::note
[`nypm`](https://github.com/unjs/nypm) will detect your package manager and install the dependencies with it.
[`ni`](https://github.com/antfu/ni) will detect your package manager and install the dependencies with it.
::

### `useChat()`
Expand Down
230 changes: 230 additions & 0 deletions docs/content/1.docs/2.features/browser.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,230 @@
---
title: Browser Rendering
navigation.title: Browser
description: Control and interact with a headless browser instance in your Nuxt application using Puppeteer.
---

## Getting Started

Enable browser rendering in your Nuxt project by enabling the `hub.browser` option:

```ts [nuxt.config.ts]
export default defineNuxtConfig({
hub: {
browser: true
},
})
```

Lastly, install the required dependencies by running the following command:

```bash [Terminal]
npx ni @cloudflare/puppeteer puppeteer
```

::note
[ni](https://github.com/antfu/ni) will automatically detect the package manager you are using and install the dependencies.
::

## Usage

In your server API routes, you can use the `hubBrowser` function to get a [Puppeteer browser instance](https://github.com/puppeteer/puppeteer):

```ts
const { page, browser } = await hubBrowser()
```

In production, the instance will be from [`@cloudflare/puppeteer`](https://developers.cloudflare.com/browser-rendering/platform/puppeteer/) which is a fork of Puppeteer with version specialized for working within Cloudflare workers.

::tip
NuxtHub will automatically close the `page` instance when the response is sent as well as closing or disconnecting the `browser` instance when needed.
::

## Use Cases

Here are some use cases for using a headless browser like Puppeteer in your Nuxt application:
- **Web scraping:** Extract data from websites, especially those with dynamic content that requires JavaScript execution.
- **Generating PDFs or screenshots:** Create snapshots or PDF versions of web pages.
- **Performance monitoring:** Measure load times, resource usage, and other performance metrics of web applications.
- **Automating interactions or testing:** Simulating user actions on websites for tasks like form filling, clicking buttons, or navigating through multi-step processes.

## Limits

::important
Browser rendering is only available on the [Workers Paid](https://www.cloudflare.com/plans/developer-platform/) plan for now.
::

To improve the performance in production, NuxtHub will reuse browser sessions. This means that the browser will stay open after each request (for 60 seconds), a new request will reuse the same browser session if available or open a new one.

The Cloudflare limits are:
- 2 new browsers per minute per Cloudflare account
- 2 concurrent browser sessions per account
- a browser instance gets killed if no activity is detected for 60 seconds (idle timeout)

You can extend the idle timeout by giving the `keepAlive` option when creating the browser instance:

```ts
// keep the browser instance alive for 120 seconds
const { page, browser } = await hubBrowser({ keepAlive: 120 })
```

The maximum idle timeout is 600 seconds (10 minutes).

::tip
Once NuxtHub supports [Durable Objects](https://github.com/nuxt-hub/core/issues/50), you will be able to create a single browser instance that will stay open for a long time, and you will be able to reuse it across requests.
::

## Screenshot Capture

Taking a screenshot of a website is a common use case for a headless browser. Let's create an API route to capture a screenshot of a website:

```ts [server/api/screenshot.ts]
import { z } from 'zod'

export default eventHandler(async (event) => {
// Get the URL and theme from the query parameters
const { url, theme } = await getValidatedQuery(event, z.object({
url: z.string().url(),
theme: z.enum(['light', 'dark']).optional().default('light')
}).parse)

// Get a browser session and open a new page
const { page } = await hubBrowser()

// Set the viewport to full HD & set the color-scheme
await page.setViewport({ width: 1920, height: 1080 })
await page.emulateMediaFeatures([{
name: 'prefers-color-scheme',
value: theme
}])

// Go to the URL and wait for the page to load
await page.goto(url, { waitUntil: 'domcontentloaded' })

// Return the screenshot as response
setHeader(event, 'content-type', 'image/jpeg')
return page.screenshot()
})
```

On the application side, we can create a simple form to call our API endpoint:

```vue [pages/capture.vue]
<script setup>
const url = ref('https://hub.nuxt.com')
const image = ref('')
const theme = ref('light')
const loading = ref(false)
async function capture {
if (loading.value) return
loading.value = true
const blob = await $fetch('/api/browser/capture', {
query: {
url: url.value,
theme: theme.value
}
})
image.value = URL.createObjectURL(blob)
loading.value = false
}
</script>
<template>
<form @submit.prevent="capture">
<input v-model="url" type="url" />
<select v-model="theme">
<option value="light">Light</option>
<option value="dark">Dark</option>
</select>
<button type="submit" :disabled="loading">
{{ loading ? 'Capturing...' : 'Capture' }}
</button>
<img v-if="image && !loading" :src="image" style="aspect-ratio: 16/9;" />
</form>
</template>
```

That's it! You can now capture screenshots of websites using Puppeteer in your Nuxt application.

### Storing the screenshots

You can store the screenshots in the Blob storage:

```ts
const screenshot = await page.screenshot()

// Upload the screenshot to the Blob storage
const filename = `screenshots/${url.value.replace(/[^a-zA-Z0-9]/g, '-')}.jpg`
const blob = await hubBlob().put(filename, screenshot)
```

::note{to="/docs/features/blob"}
Learn more about the Blob storage.
::

## Metadata Extraction

Another common use case is to extract metadata from a website.

```ts [server/api/metadata.ts]
import { z } from 'zod'

export default eventHandler(async (event) => {
// Get the URL from the query parameters
const { url } = await getValidatedQuery(event, z.object({
url: z.string().url()
}).parse)

// Get a browser instance and navigate to the url
const { page } = await hubBrowser()
await page.goto(url, { waitUntil: 'networkidle0' })

// Extract metadata from the page
const metadata = await page.evaluate(() => {
const getMetaContent = (name) => {
const element = document.querySelector(`meta[name="${name}"], meta[property="${name}"]`)
return element ? element.getAttribute('content') : null
}

return {
title: document.title,
description: getMetaContent('description') || getMetaContent('og:description'),
favicon: document.querySelector('link[rel="shortcut icon"]')?.href
|| document.querySelector('link[rel="icon"]')?.href,
ogImage: getMetaContent('og:image'),
origin: document.location.origin
}
})

return metadata
})
```

Visiting `/api/metadata?url=https://cloudflare.com` will return the metadata of the website:

```json
{
"title": "Connect, Protect and Build Everywhere | Cloudflare",
"description": "Make employees, applications and networks faster and more secure everywhere, while reducing complexity and cost.",
"favicon": "https://www.cloudflare.com/favicon.ico",
"ogImage": "https://cf-assets.www.cloudflare.com/slt3lc6tev37/2FNnxFZOBEha1W2MhF44EN/e9438de558c983ccce8129ddc20e1b8b/CF_MetaImage_1200x628.png",
"origin": "https://www.cloudflare.com"
}
```

To store the metadata of a website, you can use the [Key Value Storage](/docs/features/kv).

Or directly leverage [Caching](/docs/features/cache) on this API route:

```ts [server/api/metadata.ts]
export default cachedEventHandler(async (event) => {
// ...
}, {
maxAge: 60 * 60 * 24 * 7, // 1 week
swr: true,
// Use the URL as key to invalidate the cache when the URL changes
// We use btoa to transform the URL to a base64 string
getKey: (event) => btoa(getQuery(event).url),
})
```
2 changes: 1 addition & 1 deletion docs/content/1.docs/3.recipes/5.postgres.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ The module ensures that you can connect to your PostgreSQL database using [Cloud
2. Install the [`postgres`](https://www.npmjs.com/package/postgres) NPM package in your project.

```bash
npx nypm add postgres
npx ni postgres
```

::tip{icon="i-ph-rocket-launch"}
Expand Down
66 changes: 66 additions & 0 deletions docs/content/4.changelog/hub-browser.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
---
title: Browser Rendering is available
description: "Taking screenshots, crawling websites, extracting information has never been easier with `hubBrowser()`."
date: 2024-08-28
image: '/images/changelog/nuxthub-browser.jpg'
authors:
- name: Sebastien Chopin
avatar:
src: https://avatars.githubusercontent.com/u/904724?v=4
to: https://x.com/atinux
username: atinux
---

::tip
This feature is available on both [free and pro plans](/pricing) of NuxtHub but on the [Workers Paid plan](https://www.cloudflare.com/plans/developer-platform/) for your Cloudflare account.
::

We are excited to introduce [`hubBrowser()`](/docs/features/browser). This new method allows you to run a headless browser directly in your Nuxt application using [Puppeteer](https://github.com/puppeteer/puppeteer).

::video{poster="https://res.cloudinary.com/nuxt/video/upload/v1725901706/nuxthub/nuxthub-browser_dsn1m1.jpg" controls class="w-full h-auto rounded"}
:source{src="https://res.cloudinary.com/nuxt/video/upload/v1725901706/nuxthub/nuxthub-browser_dsn1m1.webm" type="video/webm"}
:source{src="https://res.cloudinary.com/nuxt/video/upload/v1725901706/nuxthub/nuxthub-browser_dsn1m1.mov" type="video/mp4"}
:source{src="https://res.cloudinary.com/nuxt/video/upload/v1725901706/nuxthub/nuxthub-browser_dsn1m1.ogg" type="video/ogg"}
::

## How to use hubBrowser()

1. Update `@nuxthub/core` to the latest version (`v0.7.11` or later)

2. Enable `hub.browser` in your `nuxt.config.ts`

```ts [nuxt.config.ts]
export default defineNuxtConfig({
hub: {
browser: true
}
})
```

3. Install the required dependencies

```bash [Terminal]
npx ni @cloudflare/puppeteer puppeteer
```

4. Start using [`hubBrowser()`](/docs/features/browser) in your server routes

```ts [server/api/screenshot.ts]
export default eventHandler(async (event) => {
const { page } = await hubBrowser()

await page.setViewport({ width: 1920, height: 1080 })
await page.goto('https://cloudflare.com')

setHeader(event, 'content-type', 'image/jpeg')
return page.screenshot()
})
```

5. Before deploying, make sure you are subscribed to the [Workers Paid plan](https://www.cloudflare.com/plans/developer-platform/)

6. [Deploy your project with NuxtHub](/docs/getting-started/deploy)

::note{to="/docs/features/browser"}
Read the documentation about `hubBrowser()` with more examples.
::
2 changes: 1 addition & 1 deletion docs/pages/index.vue
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ onMounted(() => {
</template>

<template #title>
<MDC :value="page?.hero.title" />
<span v-html="page?.hero.title" />
</template>

<template #description>
Expand Down
Binary file added docs/public/images/changelog/nuxthub-browser.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
5 changes: 3 additions & 2 deletions playground/app/app.vue
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,10 @@ useSeoMeta({
const links = [
{ label: 'Home', to: '/' },
{ label: 'AI', to: '/ai' },
{ label: 'Browser', to: '/browser' },
{ label: 'Blob', to: '/blob' },
{ label: 'Database', to: '/database' },
{ label: 'KV', to: '/kv' },
{ label: 'Blob', to: '/blob' }
{ label: 'KV', to: '/kv' }
]
</script>

Expand Down
Loading

0 comments on commit 0f928aa

Please sign in to comment.