-
Notifications
You must be signed in to change notification settings - Fork 5
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
4937787
commit 8a95651
Showing
4 changed files
with
339 additions
and
185 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,218 @@ | ||
| <!DOCTYPE html> | ||
| <html lang="en"> | ||
| <head> | ||
| <meta charset="utf-8"> | ||
| <title>Using Google Chrome instead of Chromium in Google Cloud Functions</title> | ||
| <meta name="viewport" content="width=device-width, initial-scale=1"> | ||
| <script>document.documentElement.classList.add(localStorage.theme || ((matchMedia && matchMedia('(prefers-color-scheme: dark)').matches) ? 'dark' : 'light'))</script> | ||
| <link rel="stylesheet" href="/css/normalize.css"> | ||
| <link rel="stylesheet" href="/css/github-20220617.css"> | ||
| <link rel="stylesheet" href="/css/main-20230317.css"> | ||
| <link rel="icon" href="/favicon.ico" data-emoji="🏕️"> | ||
| <link rel="alternate" href="/feed.xml" type="application/atom+xml" title="CodeJam"> | ||
| <meta property="og:title" content="Using Google Chrome instead of Chromium in Google Cloud Functions"> | ||
| <meta property="og:description" content="When using Puppeteer, Playwright and similar, you need to have Chrome installed. When you’re running on AWS Lambda or Google Cloud Functions, it can get tricky."> | ||
| <meta property="og:image" content="https://www.codejam.info/img/profile.jpg"> | ||
| <meta name="twitter:card" content="summary"> | ||
| <meta name="twitter:site" content="@valeriangalliat"> | ||
| <meta name="twitter:creator" content="@valeriangalliat"> | ||
| </head> | ||
| <body class="post"> | ||
| <nav> | ||
| <ul> | ||
| <li><a href="/">Home</a></li> | ||
| <li><a href="/val.html">About</a></li> | ||
| <li><a href="https://photography.codejam.info/">Photography</a></li> | ||
| <li class="pull"><button title="Toggle theme" class="change-color-theme">🌝</button></li> | ||
| </ul> | ||
| </nav> | ||
| <header> | ||
| <div class="header-inner"> | ||
| <div class="content"> | ||
| <h1>Using Google Chrome instead of Chromium in Google Cloud Functions</h1> | ||
| <p class="date">May 5, 2024</p> | ||
| </div> | ||
| </div> | ||
| </header> | ||
| <div class="content"> | ||
| <p>When using Puppeteer, Playwright and similar, you need to have Chrome | ||
| installed. When you’re running on AWS Lambda or Google Cloud Functions, | ||
| it can get tricky.</p> | ||
| <p>Google Cloud Functions <em>used to</em> bundle Chromium in their base images, | ||
| but it’s been a few years it’s no longer the case. That’s where packages | ||
| like <a href="https://github.com/alixaxel/chrome-aws-lambda"><code>chrome-aws-lambda</code></a> | ||
| come in handy, by bundling Chromium directly inside a npm package, and | ||
| exposing a function that extracts the Chromium binary and returns the | ||
| path:</p> | ||
| <pre><code class="hljs language-js"><span class="hljs-keyword">const</span> chromium = <span class="hljs-built_in">require</span>(<span class="hljs-string">'chrome-aws-lambda'</span>) | ||
|
|
||
| <span class="hljs-keyword">const</span> path = <span class="hljs-keyword">await</span> chromium.<span class="hljs-property">executablePath</span> | ||
| </code></pre> | ||
| <div class="note"> | ||
| <p><strong>Note:</strong> unnecessary pedantic detail: the above code doesn’t look like a function, | ||
| but <a href="https://github.com/alixaxel/chrome-aws-lambda/blob/f9d5a9ff0282ef8e172a29d6d077efc468ca3c76/source/index.ts#L147">it is, in fact</a>, | ||
| a getter function that returns a promise. 😄</p> | ||
| </div> | ||
| <p>However that’s Chromium, and you may have reasons to want Google Chrome | ||
| instead (mainly, proprietary codecs).</p> | ||
| <h2 id="a-totally-unrelated-note-about-aws-lambda" tabindex="-1"><a class="header-anchor" href="#a-totally-unrelated-note-about-aws-lambda"><span>A totally unrelated note about AWS Lambda</span></a></h2> | ||
| <p>This article is about Google Cloud Functions, but if you’re on AWS | ||
| Lambda, the above option is your best bet. Because of the Lambda total | ||
| size limit of 250 MB (all layers combined), it’s really hard to get a | ||
| binary of Chrome that fits in there.</p> | ||
| <p>That’s why <code>chrome-aws-lambda</code> uses <a href="https://github.com/alixaxel/lambdafs">LambdaFS</a> | ||
| under the hood, to aggressively compress the Chrome installation with | ||
| Brotli and make it fit in that limited space.</p> | ||
| <p>But again with that build, you won’t have proprietary codecs. I tried to | ||
| trim down a Chrome Linux build and compress it with the same technique | ||
| but never managed to make it fit on AWS Lambda. Recent Chrome versions | ||
| are just too big.</p> | ||
| <p>There’s another option, which is to compile Chromium yourself with | ||
| proprietary codecs. I never found any prebuilt binaries of Chromium that | ||
| include proprietary codecs (maybe because of license issues | ||
| redistributing them 🙃) so you’re on your own here.</p> | ||
| <p><a href="https://www.remotion.dev/">Remotion</a> successfully does that for | ||
| <a href="https://www.remotion.dev/docs/lambda">Remotion Lambda</a>. | ||
| Here’s <a href="https://github.com/remotion-dev/chrome-build-instructions">their instructions</a> | ||
| to compile Chromium with proprietary codecs for Lambda.</p> | ||
| <p>Fair warning: it gets hairy, fast.</p> | ||
| <h2 id="back-to-google-cloud-functions" tabindex="-1"><a class="header-anchor" href="#back-to-google-cloud-functions"><span>Back to Google Cloud Functions</span></a></h2> | ||
| <p>Google Cloud Functions is more generous as for bundle size, so we don’t | ||
| need to resort to those tricks, and we can include a complete, | ||
| uncompressed, Google Chrome installation.</p> | ||
| <p>Google publishes <a href="https://googlechromelabs.github.io/chrome-for-testing/">Chrome for Testing</a>, | ||
| builds <a href="https://developer.chrome.com/blog/chrome-for-testing">specifically made</a> | ||
| for headless usage.</p> | ||
| <p>We can just download the latest build from there as part of the | ||
| <code>gcp-build</code> script in our <code>package.json</code>.</p> | ||
| <pre><code class="hljs language-json"><span class="hljs-punctuation">{</span> | ||
| <span class="hljs-attr">"scripts"</span><span class="hljs-punctuation">:</span> <span class="hljs-punctuation">{</span> | ||
| <span class="hljs-attr">"gcp-build"</span><span class="hljs-punctuation">:</span> <span class="hljs-string">"curl -s -O 'https://storage.googleapis.com/chrome-for-testing-public/124.0.6367.91/linux64/chrome-linux64.zip' && unzip chrome-linux64.zip && rm chrome-linux64.zip"</span> | ||
| <span class="hljs-punctuation">}</span> | ||
| <span class="hljs-punctuation">}</span> | ||
| </code></pre> | ||
| <div class="note"> | ||
| <p><strong>Note:</strong> the <code>gcp-build</code> script allows you to <a href="https://cloud.google.com/appengine/docs/standard/nodejs/running-custom-build-step">run a custom build step</a> | ||
| in Google Cloud Build, which is what Cloud Functions (both 1st and 2nd | ||
| gen, as well as Cloud Run and App Engine) use to build your function | ||
| image.</p> | ||
| <p>It would work just fine with a <code>postinstall</code> script as well, but | ||
| <code>gcp-build</code> makes sure you run it only on Google Cloud Build, which is | ||
| probably desirable in this particular case.</p> | ||
| </div> | ||
| <p>You will then have the Chrome binary in <code>chrome-linux64/chrome</code>, that | ||
| you can pass to the tool of your choice.</p> | ||
| <h2 id="with-puppeteer" tabindex="-1"><a class="header-anchor" href="#with-puppeteer"><span>With Puppeteer</span></a></h2> | ||
| <p>Courtesy of <a href="https://medium.com/@jackklpan/run-puppeteer-in-google-cloud-functions-v2-b18a353e609b">this post</a>, | ||
| with Puppeteer, you don’t need to download Chrome manually, since it | ||
| provides a nifty script to do just that.</p> | ||
| <p>Actually, Puppeteer’s <a href="https://github.com/puppeteer/puppeteer/blob/f23646b3526aa87145c17b22e9967ec8f77d82d2/packages/puppeteer/package.json#L41"><code>postinstall</code> script</a> | ||
| automatically downloads the latest version of Chrome for Testing for | ||
| your platform.</p> | ||
| <p>The caveat is that this script by default installs it to | ||
| <code>~/.cache/puppeteer</code>, which in the case of Google Cloud Build, is not | ||
| gonna be preserved in the final image. So we need to instruct Puppeteer | ||
| to install Chrome in a directory that Cloud Build will keep.</p> | ||
| <p>This can be done with the following <code>.puppeteerrc.js</code>:</p> | ||
| <pre><code class="hljs language-js"><span class="hljs-variable language_">module</span>.<span class="hljs-property">exports</span> = { | ||
| <span class="hljs-attr">cacheDirectory</span>: <span class="hljs-string">`<span class="hljs-subst">${__dirname}</span>/.cache/puppeteer`</span> | ||
| } | ||
| </code></pre> | ||
| <p>But even then, there’s another caveat. Puppeteer’s <code>postinstall</code> script | ||
| will only run after it gets installed. However, because of build | ||
| caching, you will get in a state where <code>node_modules</code> is restored, with | ||
| Puppeteer already installed (so <code>postinstall</code> will <em>not</em> run), but the | ||
| <code>.cache/puppeteer</code> directory will also <em>not</em> be restored.</p> | ||
| <p>To mitigate that, we need to make sure to install Chrome systematically. | ||
| Again we can leverage the <code>gcp-build</code> for that:</p> | ||
| <pre><code class="hljs language-json"><span class="hljs-punctuation">{</span> | ||
| <span class="hljs-attr">"scripts"</span><span class="hljs-punctuation">:</span> <span class="hljs-punctuation">{</span> | ||
| <span class="hljs-attr">"gcp-build"</span><span class="hljs-punctuation">:</span> <span class="hljs-string">"npx puppeteer browsers install chrome"</span> | ||
| <span class="hljs-punctuation">}</span> | ||
| <span class="hljs-punctuation">}</span> | ||
| </code></pre> | ||
| <div class="note"> | ||
| <p><strong>Note:</strong> you could call Puppeteer’s <code>postinstall</code> script directly by | ||
| doing <code>node node_modules/puppeteer/install.mjs</code> instead, but I found the | ||
| above command cleaner.</p> | ||
| </div> | ||
| <p>The good thing is that this script knows to not re-download Chrome if | ||
| it’s already found in the cache directory, so when the <code>postinstall</code> | ||
| script <em>does</em> run, the extra <code>gcp-build</code> command will be a no-op.</p> | ||
| <section class="post-footer"> | ||
| <h3>Want to leave a comment?</h3> | ||
| <p> | ||
| Start a conversation on <a href="https://twitter.com/valeriangalliat">Twitter</a> or send me an <a href="mailto:val@codejam.info">email</a>! 💌<br> | ||
| This post helped you? <a href="https://ko-fi.com/funkyval">Buy me a coffee</a>! 🍻 | ||
| </p> | ||
| </section> | ||
| </div> | ||
| <footer> | ||
| <div class="footer-inner"> | ||
| <figure> | ||
| <a href="/val.html"> | ||
| <img alt="The author (Val) in front of a mountain" src="/img/profile.jpg"> | ||
| </a> | ||
| </figure> | ||
| <div class="footer-content"> | ||
| <p> | ||
| Made with 🧡 by <a href="/val.html">Val</a>. | ||
| Generated from a <abbr title="Pull requests welcome!"><a href="https://github.com/valeriangalliat/blog/tree/master/2024/05/google-chrome-cloud-functions.md">public repository</a></abbr>. | ||
| </p> | ||
| <ul class="social"> | ||
| <li> | ||
| <a href="https://twitter.com/valeriangalliat"> | ||
| <svg width="16" height="16" viewBox="0 0 16 16"> | ||
| <title>Twitter</title> | ||
| <use href="/img/icons/407-twitter.svg#icon"></use> | ||
| </svg> | ||
| </a> | ||
| </li> | ||
| <li> | ||
| <a href="https://github.com/valeriangalliat"> | ||
| <svg width="16" height="16" viewBox="0 0 16 16"> | ||
| <title>GitHub</title> | ||
| <use href="/img/icons/433-github.svg#icon"></use> | ||
| </svg> | ||
| </a> | ||
| </li> | ||
| <li> | ||
| <a href="https://www.instagram.com/funkyval_/"> | ||
| <svg width="16" height="16" viewBox="0 0 16 16"> | ||
| <title>Instagram</title> | ||
| <use href="/img/icons/403-instagram.svg#icon"></use> | ||
| </svg> | ||
| </a> | ||
| </li> | ||
| <li> | ||
| <a href="https://www.youtube.com/@FunkyVal"> | ||
| <svg width="16" height="16" viewBox="0 0 16 16"> | ||
| <title>YouTube</title> | ||
| <use href="/img/icons/414-youtube.svg#icon"></use> | ||
| </svg> | ||
| </a> | ||
| </li> | ||
| <li> | ||
| <a href="https://ko-fi.com/funkyval"> | ||
| <svg width="16" height="16" viewBox="0 0 16 16"> | ||
| <title>Buy me a coffee!</title> | ||
| <use href="/img/icons/219-heart.svg#icon"></use> | ||
| </svg> | ||
| </a> | ||
| </li> | ||
| <li> | ||
| <a href="/feed.xml"> | ||
| <svg width="16" height="16" viewBox="0 0 16 16"> | ||
| <title>RSS</title> | ||
| <use href="/img/icons/412-rss.svg#icon"></use> | ||
| </svg> | ||
| </a> | ||
| </li> | ||
| </ul> | ||
| </div> | ||
| </div> | ||
| </footer> | ||
| <script src="/js/emojicon.js"></script> | ||
| <script src="/js/main-20230317.js"></script> | ||
| </body> | ||
| </html> |
Oops, something went wrong.