diff --git a/2024/05/google-chrome-cloud-functions.html b/2024/05/google-chrome-cloud-functions.html new file mode 100644 index 0000000..cd43762 --- /dev/null +++ b/2024/05/google-chrome-cloud-functions.html @@ -0,0 +1,218 @@ + + + + + Using Google Chrome instead of Chromium in Google Cloud Functions + + + + + + + + + + + + + + + + +
+
+
+

Using Google Chrome instead of Chromium in Google Cloud Functions

+

May 5, 2024

+
+
+
+
+

When using Puppeteer, Playwright and similar, you need to have Chrome +installed. When youā€™re running on AWS Lambda or Google Cloud Functions, +it can get tricky.

+

Google Cloud Functions used to bundle Chromium in their base images, +but itā€™s been a few years itā€™s no longer the case. Thatā€™s where packages +like chrome-aws-lambda +come in handy, by bundling Chromium directly inside a npm package, and +exposing a function that extracts the Chromium binary and returns the +path:

+
const chromium = require('chrome-aws-lambda')
+
+const path = await chromium.executablePath
+
+
+

Note: unnecessary pedantic detail: the above code doesnā€™t look like a function, +but it is, in fact, +a getter function that returns a promise. šŸ˜„

+
+

However thatā€™s Chromium, and you may have reasons to want Google Chrome +instead (mainly, proprietary codecs).

+

A totally unrelated note about AWS Lambda

+

This article is about Google Cloud Functions, but if youā€™re on AWS +Lambda, the above option is your best bet. Because of the Lambda total +size limit of 250 MB (all layers combined), itā€™s really hard to get a +binary of Chrome that fits in there.

+

Thatā€™s why chrome-aws-lambda uses LambdaFS +under the hood, to aggressively compress the Chrome installation with +Brotli and make it fit in that limited space.

+

But again with that build, you wonā€™t have proprietary codecs. I tried to +trim down a Chrome Linux build and compress it with the same technique +but never managed to make it fit on AWS Lambda. Recent Chrome versions +are just too big.

+

Thereā€™s another option, which is to compile Chromium yourself with +proprietary codecs. I never found any prebuilt binaries of Chromium that +include proprietary codecs (maybe because of license issues +redistributing them šŸ™ƒ) so youā€™re on your own here.

+

Remotion successfully does that for +Remotion Lambda. +Hereā€™s their instructions +to compile Chromium with proprietary codecs for Lambda.

+

Fair warning: it gets hairy, fast.

+

Back to Google Cloud Functions

+

Google Cloud Functions is more generous as for bundle size, so we donā€™t +need to resort to those tricks, and we can include a complete, +uncompressed, Google Chrome installation.

+

Google publishes Chrome for Testing, +builds specifically made +for headless usage.

+

We can just download the latest build from there as part of the +gcp-build script in our package.json.

+
{
+  "scripts": {
+    "gcp-build": "curl -s -O 'https://storage.googleapis.com/chrome-for-testing-public/124.0.6367.91/linux64/chrome-linux64.zip' && unzip chrome-linux64.zip && rm chrome-linux64.zip"
+  }
+}
+
+
+

Note: the gcp-build script allows you to run a custom build step +in Google Cloud Build, which is what Cloud Functions (both 1st and 2nd +gen, as well as Cloud Run and App Engine) use to build your function +image.

+

It would work just fine with a postinstall script as well, but +gcp-build makes sure you run it only on Google Cloud Build, which is +probably desirable in this particular case.

+
+

You will then have the Chrome binary in chrome-linux64/chrome, that +you can pass to the tool of your choice.

+

With Puppeteer

+

Courtesy of this post, +with Puppeteer, you donā€™t need to download Chrome manually, since it +provides a nifty script to do just that.

+

Actually, Puppeteerā€™s postinstall script +automatically downloads the latest version of Chrome for Testing for +your platform.

+

The caveat is that this script by default installs it to +~/.cache/puppeteer, which in the case of Google Cloud Build, is not +gonna be preserved in the final image. So we need to instruct Puppeteer +to install Chrome in a directory that Cloud Build will keep.

+

This can be done with the following .puppeteerrc.js:

+
module.exports = {
+  cacheDirectory: `${__dirname}/.cache/puppeteer`
+}
+
+

But even then, thereā€™s another caveat. Puppeteerā€™s postinstall script +will only run after it gets installed. However, because of build +caching, you will get in a state where node_modules is restored, with +Puppeteer already installed (so postinstall will not run), but the +.cache/puppeteer directory will also not be restored.

+

To mitigate that, we need to make sure to install Chrome systematically. +Again we can leverage the gcp-build for that:

+
{
+  "scripts": {
+    "gcp-build": "npx puppeteer browsers install chrome"
+  }
+}
+
+
+

Note: you could call Puppeteerā€™s postinstall script directly by +doing node node_modules/puppeteer/install.mjs instead, but I found the +above command cleaner.

+
+

The good thing is that this script knows to not re-download Chrome if +itā€™s already found in the cache directory, so when the postinstall +script does run, the extra gcp-build command will be a no-op.

+
+

Want to leave a comment?

+

+ Start a conversation on Twitter or send me an email! šŸ’Œ
+ This post helped you? Buy me a coffee! šŸ» +

+
+
+ + + + + diff --git a/feed.xml b/feed.xml index 237dc5e..d2f7bcd 100644 --- a/feed.xml +++ b/feed.xml @@ -5,10 +5,128 @@ https://www.codejam.info/ - 2024-05-05T21:40:48.166Z + 2024-05-06T00:12:25.926Z Val + + Using Google Chrome instead of Chromium in Google Cloud Functions + + https://www.codejam.info/2024/05/google-chrome-cloud-functions.html + 2024-05-05T07:00:00.000Z + When using Puppeteer, Playwright and similar, you need to have Chrome +installed. When youā€™re running on AWS Lambda or Google Cloud Functions, +it can get tricky.

+

Google Cloud Functions used to bundle Chromium in their base images, +but itā€™s been a few years itā€™s no longer the case. Thatā€™s where packages +like chrome-aws-lambda +come in handy, by bundling Chromium directly inside a npm package, and +exposing a function that extracts the Chromium binary and returns the +path:

+
const chromium = require('chrome-aws-lambda')
+
+const path = await chromium.executablePath
+
+
+

Note: unnecessary pedantic detail: the above code doesnā€™t look like a function, +but it is, in fact, +a getter function that returns a promise. šŸ˜„

+
+

However thatā€™s Chromium, and you may have reasons to want Google Chrome +instead (mainly, proprietary codecs).

+

A totally unrelated note about AWS Lambda

+

This article is about Google Cloud Functions, but if youā€™re on AWS +Lambda, the above option is your best bet. Because of the Lambda total +size limit of 250 MB (all layers combined), itā€™s really hard to get a +binary of Chrome that fits in there.

+

Thatā€™s why chrome-aws-lambda uses LambdaFS +under the hood, to aggressively compress the Chrome installation with +Brotli and make it fit in that limited space.

+

But again with that build, you wonā€™t have proprietary codecs. I tried to +trim down a Chrome Linux build and compress it with the same technique +but never managed to make it fit on AWS Lambda. Recent Chrome versions +are just too big.

+

Thereā€™s another option, which is to compile Chromium yourself with +proprietary codecs. I never found any prebuilt binaries of Chromium that +include proprietary codecs (maybe because of license issues +redistributing them šŸ™ƒ) so youā€™re on your own here.

+

Remotion successfully does that for +Remotion Lambda. +Hereā€™s their instructions +to compile Chromium with proprietary codecs for Lambda.

+

Fair warning: it gets hairy, fast.

+

Back to Google Cloud Functions

+

Google Cloud Functions is more generous as for bundle size, so we donā€™t +need to resort to those tricks, and we can include a complete, +uncompressed, Google Chrome installation.

+

Google publishes Chrome for Testing, +builds specifically made +for headless usage.

+

We can just download the latest build from there as part of the +gcp-build script in our package.json.

+
{
+  "scripts": {
+    "gcp-build": "curl -s -O 'https://storage.googleapis.com/chrome-for-testing-public/124.0.6367.91/linux64/chrome-linux64.zip' && unzip chrome-linux64.zip && rm chrome-linux64.zip"
+  }
+}
+
+
+

Note: the gcp-build script allows you to run a custom build step +in Google Cloud Build, which is what Cloud Functions (both 1st and 2nd +gen, as well as Cloud Run and App Engine) use to build your function +image.

+

It would work just fine with a postinstall script as well, but +gcp-build makes sure you run it only on Google Cloud Build, which is +probably desirable in this particular case.

+
+

You will then have the Chrome binary in chrome-linux64/chrome, that +you can pass to the tool of your choice.

+

With Puppeteer

+

Courtesy of this post, +with Puppeteer, you donā€™t need to download Chrome manually, since it +provides a nifty script to do just that.

+

Actually, Puppeteerā€™s postinstall script +automatically downloads the latest version of Chrome for Testing for +your platform.

+

The caveat is that this script by default installs it to +~/.cache/puppeteer, which in the case of Google Cloud Build, is not +gonna be preserved in the final image. So we need to instruct Puppeteer +to install Chrome in a directory that Cloud Build will keep.

+

This can be done with the following .puppeteerrc.js:

+
module.exports = {
+  cacheDirectory: `${__dirname}/.cache/puppeteer`
+}
+
+

But even then, thereā€™s another caveat. Puppeteerā€™s postinstall script +will only run after it gets installed. However, because of build +caching, you will get in a state where node_modules is restored, with +Puppeteer already installed (so postinstall will not run), but the +.cache/puppeteer directory will also not be restored.

+

To mitigate that, we need to make sure to install Chrome systematically. +Again we can leverage the gcp-build for that:

+
{
+  "scripts": {
+    "gcp-build": "npx puppeteer browsers install chrome"
+  }
+}
+
+
+

Note: you could call Puppeteerā€™s postinstall script directly by +doing node node_modules/puppeteer/install.mjs instead, but I found the +above command cleaner.

+
+

The good thing is that this script knows to not re-download Chrome if +itā€™s already found in the cache directory, so when the postinstall +script does run, the extra gcp-build command will be a no-op.

+
+

Want to leave a comment?

+

+ Start a conversation on Twitter or send me an email! šŸ’Œ
+ This post helped you? Buy me a coffee! šŸ» +

+
+]]> + Knex: timeout acquiring a connection, the pool is probably full @@ -2457,189 +2575,6 @@ North America. Cheers!

This post helped you? Buy me a coffee! šŸ»

-]]> - - - macOS harvest cursor from any app šŸ˜ - - https://www.codejam.info/2023/07/macos-harvest-cursor-from-any-app.html - 2023-07-27T04:00:00.000Z - As a pet project I was building a screenshot app, -and I wanted its cursors to match the ones of macOS screenshot utility: -Crosshair -and Camera.

-

This was harder than expected. Iā€™ll tell you the whole story because I -find it fun and interesting, but feel free to jump straight to the solution.

-

Default system cursors in NSCursor

-

In a Mac app, the NSCursor class exposes a number of default cursors, -like the arrow Arrow, -I-beam I-beam, -pointing hand Pointing hand, -various resize cursors, and even a cute ā€œdisappearing itemā€ cursor Disappearing item -(that I kinda want to name ā€œpoofā€ for some reason).

-

There is also a crosshair cursor Crosshair, -however itā€™s not the same that the system screenshot utility uses. And -the camera cursor is nowhere to be found.

-

So our last resort is to set a custom cursor from an image, e.g. for a -cursor thatā€™s 32x32 pixels where we want the ā€œhot spotā€ to be in the -middle:

-
let image = NSImage(named: "cursor.png")
-let hotSpot = NSPoint(x: 16, y: 16)
-let cursor = NSCursor(image: image, hotSpot: hotSpot)
-
-

But what image do we use here?

-

macOS default cursors source location?

-

By doing a bit of digging in the /System directory, we find the -following path:

-
/System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/HIServices.framework/Versions/A/Resources/cursors
-
-

This seems to contain all the system cursors, one directory for each, -containing a cursor.pdf and info.plist!

-

Here, we effectively have screenshotselection that matches the -screen capture utilityā€™s crosshair, and screenshotwindow that matches -the camera cursor shown during window selection. Neat.

-

Parsing the info.plist, we find the hot spot coordinates:

-
$ plutil -p /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/HIServices.framework/Versions/A/Resources/cursors/screenshotselection/info.plist
-{
-  "hotx" => 15
-  "hotx-scaled" => 15
-  "hoty" => 15
-  "hoty-scaled" => 15
-}
-
-

We can now load those programmatically:

-
func loadCursor(_ name: String) -> NSCursor? {
-  let root =
-    "/System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/HIServices.framework/Versions/A/Resources/cursors"
-
-  guard let data = FileManager.default.contents(atPath: "\(root)/\(name)/info.plist")
-  else {
-    return nil
-  }
-
-  guard
-    let plist = try? PropertyListSerialization.propertyList(from: data, options: [], format: nil)
-      as? [String: Any]
-  else {
-    return nil
-  }
-
-  guard let pdfData = try? Data(contentsOf: URL(fileURLWithPath: "\(root)/\(name)/cursor.pdf"))
-  else {
-    return nil
-  }
-
-  guard let cursorImage = NSImage(data: pdfData) else {
-    return nil
-  }
-
-  let hotSpot = NSPoint(
-    x: plist["hotx"] as! Int? ?? Int(cursorImage.size.width) / 2,
-    y: plist["hoty"] as! Int? ?? Int(cursorImage.size.height) / 2
-  )
-
-  return NSCursor(image: cursorImage, hotSpot: hotSpot)
-}
-
-

Letā€™s use this function in a basic example to demonstrate it:

-
import Cocoa
-
-let app = NSApplication.shared
-
-if let cursor = loadCursor("screenshotselection") {
-  DispatchQueue.main.asyncAfter(deadline: .now() + 1) {
-    cursor.set()
-  }
-}
-
-app.setActivationPolicy(.regular)
-app.activate(ignoringOtherApps: true)
-app.run()
-
-
-

Note: here we call cursor.set() after a delay because it -doesnā€™t -always work when called right -away for reasons that are not familiar to me.

-

In a real app, you probably want to subclass NSView, override -resetCursorRects, and call addCursorRect in it.

-
-

This actually looks good for the camera! But for the crosshair, it -doesnā€™t seem to match the original one.

-

The original crosshair size appears to be 50x50 pixels, while this one -is 46x46. More importantly, the original one has some kind of light outline -that makes it visible on darker backgrounds, that is completely missing -from that cursor PDF we just found. You can see the difference easily:

- - - - - - - - - - - - - -
OriginalCustom
Original crosshair over grey backgroundCustom crosshair over grey background
Original crosshair over dark backgroundCustom crosshair over dark background
-

So the screen capture utility doesnā€™t seem to be using this cursor from -HIServices.framework.

-

I tried exploring the contents of the screen capture app in -/System/Library/CoreServices/screencaptureui.app, especially the -Contents/Resources/Assets.car file, exploring it using -Asset Catalog Tinkerer, -but it didnā€™t contain anything useful.

-

Harvesting the cursor programmatically

-

The next idea I tried was to see if I could somehow access the cursor -data from other apps from my Swift app.

-

It turns out NSCursor exposes a currentSystem -property, containing current system cursor (as opposed to -NSCursor.current that contains your own applicationā€™s current cursor).

-

This way we can easily access the image data of the currentSystem -cursor, as well as its hotSpot to be used later in our own custom -cursor.

-
import Cocoa
-
-let cursor = NSCursor.currentSystem!
-
-print(cursor.hotSpot)
-
-let image = cursor.image.cgImage(forProposedRect: nil, context: nil, hints: nil)!
-let bitmap = NSBitmapImageRep(cgImage: image)
-let data = bitmap.representation(using: .png, properties: [:])!
-try! data.write(to: URL(fileURLWithPath: "cursor.png"))
-
-

We can put this code in a file test.swift, and run it with sleep 5 && swift test.swift. -This gives us 5 seconds to do whatever is needed to show the cursor we -want to harvest, before our script actually runs and saves the current -system cursor to a PNG file.

-

In the case of the screen capture utility crosshair, Iā€™ve got this -(pictured over transparent, grey and dark background to show how well it -reacts to those):

- - - - - - -
Harvested crosshairHarvested crosshair over grey backgroundHarvested crosshair over dark background
-

Perfect. šŸ‘Œ

-

I didnā€™t want to get into adding support for showing the dynamic -coordinates as part of the cursor, so as far as Iā€™m concerned, I got rid -of those and used just the crosshair in my app.

-

Wrapping up

-

I hope you found this post useful! Now if you want to get the cursor -data from any app, in its original transparent quality, you can use the -simple script above to do so. Enjoy!

-
-

Want to leave a comment?

-

- Join the discussion on Twitter or send me an email! šŸ’Œ
- This post helped you? Buy me a coffee! šŸ» -

-
]]> diff --git a/index.html b/index.html index c0644b7..7554d6c 100644 --- a/index.html +++ b/index.html @@ -38,6 +38,7 @@

CodeJam by Val

Posts

All the posts

diff --git a/posts.html b/posts.html index dc33788..a448b57 100644 --- a/posts.html +++ b/posts.html @@ -37,6 +37,7 @@

Posts

2024