[🚀 Feature]: Selenium Manager support mac_arm64 drivers

[titusfortner]

Feature and motivation

This is a subset of #11357. Regardless of what all we decide to support eventually, I think supporting M1/M2 is the most important piece.
Chrome, Edge & Firefox all have drivers available for it:

• geckodriver-v0.32.0-macos-aarch64.tar.gz
• chromedriver_mac_arm64.zip
• Mac M1

I'm guessing rust would put these in mac_arm64 directory instead of mac64?
Java can get the architecture from System.getProperty("os.arch"), Ruby is RbConfig::CONFIG['host_cpu'], .NET has RuntimeInformation.OSArchitecture. We'd need to get it all tested somehow.

The compiling/cross-compiling piece I don't think should be a limiting factor. Someone with an M1 can build the binary and we package it like the others.

Usage example

n/a

[bonigarcia]

Independently of the cross-compilation, Selenium Manager always has supported ARM architectures for driver download. See, for instance, the following debug traces (executed in a macOS M1):

bonigarcia@SL-2030 rust % cargo run -- --browser chrome --debug 
    Finished dev [unoptimized + debuginfo] target(s) in 0.09s
     Running `target/debug/selenium-manager --browser chrome --debug`
DEBUG	Using shell command to find out chrome version
DEBUG	Running sh command: "/Applications/Google\\ Chrome.app/Contents/MacOS/Google\\ Chrome --version"
DEBUG	Output { status: ExitStatus(unix_wait_status(0)), stdout: "Google Chrome 109.0.5414.119 \n", stderr: "" }
DEBUG	The version of chrome is 109.0.5414.119
DEBUG	Detected browser: chrome 109
DEBUG	Reading chromedriver version from https://chromedriver.storage.googleapis.com/LATEST_RELEASE_109
DEBUG	Required driver: chromedriver 109.0.5414.74
DEBUG	Running sh command: "chromedriver --version"
DEBUG	Output { status: ExitStatus(unix_wait_status(32512)), stdout: "", stderr: "sh: chromedriver: command not found\n" }
DEBUG	Driver URL: https://chromedriver.storage.googleapis.com/109.0.5414.74/chromedriver_mac_arm64.zip
DEBUG	File extracted to /Users/bonigarcia/.cache/selenium/chromedriver/mac-arm64/109.0.5414.74/chromedriver (16012720 bytes)
INFO	/Users/bonigarcia/.cache/selenium/chromedriver/mac-arm64/109.0.5414.74/chromedriver

bonigarcia@SL-2030 rust % cargo run -- --browser firefox --debug
    Finished dev [unoptimized + debuginfo] target(s) in 0.09s
     Running `target/debug/selenium-manager --browser firefox --debug`
DEBUG	Using shell command to find out firefox version
DEBUG	Running sh command: "/Applications/Firefox.app/Contents/MacOS/firefox -v"
DEBUG	Output { status: ExitStatus(unix_wait_status(0)), stdout: "Mozilla Firefox 108.0b7\n", stderr: "" }
DEBUG	The version of firefox is 108.07
DEBUG	Detected browser: firefox 108
DEBUG	Required driver: geckodriver 0.32.0
DEBUG	Running sh command: "geckodriver --version"
DEBUG	Output { status: ExitStatus(unix_wait_status(32512)), stdout: "", stderr: "sh: geckodriver: command not found\n" }
DEBUG	Driver URL: https://github.com/mozilla/geckodriver/releases/download/v0.32.0/geckodriver-v0.32.0-macos-aarch64.tar.gz
INFO	/Users/bonigarcia/.cache/selenium/geckodriver/mac-arm64/0.32.0/geckodriver

bonigarcia@SL-2030 rust % cargo run -- --browser edge --debug  
    Finished dev [unoptimized + debuginfo] target(s) in 0.09s
     Running `target/debug/selenium-manager --browser edge --debug`
DEBUG	Using shell command to find out edge version
DEBUG	Running sh command: "/Applications/Microsoft\\ Edge.app/Contents/MacOS/Microsoft\\ Edge --version"
DEBUG	Output { status: ExitStatus(unix_wait_status(0)), stdout: "Microsoft Edge 109.0.1518.70 \n", stderr: "" }
DEBUG	The version of edge is 109.0.1518.70
DEBUG	Detected browser: edge 109
DEBUG	Reading msedgedriver version from https://msedgedriver.azureedge.net/LATEST_RELEASE_109_MACOS
DEBUG	Required driver: msedgedriver 109.0.1518.70
DEBUG	Running sh command: "msedgedriver --version"
DEBUG	Output { status: ExitStatus(unix_wait_status(32512)), stdout: "", stderr: "sh: msedgedriver: command not found\n" }
DEBUG	Driver URL: https://msedgedriver.azureedge.net/109.0.1518.70/edgedriver_mac64_m1.zip
DEBUG	File extracted to /Users/bonigarcia/.cache/selenium/msedgedriver/mac-arm64/109.0.1518.70/msedgedriver (18758992 bytes)
INFO	/Users/bonigarcia/.cache/selenium/msedgedriver/mac-arm64/109.0.1518.70/msedgedriver

As you can see, the downloaded artifacts are chromedriver_mac_arm64.zip (Chrome), geckodriver-v0.32.0-macos-aarch64.tar.gz(Firefox), and edgedriver_mac64_m1.zip (Edge).

[titusfortner]

So we just need to compile selenium manager locally and add it to repo, and then update bindings to check for architecture and use it.

[titusfortner]

ooc, how many of these does Selenium Manager support right now? #11357

[bonigarcia]

Since we build the Selenium Manager using the GitHub Actions virtual environments (see workflow), we support the following:

• win64 (Rust toolchain: x86_64-pc-windows-msvc)
• linux64 (Rust toolchain: x86_64-unknown-linux-gnu)
• mac64 (Rust toolchain: x86_64-apple-darwin)

[titusfortner]

I mean, how many driver downloads are supported for the architectures? Like, if I compiled on m1 and requested Firefox, it would download geckodriver-v0.32.0-macos-aarch64.tar.gzhttps://github.com/mozilla/geckodriver/releases/download/v0.32.0/geckodriver-v0.32.0-macos-aarch64.tar.gz. But If I compiled Selenium Manager on windows arm64, would it download geckodriver-v0.32.0-win-aarch64.zip?

[diemol]

If I understand correctly, seems there are two things:

• Selenium Manager knows what architecture is running on, so it is able to download the correct driver for that architecture. As @titusfortner says, if Selenium Manager is running on an ARM, then for Firefox it downloads geckodriver-v0.32.0-macos-aarch64.tar.gzh. If it was running on Windows, it would download geckodriver-v0.32.0-win-aarch64.zip.
• We ship Selenium Manager on x64 (Win, Linux, macOS) only because that is what is available on GitHub Actions. Currently we do not support Selenium Manager on ARM because we do not have a central place to built it.

About the second item... Let's say we build it locally and we check it into the repository, now the issue is in the bindings, because the bindings need to know on which architecture they are running to select the right Selenium Manager binary. Also, not sure if a binary compiled for macOS M1 works on ARM Linux, so if we want to support ARM on Linux then we need to figure out where to build it. All in all, if we build the binary for ARM macOS, we need to pack it into the bindings as well. If we end up supporting ARM on Linux, we need to pack it into the bindings as well... Should we start thinking about downloading Selenium Manager on demand?

[bonigarcia]

@diemol You are correct in everything.

Besides all you said in the last paragraph, I would add that, for example, the current Selenium Manager macOS binary (x64) also works in an M1 (ARM) macOS. Therefore, I am not sure about the benefits of having two Selenium Manager macOS binaries (x64 and ARM). Maybe the ARM flavor is a bit faster in macOS M1/M2? Not sure if that benefit worths the added complexity to the release procedure, the extra weight in the Selenium distributions, and the required logic in the bindings to distinguish the local architecture.

Another example is win32. It is true (I tested it in a Windows x32 Virtual Machine) that the current Selenium Manager binary (x64) does not work on that platform. But how many people are using Windows 32 nowadays? I would say that number is tiny.

All in all, I would say that a good compromise is what we have now, i.e., Selenium Manager for win64, linux64, and mac64.

To handle rare cases, maybe we can implement a procedure to use a custom Selenium Manager binary (e.g., compiled for win32 or mac-arm64, to name a few). If fact, that is an idea we have already discussed in the past. For instance, in the Java bindings, the Selenium Manager binary is currently decompressed from the JAR, as used in a temporal file (which is later deleted using a singleton). But instead, if the bindings use Selenium Manager from a known path, such as ~/.cache/selenium/manager/1.0.0/selenium-manager, the need to decompress it each time is reduced. Moreover, this approach has two extra benefits.

First, users can override that binary with a custom compilation (e.g., by a mac-arm64 in macOS). Second, users can manually invoke Selenium Manager (from the shell, as a CLI tool) when Selenium Manager provides additional features (e.g., create project scaffolding).

What do you think?

[bonigarcia]

There is another alternative, which may be the most costly at the beginning but more optimal in the long run. In fact, it is something we have also discussed in the past.

It consists on implementing a mini-manager logic in each binding to download the latest Selenium Manager release to the local machine (to the .cache/selenium folder). To do this, we need the following:

• Make a release of Selenium Manager in GitHub (together with the rest of each release artifacts).
• Each binding will use its HTTP client to request the latest release of the Selenium Manager version (perhaps we can use the latest Selenium tag, and follow the same version numbering in Selenium Manager as in the rest of Selenium artifacts, i.e., 4.9.0, etc.).
• Download the proper Selenium Manager binary, i.e., for the local operating system (at this moment, and maybe for different platforms in the future), to the local cache.
• Use a metadata file (we can reuse the same used by Selenium Manager) to store a TTL value (in which the Selenium Manager binary is considered "fresh").
• Invoke the local Selenium Manager (as it is done already) in each binding.
• Each time the TTL expires, the process is repeated (to ensure that no new versions of Selenium Manager have been released).

The pros of this approach are:

• We avoid packaging the bindings in each Selenium release.
• We can easily add more Selenium Manager platforms (e.g., mac-arm64).
• We have a figure about the usage of Selenium Manager out of the box (looking at the GitHub releases)

The cons of this approach are:

• We need to implement (and maintain) the mini-manager in each binding.
• We need to be very careful in Selenium Manager to warranty backward compatibility.

We have previous experience with WebDriverManager and Selenium Manager to do this, so it is technically possible. If we decide to do this, I can do a reference implementation in the Java bindings (and do it similarly to the rest of the Selenium bindings).

[bonigarcia]

I am thinking more about this, and there is also another possibility. Each binding version (e.g., 4.9.0) only uses a fixed version of Selenium Manager (let's say the same number, 4.9.0). This way, we avoid the need to request the latest tag and the TTL value. Also, we avoid the problem of backward compatibility (since each binding released is supposed to be tested with a given Selenium Manager version).

On the other hand, this way, we lost the auto-upgrade capability (for instance, when bug fixes arrive in new versions of Selenium Manager). But well, in that case, users simply need to bump to the next Selenium version (e.g., selenium-java 4.10.0 or whatever).

[p0deje]

Selenium-Manager amd64 works perfectly fine on my MacBook M1 because Rosetta 2 handles it for me. Considering the binary is lightweight, I am not sure we need to provide the arm64 version of Selenium-Manager. However, I think there is a bug in the implementation of Selenium-Manager because it should download the drivers for the host architecture, while currently it always downloads the amd64 version:

$ lipo -archs common/manager/macos/selenium-manager
x86_64
$ common/manager/macos/selenium-manager --driver chromedriver
INFO	/Users/p0deje/.cache/selenium/chromedriver/mac64/109.0.5414.74/chromedriver
$ lipo -archs /Users/p0deje/.cache/selenium/chromedriver/mac64/109.0.5414.74/chromedriver
x86_64

[bonigarcia]

@p0deje Maybe the bindings are using an old version of Selenium Manager. But with the latest version, that should work. For instance, I have just executed the following in a macOS M1:

bonigarcia@SL-2030 rust % cargo run -- --browser chrome                                                                  
    Finished dev [unoptimized + debuginfo] target(s) in 0.10s
     Running `target/debug/selenium-manager --driver chromedriver`
INFO	/Users/bonigarcia/.cache/selenium/chromedriver/mac-arm64/109.0.5414.74/chromedriver

bonigarcia@SL-2030 rust % lipo -archs /Users/bonigarcia/.cache/selenium/chromedriver/mac-arm64/109.0.5414.74/chromedriver
arm64

You can get the current version of Selenium Manager (at trunk) from here (you need to download selenium-manager_macos-x64).

[p0deje]

@bonigarcia I've just downloaded the link you provided, but it behaves the same. Maybe it works for you because of cargo run? I imagine it compiles to arm64 so it downloads it too.

$ lipo -archs ./selenium-manager
x86_64
$ ./selenium-manager --driver chromedriver
INFO	/Users/p0deje/.cache/selenium/chromedriver/mac64/109.0.5414.74/chromedriver
$ lipo -archs /Users/p0deje/.cache/selenium/chromedriver/mac64/109.0.5414.74/chromedriver
x86_64

[titusfortner]

the current Selenium Manager macOS binary (x64) also works in an M1 (ARM) macOS

Ah, I didn't realize selenium manager worked with Rosetta 2. Maybe it does make sense short term to not have a separate selenium manager while we are checking them in.

That said, as I understand it, it does not download the native arm drivers, though, which I think is important.

[bonigarcia]

@p0deje Indeed. With the compiled version, for some reason, the architecture is not recognized, and it downloads the x86_64 flavor. That's a bug, thanks a lot for reporting. Maybe it has to do with the compilation. I am going to investigate it.

[titusfortner]

We can close this issue once that bug is fixed, thanks!

[bonigarcia]

I have just created PR #11611 to fix this issue.

@p0deje The Selenium Manager binary for macOS built using that PR is this one: selenium-manager_macos-x64

[p0deje]

@bonigarcia Wonderful, I can confirm it's working correctly now:

$ lipo -archs ./selenium-manager2
x86_64
$ ./selenium-manager2 --driver chromedriver
INFO	/Users/p0deje/.cache/selenium/chromedriver/mac-arm64/109.0.5414.74/chromedriver
$ lipo -archs /Users/p0deje/.cache/selenium/chromedriver/mac-arm64/109.0.5414.74/chromedriver
arm64

[titusfortner]

closing since the binary currently packaged works as desired

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.