Arm64 Docker support #1770
Merged
Arm64 Docker support #1770
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The mythril/myth image now builds for linux/arm64 as well as
linux/amd64. To achieve this, we now use docker buildx to build the
images and handle create a multi-platform image manifest. The build
config is defined in a buildx bake file.
By default it'll build both platforms at once, but you can build just
one by overriding the platform on the command line:
$ docker buildx bake --set='*.platform=linux/arm64'
The solcx Python package doesn't support downloading solc for arm64, so
the image now includes the svm command-line tool, which does. (svm is
used by foundry to provide solc versions.) Integration with solcx is
not automatic, so currently the image's docker-entrypoint.sh handles
symlinking solc versions from svm into solcx's directory.
In addition to supporting arm64, the image is now quite a bit smaller.
~400M vs 1.3G before.
|
|
When installing wheels in the Docker image, we previously used an ugly hack to force pip to install the z3-solver wheel, despite it having invalid platform metadata. Instead of bodging pip install in this way, we now fix the z3-solver wheel's metadata after building it, using `auditwheel addtag` to infer and apply compatible platform metadata, which allows pip to install the wheel normally.
|
No problem, thanks for merging! If you have trouble with building the arm64 version, let me know, as I was experimenting with cross-compiling to avoid needing to emulate arm64 from amd64. I've got a version that can build on GitHub Actions which only has amd64 runners. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This resolves #1517.
I was trying to use mythril in a vscode devcontainer environment on an ARM Mac. It turned out to be quite a challenge to install mythril because many of your dependencies don't provide arm64 binaries, and some don't publish recent source releases to build from (blake2b).
I also found that your Docker image is only built for amd64, so I tried to build it on arm64 locally, and found it couldn't build for arm64. So set about adjusting the image so that I could use it. Now that I've got a working arm64 image, I figured I should offer the changes back. Hopefully it might make life easier for other people on ARM systems!
I've configured the build with a Docker Buildx bake file, so you can build the mythril/myth image for amd64 and arm64 simultaneously like this:
$ docker buildx bakeWhen you're not doing a CI build, it probably makes more sense to just build the image for your platform, which you can do like this:
(Also, running with
--printis useful to understand what bake is going to do, it'll print out the configuration that would be built without actually building.)The image should be functionally the same as the previous version, but there are a few changes.
The base image is now python:3.10-slim rather than ubuntu:focal.
It's smaller now, ~400M arm64 / 500M amd64 vs 1.3G before. This is partly because the base image is a lot smaller, and partly because it's now a multi-stage build that builds wheels in separate stages and copies them into the final stage, so there aren't any dev dependencies/files in the image now.
It uses svm to download solc versions, both at image build time and in the image entrypoint. This is because the solcx Python package doesn't provide arm64 solc binaries. svm does, and it's also used by foundry.
This doesn't seem ideal, but hopefully it's a reasonable starting point.
The image runs as the mythril user rather than root by default. It can still be run as root using the
--userdocker option. This has the potential surface file permission problems for users bind-mounting directories into a myth container that aren't readable by uid 1000.I also updated
docker_build_and_deploy.shto build & publish usingbuildx bake, as shown above. It's possible that your circleci environment will need a tweak. I don't use circleci myself, but they seem to support Docker Buildx: https://support.circleci.com/hc/en-us/articles/360058095471-How-To-Use-Docker-Buildx-in-Remote-Docker-