Dockerize "Direct Execution of Fuzz Targets" #1904
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Adds a Dockerfile to enable easily executing the fuzz targets directly inside a container environment instead of directly on a host machine. This addresses concerns raised in PR gitpython-developers#1901 related to how `fuzz_tree.py` writes to the real `/tmp` directory of the file system it is executed on as part of setting up its own test fixtures, but also makes for an easier to use development workflow. See this related comment on PR gitpython-developers#1901 for additional context: gitpython-developers#1901 (comment)
Byron
reviewed
Apr 21, 2024
There was a problem hiding this comment.
Thanks a lot, much appreciated!
I did take it out for a spin by following the documentation, but encountered an error when building the image (please see below).
Normally I'd ask if this is me, but since it's docker, I'd hope it cannot be. The docker-port is provided by orbstack in my case, and it's possible that that something about that VM is different enough to cause failures. Also, I am doing this on an ARM machine, and probably the VM isn't x86 either.
12.22 /tmp/pip-install-wpyvqb77/atheris_1178434add2a4d5f977b1997318ecda9/setup_utils/find_libfuzzer.sh: line 44: clang: command not found
12.22 Failed to find libFuzzer; set either $CLANG_BIN to point to your Clang binary, or $LIBFUZZER_LIB to point directly to your libFuzzer .a file. If needed, download and build the latest version of Clang:
12.22 git clone --depth=1 https://github.com/llvm/llvm-project.git
12.22 cd llvm-project
12.22 cmake -DLLVM_ENABLE_PROJECTS='clang;compiler-rt' -G "Unix Makefiles" -S llvm -B build
12.22 NPROC=$(sysctl -n hw.logicalcpu 2>/dev/null || nproc)
12.22 cmake --build build --parallel $NPROC # This step is very slow.
12.22 Then, set $CLANG_BIN="$(pwd)/bin/clang" and run pip again.
12.22 You should use this same Clang for building any Python extensions you plan to fuzz.
12.22
12.22 Traceback (most recent call last):
12.22 File "<string>", line 105, in get_libfuzzer_lib
12.22 File "/usr/local/lib/python3.8/subprocess.py", line 415, in check_output
12.22 return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
12.22 File "/usr/local/lib/python3.8/subprocess.py", line 516, in run
12.22 raise CalledProcessError(retcode, process.args,
12.22 subprocess.CalledProcessError: Command '['/tmp/pip-install-wpyvqb77/atheris_1178434add2a4d5f977b1997318ecda9/setup_utils/find_libfuzzer.sh']' returned non-zero exit status 1.
12.22
12.22 During handling of the above exception, another exception occurred:
12.22
12.22 Traceback (most recent call last):
12.22 File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
12.22 main()
12.22 File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
12.22 json_out['return_val'] = hook(**hook_input['kwargs'])
12.22 File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
12.22 return _build_backend().build_wheel(wheel_directory, config_settings,
12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 410, in build_wheel
12.22 return self._build_with_temp_dir(
12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 395, in _build_with_temp_dir
12.22 self.run_setup()
12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 487, in run_setup
12.22 super().run_setup(setup_script=setup_script)
12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 311, in run_setup
12.22 exec(code, locals())
12.22 File "<string>", line 444, in <module>
12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/__init__.py", line 104, in setup
12.22 return distutils.core.setup(**attrs)
12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 184, in setup
12.22 return run_commands(dist)
12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
12.22 dist.run_commands()
12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
12.22 self.run_command(cmd)
12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 967, in run_command
12.22 super().run_command(command)
12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
12.22 cmd_obj.run()
12.22 File "/tmp/pip-build-env-97wquo94/normal/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 368, in run
12.22 self.run_command("build")
12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
12.22 self.distribution.run_command(command)
12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 967, in run_command
12.22 super().run_command(command)
12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
12.22 cmd_obj.run()
12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 132, in run
12.22 self.run_command(cmd_name)
12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
12.22 self.distribution.run_command(command)
12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 967, in run_command
12.22 super().run_command(command)
12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
12.22 cmd_obj.run()
12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 91, in run
12.22 _build_ext.run(self)
12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
12.22 self.build_extensions()
12.22 File "<string>", line 333, in build_extensions
12.22 File "<string>", line 109, in get_libfuzzer_lib
12.22 RuntimeError: Failed to find libFuzzer; set either $CLANG_BIN to point to your Clang binary, or $LIBFUZZER_LIB to point directly to your libFuzzer .a file. If needed, download and build the latest version of Clang:
12.22 git clone --depth=1 https://github.com/llvm/llvm-project.git
12.22 cd llvm-project
12.22 cmake -DLLVM_ENABLE_PROJECTS='clang;compiler-rt' -G "Unix Makefiles" -S llvm -B build
12.22 NPROC=$(sysctl -n hw.logicalcpu 2>/dev/null || nproc)
12.22 cmake --build build --parallel $NPROC # This step is very slow.
12.22 Then, set $CLANG_BIN="$(pwd)/bin/clang" and run pip again.
12.22 You should use this same Clang for building any Python extensions you plan to fuzz.
12.22
12.22 [end of output]
12.22
12.22 note: This error originates from a subprocess, and is likely not a problem with pip.
12.23 ERROR: Failed building wheel for atheris
12.23 Failed to build atheris
12.23 ERROR: Could not build wheels for atheris, which is required to install pyproject.toml-based projects
------
Dockerfile:13
--------------------
12 | # Update package managers, install necessary packages, and cleanup unnecessary files in a single RUN to keep the image smaller.
13 | >>> RUN apt-get update && \
14 | >>> apt-get install --no-install-recommends -y git && \
15 | >>> python -m pip install --upgrade pip && \
16 | >>> python -m pip install atheris && \
17 | >>> python -m pip install -e . && \
18 | >>> apt-get clean && \
19 | >>> apt-get autoremove -y && \
20 | >>> rm -rf /var/lib/apt/lists/* /root/.cache
21 |
--------------------
ERROR: failed to solve: process "/bin/sh -c apt-get update && apt-get install --no-install-recommends -y git && python -m pip install --upgrade pip && python -m pip install atheris && python -m pip install -e . && apt-get clean && apt-get autoremove -y && rm -rf /var/lib/apt/lists/* /root/.cache" did not complete successfully: exit code: 1
|
Interesting, your image is missing Clang. I'm not familiar with Orbstack (looks neat though) but ARM shouldn't be an issue. I have an ARM machine I can test on and I can probably install Orbstack too if necessary, but before I do would you mind adding E.g, diff --git a/fuzzing/local-dev-helpers/Dockerfile b/fuzzing/local-dev-helpers/Dockerfile
index 77808ed1..e5b171c4 100644
--- a/fuzzing/local-dev-helpers/Dockerfile
+++ b/fuzzing/local-dev-helpers/Dockerfile
@@ -11,7 +11,7 @@ COPY . .
# Update package managers, install necessary packages, and cleanup unnecessary files in a single RUN to keep the image smaller.
RUN apt-get update && \
- apt-get install --no-install-recommends -y git && \
+ apt-get install --no-install-recommends -y git clang-15 && \
python -m pip install --upgrade pip && \
python -m pip install atheris && \
python -m pip install -e . && \ |
|
Thanks for your help! Now I am seeing this, and wonder how it's possible that this ever worked elsewhere - it's docker, right? Maybe it's easiest to install Orbstack as it might have something to do with it. |
Yep, it's Docker. Very strange. I think I might have an idea of what is going on but I'm not totally sure until I can get onto my other machine and test it. I'll let you know if I learn anything.
Yeah, I'll try on my Mac M2 machine with vanilla Docker Desktop as well as with Orbstack to see if I can resolve it. |
The Atheris package bundles a binary that supplies libFuzzer on some host machines, but in some cases (such as ARM based mac hosts) Atheris seems to require building libFuzzer at install time while pip builds the wheel. In the latter case, clang and related dependencies must be present and available for the build, which itself requires using a non "slim" version of the Python base image and not passing the `--no-install-recommends` flag to `apt-get install` as both prevent the required related libraries from being automatically installed. It is also worth noting that at the time of this commit, the default version of LLVM & Clang installed when `clang` is installed from `apt` is version 14, while the latest stable version is 17 and OSS-Fuzz uses 15. The decision to install the default version (14) available via the debian repos was intentional because a) it appears to work fine for our needs and Atheris version b) specifying a different version requires more complexity depending on install method, but the goal of this Dockerfile is simplicity and low maintenance. If it becomes neccissary to upgrade Clang/LLVM in the future, one option to consider besides installing from source is the apt repository maintained by the LLVM project: https://apt.llvm.org/ See the discussion in this issue for additional context to this change: gitpython-developers#1904
|
@Byron you should be able to build the image without issue now. The problem and solution are described in the commit message1 for f145121. The image takes a little longer to build now, and is slightly larger overall, but it should work for most folks. Footnotes
|
Byron
approved these changes
Apr 22, 2024
There was a problem hiding this comment.
That's great work right there, thanks so much!
I could validate that the image generation now works, and was able to complete a fuzzer run with 10.000 iterations as well as per the example provided in the README.
The commit message was an interesting read, and shows a lot about the complexities of today's software environment. There is a lot of logic included in installers, that's for sure, and having an image that always works seems like a good basis. The larger size and time to build seems like a price worth paying.
Byron
added
the
📣highlight-in-changelog📣
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds a Dockerfile to enable easily executing the fuzz targets directly inside a container environment instead of directly on a host machine. This addresses concerns raised in PR #1901 related to how
fuzz_tree.pywrites to the real/tmpdirectory of the file system it is executed on as part of setting up its own test fixtures, but also makes for an easier to use development workflow.I also added a warning to the
fuzzing/READEMEstrongly suggesting users only execute the targets inside of Docker.See this related comment on PR #1901 for additional context: #1901 (comment)