Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
fix: using thread pool on macOS (#1861)
The python parser uses ProcessPoolExecutor, which is problematic on macOS when it is distributed as a zip file, leading to errors like: ``` Traceback (most recent call last): File "<string>", line 1, in <module> File "/var/folders/fw/vythc6112ygfsvky8mdb5p580000gn/T/Bazel.runfiles_esxfeg_v/runfiles/python3_aarch64-apple-darwin/lib/python3.9/multiprocessing/resource_tracker.py", line 24, in <module> from . import spawn File "/var/folders/fw/vythc6112ygfsvky8mdb5p580000gn/T/Bazel.runfiles_esxfeg_v/runfiles/python3_aarch64-apple-darwin/lib/python3.9/multiprocessing/spawn.py", line 13, in <module> import runpy File "/var/folders/fw/vythc6112ygfsvky8mdb5p580000gn/T/Bazel.runfiles_esxfeg_v/runfiles/python3_aarch64-apple-darwin/lib/python3.9/runpy.py", line 19, in <module> from pkgutil import read_code, get_importer ModuleNotFoundError: No module named 'pkgutil' ``` According to ["Contexts and start methods" section](https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods) of the documentation: > On macOS, the spawn start method is now the default. The fork start method should be considered unsafe as it can lead to crashes of the subprocess as macOS system libraries may start threads. meanwhile: > The 'spawn' and 'forkserver' start methods generally cannot be used with “frozen” executables (i.e., binaries produced by packages like PyInstaller and cx_Freeze) on POSIX systems. This means there is no way to start a ProcessPoolExecutor when the Python zip file is running on macOS. This PR switches it to ThreadPoolExecutor instead.
- Loading branch information