Python 3.10.0 crashes on MacOS x86_64 #522
Comments
|
I'll add to this (because I think they are related) the |
|
So something is definitely broken about ctypes in the conda-forge build. Shapely builds and runs tests fine on my system with Homebrew Python 3.10; more importantly, if I copy |
|
I took the Python recipe and did a local build where I removed Is there any particular reason to avoid the macOS copy of libffi? This Python issue would seem to indicate that using the system copy for macos is the norm. |
|
Pinging @isuruf and @erykoff here as they have had to experience a lot of issues with One important difference to note here is that these issues only seem to happen on |
|
Actually this raises a question that is unclear (at least to me). Is this occurring with macOS ARM or x86? Edit: If it is ARM related, this comment ( libffi/libffi#571 (comment) ) and related issue may be of interest |
|
I will investigate. What is notable is that the issue above to get python to work with both system and forge libffi was first put in 3.10 and then backported. But 3.9 isn’t exhibiting this problem. |
|
Interesting, I was unable to replicate the segfault with the shapely py3.10 build on osx-arm64. So it may be limited to x86 (or possibly macos<11). I will try to replicate locally on x86/macos 10.14. |
|
For completeness, my tests locally were done on macOS 11.6.1. |
|
My reproducer is on macOS 10.13.6 🐄 and I have not seen this issue reproduced on arm64 |
jakirkham
changed the title
|
Retitled the issue to indicate this is an x86_64 issue |
|
I have been digging, and haven't figured out what's going wrong. Within shapely itself it crashes when the c++ throws an error, but only then (that is, successful ctypes calls work just fine). (I don't know if that's a clue or a coincidence). However, I have also been able to replicate the crash running on my M1 macbook when trying it in x86 rosetta translation. So that does make debugging marginally easier. |
|
Do you have the |
I added one to shapely/shapely#1227, copying here: |
|
Is this known to reproduce on macOS 10.14 / x86_64? I tried to reproduce the related libarchive issue locally on 10.14, but did not. |
|
Thanks @akrherz! 😄 Would you be able to share some more details from the environment where this occurs as well? Would suggest running the command |
|
Here is a script I put together that will segmentation fault (and show a backtrace with lldb). I get the same results on an old x86 laptop running 10.14, and an M1 running 11.6 using x86 mambaforge with rosetta. The native arm64 seems to work fine. The backtrace in this case looks like: |
|
Nicely done! 👏 So should |
Indeed, that should normally error. On linux (Python 3.10, latest conda-forge shapely 1.8) this gives me: This is coming from an exception in C++, which the C API layer of GEOS turns into a error return value (a null pointer in this case), which then Shapely should normally catch and raise a Python error. The code specifically for this failing example is https://github.com/Toblerity/Shapely/blob/d1cda1f97227f864710cd49d3f3577fa506ff65f/shapely/geos.py#L291-L299 |
|
Another crash came from the Also here it is a case of catching an error return / translating an error handle message into a python log. In case it is relevant, the C++ code that calls this |
|
Interesting yeah was about to ask about this. As it seems like the problem would happen on line 297 in the original example. All the code that comes afterwards is pure Python. So this wouldn't explain the C-level error handling whereas line 297 would. Are we able to reproduce just with a call to |
|
Am now wondering if it has something to do with the variadic arguments. |
|
I can't test myself (on linux the snippet works fine), but a potential reproducer just with |
|
Does |
Yes, I've just rebuilt locally with 3.3 and tested it out. All things pass now for the tests from conda-forge/llvmlite-feedstock#58 . |
|
This Python bug demonstrates crashes with Python’s test suite on macOS x86_64 using libffi 3.4.2, which don’t occur with libffi 3.3 |
|
@jorisvandenbossche 's example was crashing for me. Rebuilding Python with libffi allowed me to build a shapely package (instead of crashing), and the snippet no longer crashes for me either. |
|
Great find on the python bug! So this isn’t limited to the conda forge builds. |
|
I think this is probably the same failure that I've encountered on the clangdev feedstock just after 3.10 came out (and then got selected in the Things worked when restricting to |
|
Okay, I'm knee deep in autoconf pain, but what seems to be happening is that something in Hopefully I'll have more soon. |
|
I have found the problem! For whatever reason (and I am not sufficiently knowledgeable in autoconf to debug a 20000 line configure script), the default sussing of The conda forge solution is to add |
|
Thanks for the sleuthing @erykoff! 😄 This is an interesting discovery. We do set A few lines above that it is suggested we do this in compiler activation. May be worth following up on as it seems not specifying this path is causing us pain in a few places (this being one of them). It's worth noting there is some trickiness here |
|
I've tried to write this up in the Python bug ( https://bugs.python.org/issue44556 ). Feel free to add any additional details and/or correct me as needed |
|
Thanks everyone! |
Issue: Two conda-forge repos having strange crashes within CI on MacOS with python 3.10.0
Note: I am not a mac user, but am just attempting to help conda-forge repos get green CI builds :)
Examples of CI crashes: conda-forge/magics-feedstock#106 and conda-forge/shapely-feedstock#92
I do have an old mac that I am able to reproduce the shapely crashes and have gotten some iterative feedback here along with
lldboutput: shapely/shapely#1227Does anybody here have interest / mac experience to help figure out what is wrong? Thanks!
The text was updated successfully, but these errors were encountered: