-
-
Notifications
You must be signed in to change notification settings - Fork 30.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bpo-46329: Split calls into precall and call instructions. #30855
bpo-46329: Split calls into precall and call instructions. #30855
Conversation
…KEYWORDS protocol.
Seems that this PR has caused some segfaults in the buildbots: https://buildbot.python.org/all/#/builders/441/builds/1345/steps/5/logs/stdio Is still not clear that is exactly this one, but this is a possible candidate. |
YEah, I can reproduce the segfault locally and doesn't happen if I revert this PR:
This is the stack trace:
|
I can't reproduce the failure on Ubuntu linux with gcc. Which configure options and platform? |
I am using clang, configure with |
Ugh, seems that this issue is piling up with more issues, we have almost all buildbots failing :(+- |
I am blocking the main branch until all these issues are resolved. Please ping me if you have a fix for this |
I can reproduce with clang, but it is mystifying. |
It's a C stack overflow. >>> import _testcapi
>>> def f():
... yield _testcapi.stack_pointer()
...
>>> _testcapi.stack_pointer() - next(f()) Spot the odd one out:
I'd like to raise an exception, rather than segfaulting, but the SC rejected PEP 651. |
I'm not taking in the name of the SC here (and this is not the place to discuss this) but I'm sure we can agree this is an extreme and unhelpful oversimplification of the SC resolution. Unfortunately seems that this commit creates the stack overflow while it didn't happen before. Even if we raise an exception, that was not happening before so technically will be also a regression. Unless we have a fix for this, we probably need to revert the commit. Do you know why clang stack overflows but GCC doesn't? |
As explained above, Clang uses ridiculous amounts of stack space using
|
Maybe relegate the clang with debug buildbot(s) to unstable? |
Absolutely not. They were passing before, and this breaks the buildbots. The solution is not deactivating previous passing checks. Notice that any system running the tests under clang will fail (which by default is MacOS systems) so this is clearly not acceptable.
Yeah, I get that, but why? |
Why? In release mode, it's fine as the stack consumption is ~600 bytes. We could reduce the default recursion depth to ~800 in debug mode. That seems the best solution, short term. Longer term there are two things I want to do:
|
That doesn't answer the question I think. The question is why GCC is not going over the limit and clang does. What is this extra stack space? What is this being used for?
That is subjective. I kind of agree with you, but I don't think this is something it can be decided here in this PR. Technically this makes functions that didn't raise/segfault now change their behaviour in debug mode and that's a backwards incompatible change so at least needs to be discussed in python-dev |
You'll need to ask the Clang/llvm developers. |
Why wouldn't just increasing the stack size for Clang debug builds be a solution? It was deemed an OK resolution when a similar overflow was occurring on Windows in bpo-17206. Also, the cause of increased stack usage for Clang may be for related reasons as in afore mentioned issue. |
👍 I'm ok with that as long as it doesn't need to be manually configured by the end user (so is automatically done by the configure step). |
https://bugs.python.org/issue17206 was closed in 2013. Are you sure about the issue number? https://bugs.python.org/issue17206 |
Very. I don't see how being an old, closed issue makes a difference as to the resolution. Mind you, the stack overflow wasn't mentioned until a while into the discussion. msg190623 is where that starts, coincidentally by yourself. |
Rather than increasing the allowed stack, how about setting Clang to |
I proposed to enable optimization on Windows debug build when debugging a test_exceptions crash: https://bugs.python.org/issue44348 => this idea was rejected, it's a bad idea: https://bugs.python.org/issue45115 The lib2to3 crash is tracked at https://bugs.python.org/issue46542 I already wrote a fix for test_json: use For the lib2to3 crash, my problem is that so far I failed to reproduce it. Does someone know how to reproduce the test_lib2to3 crash? |
I proposed PR #31035 to prevent the crash without having to revert this change. |
Splits most calls (excluding
CALL_FUNCTION_EX
) into aPRECALL
and aCALL
.The two call opcodes
CALL_FUNCTION
andCALL_METHOD
are unified into a singleCALL
opcode.We already split method calls into
PRECALL_METHOD
andCALL_METHOD
, this PR adds aPRECALL_FUNCTION
andKW_NAMES
."normal" calls,
f(args)
are split intoPRECALL_FUNCTION; CALL
."method" calls,
o.m(args)
are split intoPRECALL_METHOD; CALL
Any calls with named arguments have
KW_NAMES
inserted in between thePRECALL
andCALL
.https://bugs.python.org/issue46329