Segmentation fault after deleting __fields__ attribute from ast.AST

[federicovalenso]

Crash report

What happened?

>>> import ast
>>> dir(ast.AST)
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__match_args__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '_attributes', '_fields']
>>> del ast.AST._fields
>>> dir(ast.AST)
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__match_args__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '_attributes']
>>> t = ast.AST(arg1=123)
Segmentation fault

This happens because fields variable is null-pointer here, moreover it was previously checked for null here.

CPython versions tested on:

3.11

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.11.9 (main, Oct 28 2024, 23:26:29) [GCC 13.2.0]

Linked PRs

• gh-126105: Fix crash in ast module, when ._fields is deleted #126115
• [3.13] gh-126105: Fix crash in ast module, when ._fields is deleted (GH-126115) #126130
• [3.12] gh-126105: Fix crash in ast module, when ._fields is delet… #126132

[picnixz]

cc @Eclips4 @JelleZijlstra (Would adding Py_TPFLAGS_IMMUTABLETYPE work or is it too restrictive?)

[sobolevn]

I can confirm, that this still happens on main:

Python 3.14.0a1+ (heads/main:19e93e2e269, Oct 28 2024, 11:42:01) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ast
>>> del ast.AST._fields
>>> t = ast.AST(arg1=123)
[1]    85885 segmentation fault  ./python.exe
                          

Py_TPFLAGS_IMMUTABLETYPE

I think that this is an overkill and it change the semantics here. We can just modify the code and check for NULL explicitly.

See

cpython/Parser/asdl_c.py

Line 877 in 9b14083

ast_type_init(PyObject *self, PyObject *args, PyObject *kw)

[picnixz]

Oh yes, we only need to modify:

    if (PyObject_GetOptionalAttr((PyObject*)Py_TYPE(self), state->_fields, &fields) < 0) {
        goto cleanup;
    }

with a NULL check beforehand I think. What do you think?

[Eclips4]

cc @Eclips4 @JelleZijlstra (Would adding Py_TPFLAGS_IMMUTABLETYPE work or is it too restrictive?)

If you are talking about making nodes immutable, we can't do that because a lot of code relies on the fact that nodes are mutable. So it would be a huge breaking change.

[Eclips4]

Thank you Valery for the report and Nikita for the fix.