Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test_error_on_parser_stack_overflow from test_syntax triggers triggers-out-of-bounds memory protection for a debug build under WASI #111807

Closed
brettcannon opened this issue Nov 7, 2023 · 8 comments
Closed

test_error_on_parser_stack_overflow from test_syntax triggers triggers-out-of-bounds memory protection for a debug build under WASI #111807

brettcannon opened this issue Nov 7, 2023 · 8 comments
Labels
OS-wasi

Comments

@brettcannon
Copy link
Member

brettcannon commented Nov 7, 2023 •
edited by bedevere-app bot
Loading

Detected with WASI-SDK 20 and wasmtime 14.

Linked PRs
@brettcannon
Copy link
Member Author

Looks like an out-of-bounds memory access:

         1325: 0x36592 - <unknown>!factor_rule
         1326: 0x36027 - <unknown>!term_rule
         1327: 0x351fb - <unknown>!sum_rule
         1328: 0x34a30 - <unknown>!shift_expr_rule
         1329: 0x34265 - <unknown>!bitwise_and_rule
         1330: 0x32dc2 - <unknown>!bitwise_xor_rule
         1331: 0x2dc0e - <unknown>!bitwise_or_rule
         1332: 0x47aed - <unknown>!comparison_rule
         1333: 0x47867 - <unknown>!inversion_rule
         1334: 0x43af8 - <unknown>!conjunction_rule
         1335: 0x2f2c9 - <unknown>!disjunction_rule
         1336: 0x14fde - <unknown>!expression_rule
         1337: 0x4ac44 - <unknown>!star_expression_rule
         1338: 0x24e67 - <unknown>!star_expressions_rule
         1339: 0x1fcd4 - <unknown>!simple_stmt_rule
         1340: 0x1321b - <unknown>!simple_stmts_rule
         1341: 0x114c0 - <unknown>!statements_rule
         1342: 0xd694 - <unknown>!_PyPegen_parse
         1343: 0x76a8 - <unknown>!_PyPegen_run_parser
         1344: 0x7ba3 - <unknown>!_PyPegen_run_parser_from_string
         1345: 0x8f1b1 - <unknown>!_PyParser_ASTFromString
         1346: 0x318dec - <unknown>!Py_CompileStringObject
         1347: 0x242f85 - <unknown>!builtin_compile
         1348: 0x13e0bd - <unknown>!cfunction_vectorcall_FASTCALL_KEYWORDS
         1349: 0xc8db9 - <unknown>!_PyObject_VectorcallTstate
         1350: 0xc9f44 - <unknown>!PyObject_Vectorcall
         1351: 0x2695da - <unknown>!_PyEval_EvalFrameDefault
         1352: 0x26f5dd - <unknown>!_PyEval_Vector
         1353: 0xca244 - <unknown>!_PyFunction_Vectorcall
         1354: 0xcc3f4 - <unknown>!_PyObject_VectorcallTstate
         1355: 0xcc231 - <unknown>!method_vectorcall
         1356: 0xc9e3e - <unknown>!_PyVectorcall_Call
         1357: 0xc9ff7 - <unknown>!_PyObject_Call
         1358: 0xca159 - <unknown>!PyObject_Call
         1359: 0x2675a1 - <unknown>!_PyEval_EvalFrameDefault
         1360: 0x26f5dd - <unknown>!_PyEval_Vector
         1361: 0xca244 - <unknown>!_PyFunction_Vectorcall
         1362: 0xc9282 - <unknown>!_PyObject_VectorcallDictTstate
         1363: 0xca3fe - <unknown>!_PyObject_Call_Prepend
         1364: 0x18c442 - <unknown>!slot_tp_call
         1365: 0xc90af - <unknown>!_PyObject_MakeTpCall
         1366: 0xc8da3 - <unknown>!_PyObject_VectorcallTstate
         1367: 0xc9f44 - <unknown>!PyObject_Vectorcall
         1368: 0x2695da - <unknown>!_PyEval_EvalFrameDefault
         1369: 0x26f5dd - <unknown>!_PyEval_Vector
         1370: 0xca244 - <unknown>!_PyFunction_Vectorcall
         1371: 0xcc3f4 - <unknown>!_PyObject_VectorcallTstate
         1372: 0xcc231 - <unknown>!method_vectorcall
         1373: 0xc9e3e - <unknown>!_PyVectorcall_Call
         1374: 0xc9ff7 - <unknown>!_PyObject_Call
         1375: 0xca159 - <unknown>!PyObject_Call
         1376: 0x2675a1 - <unknown>!_PyEval_EvalFrameDefault
         1377: 0x26f5dd - <unknown>!_PyEval_Vector
         1378: 0xca244 - <unknown>!_PyFunction_Vectorcall
         1379: 0xc9282 - <unknown>!_PyObject_VectorcallDictTstate
         1380: 0xca3fe - <unknown>!_PyObject_Call_Prepend
         1381: 0x18c442 - <unknown>!slot_tp_call
         1382: 0xc90af - <unknown>!_PyObject_MakeTpCall
         1383: 0xc8da3 - <unknown>!_PyObject_VectorcallTstate
         1384: 0xc9f44 - <unknown>!PyObject_Vectorcall
         1385: 0x2695da - <unknown>!_PyEval_EvalFrameDefault
         1386: 0x26f5dd - <unknown>!_PyEval_Vector
         1387: 0xca244 - <unknown>!_PyFunction_Vectorcall
         1388: 0xcc3f4 - <unknown>!_PyObject_VectorcallTstate
         1389: 0xcc231 - <unknown>!method_vectorcall
         1390: 0xc9e3e - <unknown>!_PyVectorcall_Call
         1391: 0xc9ff7 - <unknown>!_PyObject_Call
         1392: 0xca159 - <unknown>!PyObject_Call
         1393: 0x2675a1 - <unknown>!_PyEval_EvalFrameDefault
         1394: 0x26f5dd - <unknown>!_PyEval_Vector
         1395: 0xca244 - <unknown>!_PyFunction_Vectorcall
         1396: 0xc9282 - <unknown>!_PyObject_VectorcallDictTstate
         1397: 0xca3fe - <unknown>!_PyObject_Call_Prepend
         1398: 0x18c442 - <unknown>!slot_tp_call
         1399: 0xc90af - <unknown>!_PyObject_MakeTpCall
         1400: 0xc8da3 - <unknown>!_PyObject_VectorcallTstate
         1401: 0xc9f44 - <unknown>!PyObject_Vectorcall
         1402: 0x2695da - <unknown>!_PyEval_EvalFrameDefault
         1403: 0x26f5dd - <unknown>!_PyEval_Vector
         1404: 0xca244 - <unknown>!_PyFunction_Vectorcall
         1405: 0xc9282 - <unknown>!_PyObject_VectorcallDictTstate
         1406: 0xca3fe - <unknown>!_PyObject_Call_Prepend
         1407: 0x18d4e5 - <unknown>!slot_tp_init
         1408: 0x18197a - <unknown>!type_call
         1409: 0xc90af - <unknown>!_PyObject_MakeTpCall
         1410: 0xc8da3 - <unknown>!_PyObject_VectorcallTstate
         1411: 0xc9f44 - <unknown>!PyObject_Vectorcall
         1412: 0x2695da - <unknown>!_PyEval_EvalFrameDefault
         1413: 0x248cc4 - <unknown>!PyEval_EvalCode
         1414: 0x243a85 - <unknown>!builtin_exec
         1415: 0x13e0bd - <unknown>!cfunction_vectorcall_FASTCALL_KEYWORDS
         1416: 0xc8db9 - <unknown>!_PyObject_VectorcallTstate
         1417: 0xc9f44 - <unknown>!PyObject_Vectorcall
         1418: 0x2695da - <unknown>!_PyEval_EvalFrameDefault
         1419: 0x26f5dd - <unknown>!_PyEval_Vector
         1420: 0xca244 - <unknown>!_PyFunction_Vectorcall
         1421: 0xc9e3e - <unknown>!_PyVectorcall_Call
         1422: 0xc9ff7 - <unknown>!_PyObject_Call
         1423: 0xca159 - <unknown>!PyObject_Call
         1424: 0x343f99 - <unknown>!pymain_run_module
         1425: 0x343461 - <unknown>!Py_RunMain
         1426: 0x3445e9 - <unknown>!pymain_main
         1427: 0x344692 - <unknown>!Py_BytesMain
         1428: 0x5987 - <unknown>!main
         1429: 0x52c681 - <unknown>!__main_void
         1430: 0x5960 - <unknown>!_start
       note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information
    2: memory fault at wasm address 0x10000002c in linear memory of size 0xb30000
    3: wasm trap: out of bounds memory access

@brettcannon brettcannon changed the title test_error_on_parser_stack_overflow from test_syntax triggers stack overflow protections for a debug build under WASI test_error_on_parser_stack_overflow from test_syntax triggers triggers-out-of-bounds memory protection for a debug build under WASI Nov 8, 2023
@brettcannon
Copy link
Member Author

For some reason this problem screams @pablogsal to me. 😉 I wonder if there's a missing check on whether memory allocation succeeded or failed?

@pablogsal
Copy link
Member

This looks like the builder is stack overflowing on this platform. We do have a protection based on depth, but seems like the stack in the platform is not big enough. We can lower it flor WASI if we are able to detect it at compile time 🤔

@pablogsal
Copy link
Member

If is not this, we need more debugging information here to understand how the out of memory access is happening, but my bet is certainly on a stack overflow.

@pablogsal
Copy link
Member

Ah, we already have it set to 4000 here:

https://github.com/python/cpython/blob/31c90d5838e8d6e4c47d98500a34810ccb33a6d4/Parser/parser.c#L11

maybe we need to lower it even more for debug builds?

@brettcannon
Copy link
Member Author

Ah, another stack depth setting!

I can play with the value to see where things trigger. Is there a "reasonable" minimum to decide whether it's worth differentiating between a debug build or not under WASI?

@pablogsal
Copy link
Member

The debug mode shouldn't affect parser depth or at least I don't see how that may be possible. It may affect stack size of every frame but the difference shouldn't be too different.

With this I mean that it should not matter a lot so maybe is worth checking empirically what depth causes it to crash in both modes.

@brettcannon
Copy link
Member Author

It looks like the value needs to be 1325 or lower. I'll do a PR to set it to 1000 under WASI when in a debug build.

@bedevere-app bedevere-app bot mentioned this issue Nov 18, 2023
Merged
brettcannon added a commit to brettcannon/cpython that referenced this issue Nov 18, 2023
@brettcannon
pythonGH-111807: Lower the parser stack depth under WASI debug builds
cd321ff
pablogsal pushed a commit that referenced this issue Nov 20, 2023
@brettcannon
GH-111807: Lower the parser stack depth under WASI debug builds (#112225
56e59a4 
)
aisk pushed a commit to aisk/cpython that referenced this issue Feb 11, 2024
@brettcannon @aisk
pythonGH-111807: Lower the parser stack depth under WASI debug builds (…
dafa2ca 
…python#112225)
Glyphack pushed a commit to Glyphack/cpython that referenced this issue Sep 2, 2024
@brettcannon @Glyphack
pythonGH-111807: Lower the parser stack depth under WASI debug builds (…
fa86022 
…python#112225)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
OS-wasi
Projects
No open projects
WASI to tier 2
Status: Done
Milestone
No milestone
Development

No branches or pull requests
2 participants
@brettcannon @pablogsal