gh-92031: Deoptimize Static Code at Finalization

[sweeneyde]

#92031

TODO:

• Find a more deterministic test case
• Decide on EXTENDED_ARG_QUICK/EXTENDED_ARG behavior

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @sweeneyde for commit df424cf 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

[sweeneyde]

One remark: this will convert EXTENDED_ARG_QUICK to EXTENDED_ARG for subsequent interpreters using this code, even though EXTENDED_ARG_QUICK is the one emitted by the compiler.

We should probably add a case for EXTENDED_ARG --> EXTENDED_ARG_QUICK in _PyCode_Quicken. Though we have a choice: do we

1. Emit (unquickened) EXTENDED_ARG in the compiler, to let deopt_code() convert it back to that state. This makes unquickened EXTENDED_ARG opcodes a hair slower, but adds consistency between interpreter invocations.
2. Continue emitting EXTENDED_ARG_QUICK. Then most programs will start as EXTENDED_ARG_QUICK, but there will be an inconsistency between before and after the first Py_FINALIZE()
3. Add a special case to deopt_code that keeps EXTENDED_ARG_QUICK as is.

The central issue is that _PyOpcode_Deopt[] has two different contradictory meanings in this case: one is "restore to the original" and one is "what to do during tracing".

[markshannon]

Feel free to add another table, they are only 256 bytes.
Maybe _PyOpcode_Original for restoring back to the original?

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @sweeneyde for commit ad9a38a 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

[sweeneyde]

@markshannon is this okay to merge now?

[markshannon]

Yes it is

[vstinner]

        for i in range(50):
            out, err = run("test_repeated_init_exec", code, timeout=60)

test_repeated_init_exec() already repeats the code 4 times. What is the point of repeating the test 50 x 4 = 200 times?

[sweeneyde]

        for i in range(50):

            out, err = run("test_repeated_init_exec", code, timeout=60)

test_repeated_init_exec() already repeats the code 4 times. What is the point of repeating the test 50 x 4 = 200 times?

The failure was still intermittent when I tried it, and just 4 runs only made the test fail about 10-15% of the time when I measured.

[vstinner]

The failure was still intermittent when I tried it, and just 4 runs only made the test fail about 10-15% of the time when I measured.

The Python test suite is run many times per day on each CI job and we have tons of CI jobs. IMO there is no need to loop so many tests inside the test.

On buildbots, it's very common that bugs which is barely possible to trigger manually when I really want to reproduce them... fail all the time on a specific worker because each worker has different timings and a different operating system ;-)