GH-98831: Add macro and op and their implementation to DSL

[gvanrossum]

One deviation from the DSL spec: macro can build on both inst and op.

Also some other random improvements:

• Convert WITH_EXCEPT_START
• Fix lexer to balk at unrecognized characters, e.g. '@'
• Fix moved output names; support object pointers in cache
• Introduce error() method to print errors

• Issue: Generate the interpreter #98831

[gvanrossum]

This warning is annoying, the variable is used in an assert() call.

[brandtbucher]

Maybe stick a (void)lasti; // <helpful comment> near the assert?

[gvanrossum]

Oooh, I didn't realize you could do that. Will do.

[gvanrossum]

FWIW I requested a buildbot run for GH-99601, which is 2 commits ahead of this one. If that passes I will assume it would pass here too, without actually trying it (leave some resources for others).

[brandtbucher]

Thanks for your patience! I think this is a good step forward.

[brandtbucher]

Maybe stick a (void)lasti; // <helpful comment> near the assert?

[brandtbucher]

Eh, should we just add a read_u16 function to keep things simple? They're not really special anymore now that we've ditched the structs, and we've shown that it would compile the same.

[gvanrossum]

Oh, good idea. Will do.

[brandtbucher]

Just a note on the generated code... this looks sort of odd to me. _tmp_1 is the second item on the stack, and is used by the second instruction part. _tmp_2 is the first item on the stack and is used by the first instruction part.

Not a big deal, but it seems like they should be named in the order they are actually used, like the other superinstructions? My brain took a second to parse and re-parse the generated code a couple of times.

[gvanrossum]

I agree this is annoying, but it's hard to get the numbering consistent since there may be variables that aren't popped off the stack but are pushed back onto it. I could number them in reverse order, but there could still be cases where the numbering would be off.

E.g. for stack effect (a, b -- c) we'd see something like

PyObject *_tmp_3 = PEEK(2);
PyObject *_tmp_2 = PEEK(1);
PyObject *_tmp_1;
{
    PyObject *a = _tmp_3;
    PyObject *b = _tmp_2;
    PyObject *c;
    <c = a + b>
    _tmp_1 = c;
}
STACK_GROW(1);
PEEK(1, _tmp_1);

At some point I considered using letters instead of number the temp variables, but that wouldn't help much.

Once we have a register VM it'll be less of an issue. :-)

[brandtbucher]

The "kind" tag used here and elsewhere seems a bit odd to me. Is it just more ergonomic than using subclassing?

[gvanrossum]

It's historic. I'm slowly getting rid of these.

[gvanrossum]

I'll just land once tests pass. The read_uint16() function was needed only once. :-)