gh-87092: bring compiler code closer to a preprocessing-opt-assembler organisation

[iritkatriel]

Brings the code closer to the organisation discussed on the issue (with the exception of line number calculations which is not yet safe to move).

Also a few minor changes, like renaming a couple of functions for clarity.

• Issue: Split compiler into code-gen, optimizer and assembler. #87092

[markshannon]

Looks good in general, a few quibbles about naming and some comments.

I don't think "desugaring" is the correct term here. Desugaring implies that there is an equivalent source form, but there is not.
I can't think of a one word term for this set of operations, but I don't think we need one.

[markshannon]

Thanks

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @iritkatriel for commit e44b087 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @iritkatriel for commit 4f15173 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot ARM64 Windows Non-Debug 3.x has failed when building commit c529b45.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/730/builds/5950) and take a look at the build logs.
4. Check if the failure is related to this commit (c529b45) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/730/builds/5950

Failed tests:

• test_peg_generator

Summary of the results of the build (if available):

== Tests result: FAILURE then SUCCESS ==

403 tests OK.

10 slowest tests:

• test_multiprocessing_spawn: 1 min 53 sec
• test_asyncio: 1 min 39 sec
• test_concurrent_futures: 1 min 15 sec
• test_distutils: 1 min 15 sec
• test_socket: 1 min 6 sec
• test_urllib2_localnet: 44.7 sec
• test_mmap: 41.2 sec
• test_imaplib: 34.5 sec
• test_venv: 32.2 sec
• test_xmlrpc: 31.7 sec

34 tests skipped:
test_curses test_dbm_gnu test_dbm_ndbm test_devpoll test_epoll
test_fcntl test_fork1 test_gdb test_grp test_ioctl test_kqueue
test_multiprocessing_fork test_multiprocessing_forkserver test_nis
test_openpty test_ossaudiodev test_perf_profiler test_pipes
test_poll test_posix test_pty test_pwd test_readline test_resource
test_spwd test_syslog test_threadsignals test_tix test_tkinter
test_ttk test_wait3 test_wait4 test_xxtestfuzz test_zipfile64

1 re-run test:
test_peg_generator

Total duration: 21 min 23 sec

Click to see traceback logs

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64.nondebug\build\Lib\test\__main__.py", line 2, in <module>
    main()
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64.nondebug\build\Lib\test\libregrtest\main.py", line 763, in main
    Regrtest().main(tests=tests, **kwargs)
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64.nondebug\build\Lib\test\libregrtest\main.py", line 695, in main
    with os_helper.temp_cwd(test_cwd, quiet=True):
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64.nondebug\build\Lib\contextlib.py", line 155, in __exit__
    self.gen.throw(value)
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64.nondebug\build\Lib\test\support\os_helper.py", line 534, in temp_cwd
    with temp_dir(path=name, quiet=quiet) as temp_path:
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64.nondebug\build\Lib\contextlib.py", line 155, in __exit__
    self.gen.throw(value)
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64.nondebug\build\Lib\test\support\os_helper.py", line 488, in temp_dir
    rmtree(path)
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64.nondebug\build\Lib\test\support\os_helper.py", line 445, in rmtree
    _rmtree(path)
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64.nondebug\build\Lib\test\support\os_helper.py", line 388, in _rmtree
    _waitfor(_rmtree_inner, path, waitall=True)
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64.nondebug\build\Lib\test\support\os_helper.py", line 333, in _waitfor
    func(pathname)
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64.nondebug\build\Lib\test\support\os_helper.py", line 384, in _rmtree_inner
    _waitfor(_rmtree_inner, fullname, waitall=True)
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64.nondebug\build\Lib\test\support\os_helper.py", line 333, in _waitfor
    func(pathname)
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64.nondebug\build\Lib\test\support\os_helper.py", line 385, in _rmtree_inner
    _force_run(fullname, os.rmdir, fullname)
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64.nondebug\build\Lib\test\support\__init__.py", line 214, in _force_run
    return func(*args)
           ^^^^^^^^^^^
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Workspace\\buildarea\\3.x.linaro-win-arm64.nondebug\\build\\build\\test_python_10364�\\test_python_worker_3148�\\tmptian4i4t'


Traceback (most recent call last):
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64.nondebug\build\Lib\test\support\__init__.py", line 203, in _force_run
    return func(*args)
           ^^^^^^^^^^^
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Workspace\\buildarea\\3.x.linaro-win-arm64.nondebug\\build\\build\\test_python_10364�\\test_python_worker_3148�\\tmptian4i4t'


Traceback (most recent call last):
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64.nondebug\build\Lib\test\support\os_helper.py", line 483, in temp_dir
    yield path
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64.nondebug\build\Lib\test\support\os_helper.py", line 536, in temp_cwd
    yield cwd_dir
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64.nondebug\build\Lib\test\libregrtest\main.py", line 701, in main
    self._main(tests, kwargs)
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64.nondebug\build\Lib\test\libregrtest\main.py", line 758, in _main
    sys.exit(0)
SystemExit: 0