Renewal: Rebooting the Compiler Toolchain

What the Renewal Branch is About

Off and on until August 2018, I've spent time working on a true 64-bit self-hosted compiler for Epoch. My intention was to slowly convert the 32-bit compiler into something that could cross-compile 64-bit binaries, and then use the 64-bit cross-compiler to bootstrap a 64-bit edition of the compiler.

If I had been disciplined about the language's development over the years this should have been an easy conversion - fix up the LLVM integration a bit, patch over the 32-bit assumptions in the linker and debug information generators, and maybe stomp a rogue bug or two.

Unfortunately, I was not so clean and proper. I wound up choosing to make several major changes to the language while also attempting to port to 64-bit. The result is a cross-compiler that sort of works and generates somewhat operational 64-bit binaries (and PDB files, yayy!). But there are a lot of miscompiles, a lot of bugs, and a lot of areas where the shaky cross-compiler just isn't going to be pleasant to work on anymore.

The cross-compiler is a modified version of a 32-bit self-hosting compiler, which was in turn bootstrapped using a C++ implementation of the Epoch language. There was a time when Epoch ran on a bytecode virtual machine instead of generating native machine code with LLVM. In fact, the cross-compiler still embeds a form of that bytecode in its own EXE, and turns that into useful machine code using a DLL called EpochRuntime.dll when the process first loads up. This causes a couple of seconds of delay every time the compiler is invoked - a nuisance I still look forward to eliminating in the 64-bit self-hosting version.

There is a tremendous amount of technical debt represented here. I can barely help anyone get the language running anymore because the build process is so messy (in some cases it is hard-coded to use specific files or directories on old computers of mine). Worse, there are compilation bugs and misfeatures (such as value-vs-reference semantics confusions) which have made the language chaotic and inefficient to work with.

So in August 2018 I spun up a new git branch called Renewal. The sole purpose of this branch is to write a new compiler in 32-bit Epoch to be the 64-bit cross-compiler. Eventually that cross-compiler will be used to bootstrap a self-hosting 64-bit compiler. Along the way, I expect several major benefits:

• File structure is actually legible and sane
• Code is much cleaner and often simpler
• Missing language features and bugs can be taken care of
• Existing linking and PDB code can be used for inspiration

Yes, it means writing yet another compiler. That's nothing I haven't done before, several times. I can use a lot of the past learning to inform the new implementation. More importantly, I can unstick the project from its long-running state of being impenetrable to anyone who isn't me.

How to Use Renewal

This is mostly off the cuff and may be missing some important steps; please file an Issue if I omitted anything. Also some of this is going to be really gross until I can make a better workflow.

1. Download the legacy 32-bit compiler pack
2. Extract the files someplace, say to C:\Epoch\. However, ensure that the subfolder Epoch32 remains in place. So for example you might end up with C:\Epoch\Epoch32\EpochCompiler.exe on disk.
3. Create a registry key at HKEY_LOCAL_MACHINE\Software\Epoch\CurrentInstall
4. Add a value called InstallPath and put the path where your Epoch32 folder lives. So e.g. you might put C:\Epoch if you're using the example paths thus far. Don't put the Epoch32 and don't put a trailing slash.
5. Sync the Renewal branch from git. Make sure you don't sync the master branch at this time.
6. Open the Visual Studio 2017 solution at EpochRoot\EpochVisualStudio\EpochVSIX\EpochVSIX.sln
7. Set the startup project to EpochProjectType.ProjectType and build the solution.
8. Export your VS settings to a file someplace safe (there are guides for this all over the web).
9. If all has gone well, run the solution, with or without debugging.
10. This will launch the Experimental instance of Visual Studio, where you can play with the Epoch ecosystem. Until packaging and installing the VS extension is a priority, this is the way to get to Epoch stuff.
11. You can import your VS settings into the Experimental instance to save some headache with redoing your environment and customizations.
12. In the VS Experimental Instance, you should be able to create a new Epoch project. It'll start off with a basic Hello World implementation in it. Make sure this compiles and runs and works.
13. You will probably have to copy the DLL files from Epoch32 to wherever the .EXEs are built to. Otherwise they may not run and will almost certainly not provide helpful error messages.
14. The good stuff is in EpochRoot\EpochCompiler\EpochCompiler.sln for the time being.
15. If you make it all the way here, you're all set for working with Epoch (either legacy or renewal editions)