From b6b8c6abecabdebd8b148ccb2bf7e0e99a2111a1 Mon Sep 17 00:00:00 2001 From: water Date: Thu, 3 Sep 2020 17:52:49 -0400 Subject: [PATCH 01/13] start the ObjectFileGenerator --- doc/object_file_generation.md | 78 ++++++++ doc/registers.md | 24 ++- doc/runtime_todo.md | 4 + goalc/CMakeLists.txt | 8 +- goalc/emitter/CMakeLists.txt | 3 - goalc/emitter/Instruction.h | 8 + goalc/emitter/ObjectFileData.cpp | 20 ++ goalc/emitter/ObjectFileData.h | 18 ++ goalc/emitter/ObjectGenerator.cpp | 292 ++++++++++++++++++++++++++++++ goalc/emitter/ObjectGenerator.h | 131 ++++++++++++++ test/CMakeLists.txt | 2 +- 11 files changed, 577 insertions(+), 11 deletions(-) create mode 100644 doc/object_file_generation.md create mode 100644 doc/runtime_todo.md delete mode 100644 goalc/emitter/CMakeLists.txt create mode 100644 goalc/emitter/ObjectFileData.cpp create mode 100644 goalc/emitter/ObjectFileData.h create mode 100644 goalc/emitter/ObjectGenerator.cpp create mode 100644 goalc/emitter/ObjectGenerator.h diff --git a/doc/object_file_generation.md b/doc/object_file_generation.md new file mode 100644 index 0000000000..db88362c98 --- /dev/null +++ b/doc/object_file_generation.md @@ -0,0 +1,78 @@ +# CGO/DGO Files +The CGO/DGO file format is exactly the same - the only difference is the name of the file. The DGO name indicates that the file contains all the data for a level. The engine will load these files into a level heap, which can then be cleared and replaced with a different level. + +I suspect that the DGO file name came first, as a package containing all the data in the level which can be loaded very quickly. Names in the code all say `dgo`, and the `MakeFileName` system shows that both CGO and DGO files are stored in the `game/dgo` folder. Probably the engine and kernel were packed into a CGO file after the file format was created for loading levels. + +Each CGO/DGO file contains a bunch of individual object files. Each file has a name. There are some duplicate names - sometimes the two files with the same names are very different (example, code for an enemy, art for an enemy), and other times they are very similar (tiny differences in code/data). The files come in two versions, v4 and v3, and both CGOs and DGOs contain both versions. If an object file has code in it, it is always a v3. It is possible to have a v3 file with just data, but usually the data is pretty small. The v4 files tend to have a lot of data in them. My theory is that the compiler creates v3 files out of GOAL source code files, and that other tools for creating things like textures/levels/art-groups generate v4 objects. There are a number of optimizations in the loading process for v4 objects that are better suited for larger files. To stay at 60 fps always, a v3 object must be smaller than around 750 kB. A v4 object does not have this limitation. + +# The V3 format +The v3 format is divided into three segments: +1. Main: this contains all of the functions/data that will be used by the game. +2. Debug: this is only loaded in debug mode, and is always stored on a separate `kdebugheap`. +3. Top Level: this contains some initialization code to add functions/variables to the symbol table, and any user-written initialization. It runs once, immediately after the object is loaded, then is thrown away. + +Each segment also has linking data, which tells the linker how to link references to symbols, types, and memory (possibly in a different segment). + +This format will be different between the PC and PS2 versions, as linking data for x86-64 will need to look different from MIPS. + +Each segments can contain functions and data. The top-level segment must start with a function which will be run to initialize the object. All the data here goes through the GOAL compiler and type system. + +# The V4 format +The V4 format contains just data. Like v3, the data is GOAL objects, but was probably generated by a tool that wasn't the compiler. A V4 object has no segments, but must start with a `basic` object. After being linked, the `relocate` method of this `basic` will be called, which should do any additional linking required for the specific object. + +Because this is just data, there's no reason for the PC version to change this format. This means we can also check the + +Note: you may see references to v2 format in the code. I believe v4 format is identical to v2, except the linking data is stored at the end, to enable a "don't copy the last object" optimization. The game's code uses the `work_v2` function on v4 objects as a result, and some of my comments may refer to v2, when I really mean v4. I believe there are no v2 objects in any games. + +# Plan +- Create a library for generating obj files in V3/V4 format + - V4 should match game exactly. Doesn't support code. + - V3 is our own thing. Must support code. + +We'll eventually create tools which use the library in V4 mode to generate object files for rebuilding levels and textures. We may need to wait until more about these formats is understood before trying this. + +The compiler will use the library in V3 mode to generate object files for each `gc` (GOAL source code) file. + +# CGO files +The only CGO files read are `KERNEL.CGO` and `GAME.CGO`. + + The `KERNEL.CGO` contains the GOAL kernel and some very basic libraries (`gcommon`, `gstring`, `gstate`, ...). I believe that `KERNEL` was always loaded on boot during development, as its required for the Listener to function. + +The `GAME.CGO` file combines the contents of the `ENGINE`, `COMMON` and `ART` CGO files. `ENGINE` contains the game engine code, `COMMON` contains level-specific code (outside of the game engine) that is always loaded. If code is used in basically all the levels, it makes sense to put in in `COMMON`, so it doesn't have to be loaded for each currently active level. The `ART` CGO contains common art/textures/models, like Jak and his animations. + +The `JUNGLE.CGO`, `MAINCAVE.CGO`, `SUNKEN.CGO` file contains some copies of files used in the jungle, cave, LPC levels. Some are a tiny bit different. I believe it is unused. + +The `L1.CGO` file contains basically all the level-specific code/Jak animations and some textures. It doesn't seem to contain any 3D models. It's unused, but I'm still interested in understanding its format, as the Jak 1 demos have this file. + +The `RACERP.CGO` file contains (I think) everything needed for the Zoomer. Unused. The same data appears in the levels as needed, maybe with some slight differences. + +The `VILLAGEP.CGO` file contains common code shared in village levels, which isn't much (oracle, warp gate). Unused. The same data appears in the levels as needed. + +The `WATER-AN.CGO` file contains some small code/data for water animations. Unused. The same data appears in the levels as needed. + +# CGO/DGO Loading Process +A CGO/DGO file is loaded onto a "heap", which is just a chunk of contiguous memory. The loading process is designed to be fast, and also able to fill the entire heap, and allow each object to allocate memory after it is loaded. The process works like this: + +1. Two temporary buffers are allocated at the end of the heap. These are sized so that they can fit the largest object file, not including the last object file. +2. The IOP begins loading, and is permitted to load the first two object files to the two temporary buffers +3. The main CPU waits for the first object file to be loaded. +4. While the second object file being loaded, the first object is "linked". The first step to linking is to copy the object file data from the temporary buffer to the bottom of the heap, kicking out all the other data in the process. The linking data is checked to see if it is in the top of the heap, and is moved there if it isn't already. The run-once initialization code is copied to another temporary allocation on top of the heap and the debug data is copied to the debug heap. +5. Still, while the second object file is being loaded, the linker runs on the first object file. +6. Still, while the second object file is being loaded, the second object's initialization code is run (located in top of the heap). The second object may allocate from this heap, and will get valid memory without creating gaps in the heap. +7. Memory allocated from the top during linking is freed. +8. The IOP is allowed to load into the first buffer again. +9. The main CPU waits for the second object to be loaded, if the IOP hasn't finished yet. +10. This double-buffering pattern continues - while one object is loaded into a buffer, the other one will be copied to the bottom of the heap, linked, and initialized. When the second to last object is loaded, the IOP will wait an extra time until the main CPU has finished linking it until loading the last object (one additional wait) because the last object has a special case. +11. The last object will be loaded directly onto the bottom of the heap, as there may not be enough memory to use the temporary buffers and load the last object. The temporary buffers are freed. +12. If the last object is a v3, its linking data will be moved to the top-level, and the object data will be moved to fill in the gap left behind. If the last object is a v2, the main data will be at the beginning of the object data, so there is an optimization that will avoid copying the object data to save time, if the data is already close to being in the right place. + + +Generally the last file in a level DGO will be the largest v4 object. You can only have one file larger than a temporary buffer, and it must come last. The last file also doesn't have to be copied after being loaded into memory if it is a v4. + +V3 max size: + A V3 object is copied all at once with a single `ultimate-memcpy`. Usually linking gets to run for around 3 to 5% of a total frame. The `ultimate-memcpy` routine does a to/from scratchpad transfer. In practice, mem/spr transfers are around 1800 MB/sec, and the data has to be copied twice, so the effective bandwidth is 900 MB/sec. + + `900 MB / second * (0.04 * 0.0167 seconds) = 601 kilobytes` + + This estimate is backed up by the the chunk size of the v4 copy routine, which copies one chunk per frame. It picks 524 kB as the maximum amount that's safe to copy per frame. + \ No newline at end of file diff --git a/doc/registers.md b/doc/registers.md index 97d5280b8d..d0ff404620 100644 --- a/doc/registers.md +++ b/doc/registers.md @@ -87,13 +87,13 @@ The main RAM is mapped at `0x0` on the PS2, with the first 1 MB reserved for the In the C Kernel code, the `r15` pointer doesn't exist. Instead, `g_ee_main_memory` is a global which points to the beginning of GOAL main memory. The `Ptr` template class takes care of converting GOAL and C++ pointers in a convenient way, and catches null pointer access. -The GOAL stack pointer should likely be a real pointer, for performance reasons. This makes pushing/popping/calling/returning/accessing stack variables much faster, with the only cost being getting a GOAL stack pointer requiring some extra work. The stack pointer's value is read/written extremely rarely, so this seems like a good tradeoff. +The GOAL stack pointer should likely be a real pointer, for performance reasons. This makes pushing/popping/calling/returning/accessing stack variables much faster (can use actual `push`, `pop`), with the only cost being getting a GOAL stack pointer requiring some extra work. The stack pointer's value is read/written extremely rarely (only in kernel code that will be rewritten anyway), so this seems like a good tradeoff. The other registers are less clear. The process pointer can probably be a real pointer. But the symbol table could go a few ways: 1. Make it a real pointer. Symbol value access is fast, but comparison against false requires two extra operations. -2. Make it a GOAL pointer. Symbol value access requires more complicated addressing modes, but comparison against false is fast. +2. Make it a GOAL pointer. Symbol value access requires more complicated addressing modes to be one instruction, but comparison against false is fast. -Right now I'm leaning toward 1, but making it a configurable option in case I'm wrong. It should only be a change in a few places (emitter + where it's set up in the runtime). +Right now I'm leaning toward 2, but it shouldn't be a huge amount of work to change if I'm wrong. ### Plan for Function Call and Arguments In GOAL for MIPS, function calls are weird. Functions are always called by register using `t9`. There seems to be a different register allocator for function pointers, as nested function calls have really wacky register allocation. In GOAL-x86-64, this restriction will be removed, and a function can be called from any register. (see next section for why we can do this) @@ -101,11 +101,23 @@ In GOAL for MIPS, function calls are weird. Functions are always called by regi Unfortunately, GOAL's 128-bit function arguments present a big challenge. When calling a function, we can't know if the function we're calling is expecting an integer, float, or 128-bit integer. In fact, the caller may not even know if it has an integer, float, or 128-bit integer. The easy and foolproof way to get this right is to use 128-bit `xmm` registers for all arguments and return values, but this will cause a massive performance hit and increase code size, as we'll have to move values between register types constantly. The current plan is this: - Floats go in GPRs for arguments/return values. GOAL does this too, and takes the hit of converting between registers as well. Probably the impact on a modern CPU is even worse, but we can live with it. -- We'll compromise - +- We'll compromise for 128-bit function calls. When the compiler can figure out that the function being called expects or returns a 128-bit value, it will use the 128-bit calling convention. In all other cases, it will use 64-bit. There aren't many places where 128-bit integer are used outside of inline assembly, so I suspect this will just work. If there are more complicated instances (call a function pointer and get either a 64 or 128-bit result), we will need to special case them. ### Plan for Static Data +The original GOAL implementation always called functions by using the `t9` register. So, on entry to a function, the `t9` register contains the address of the function. If the function needs to access static data, it will move this `fp`, then do `fp` relative addressing to load data. Example: +``` +function-start: + daddiu sp, sp, -16 ;; allocate space on stack + sd fp, 8(sp) ;; back up old fp on stack + or fp, t9, r0 ;; set fp to address of function + lwc1 f0, L345(fp) ;; load relative to function start +``` + +To copy this exactly on x86 would require reserving two registers equivalent to `t9` and `gp`. A better approach for x86-64 is to use "RIP relative addressing". This can be used to load memory relative to the current instruction pointer. This addressing mode can be used with "load effective address" (`lea`) to create pointers to static data as well. ### Plan for Memory +Access memory by GOAL pointer in `rx` with constant offset (optionally zero): +``` +mov rdest, [roff + rx + offset] +``` -### Other details diff --git a/doc/runtime_todo.md b/doc/runtime_todo.md new file mode 100644 index 0000000000..d87a3c0ba1 --- /dev/null +++ b/doc/runtime_todo.md @@ -0,0 +1,4 @@ +# Runtime To-Do for Compiler Upgrade +- Handle `xmm`'s correctly for windows +- Change offset, etc +- Memory mapping so null pointer dereference causes a crash \ No newline at end of file diff --git a/goalc/CMakeLists.txt b/goalc/CMakeLists.txt index 1ec43eb75f..9cc869a2c8 100644 --- a/goalc/CMakeLists.txt +++ b/goalc/CMakeLists.txt @@ -1,8 +1,14 @@ add_subdirectory(util) add_subdirectory(goos) add_subdirectory(listener) -add_subdirectory(emitter) +add_library(compiler + SHARED + emitter/CodeTester.cpp + emitter/ObjectFileData.cpp + emitter/ObjectGenerator.cpp + emitter/Register.cpp + ) add_executable(goalc main.cpp compiler/Compiler.cpp) diff --git a/goalc/emitter/CMakeLists.txt b/goalc/emitter/CMakeLists.txt deleted file mode 100644 index 1d6cc0dc8f..0000000000 --- a/goalc/emitter/CMakeLists.txt +++ /dev/null @@ -1,3 +0,0 @@ -add_library(emitter - Register.cpp - CodeTester.cpp) \ No newline at end of file diff --git a/goalc/emitter/Instruction.h b/goalc/emitter/Instruction.h index aab76e1a42..cfe7254dfd 100644 --- a/goalc/emitter/Instruction.h +++ b/goalc/emitter/Instruction.h @@ -141,6 +141,14 @@ struct Instruction { op3 = b; } + int get_imm_size() const { + if (set_imm) { + return imm.size; + } else { + return 0; + } + } + /*! * Set modrm and rex as needed for two regs. */ diff --git a/goalc/emitter/ObjectFileData.cpp b/goalc/emitter/ObjectFileData.cpp new file mode 100644 index 0000000000..b18e1468cc --- /dev/null +++ b/goalc/emitter/ObjectFileData.cpp @@ -0,0 +1,20 @@ +#include "ObjectFileData.h" + +namespace emitter { +std::vector ObjectFileData::to_vector() const { + std::vector result; + // header + result.insert(result.end(), header.begin(), header.end()); + + // link tables + for (int seg = N_SEG; seg-- > 0;) { + result.insert(result.end(), link_tables[seg].begin(), link_tables[seg].end()); + } + + // data (code + static objects, by segment) + for (int seg = N_SEG; seg-- > 0;) { + result.insert(result.end(), segment_data[seg].begin(), segment_data[seg].end()); + } + return result; +} +} // namespace emitter \ No newline at end of file diff --git a/goalc/emitter/ObjectFileData.h b/goalc/emitter/ObjectFileData.h new file mode 100644 index 0000000000..71f95414e7 --- /dev/null +++ b/goalc/emitter/ObjectFileData.h @@ -0,0 +1,18 @@ +#ifndef JAK_OBJECTFILEDATA_H +#define JAK_OBJECTFILEDATA_H + +#include +#include +#include "common/common_types.h" +#include "common/link_types.h" + +namespace emitter { +struct ObjectFileData { + std::array, N_SEG> segment_data; + std::array, N_SEG> link_tables; + std::vector header; + std::vector to_vector() const; +}; +} // namespace emitter + +#endif // JAK_OBJECTFILEDATA_H diff --git a/goalc/emitter/ObjectGenerator.cpp b/goalc/emitter/ObjectGenerator.cpp new file mode 100644 index 0000000000..2b974b2855 --- /dev/null +++ b/goalc/emitter/ObjectGenerator.cpp @@ -0,0 +1,292 @@ +/*! + * @file ObjectGenerator.cpp + * Tool to build GOAL object files. Will eventually support v3 and v4. + * + * There are 5 steps: + * 1. The user adds static data / instructions and specifies links. + * 2. The functions and static data are laid out in memory + * 3. The user specified links are updated according to the memory layout, and jumps are patched + * 4. The link table is generated for each segment + * 5. All segments and link tables are put into a final object file, along with a header. + * + * Step 1 can be done with the add_.... and link_... functions + * Steps 2 - 5 are done in generate_data_vX() + */ + +#include "ObjectGenerator.h" +#include "common/goal_constants.h" +#include "common/versions.h" + +namespace emitter { + +ObjectFileData ObjectGenerator::generate_data_v3() { + ObjectFileData out; + + // do functions (step 2, part 1) + for (int seg = N_SEG; seg-- > 0;) { + auto& data = m_data_by_seg.at(seg); + // loop over functions in this segment + for (auto& function : m_function_data_by_seg.at(seg)) { + // align + while (data.size() % function.min_align) { + insert_data(seg, 0); + } + + // add a type tag link + m_type_ptr_links_by_seg.at(seg)["function"].push_back(data.size()); + + // add room for a type tag + for (int i = 0; i < POINTER_SIZE; i++) { + insert_data(seg, 0xae); + } + + // insert instructions! + for (const auto& instr : function.instructions) { + u8 temp[128]; + auto count = instr.emit(temp); + assert(count < 128); + function.instruction_to_byte_in_data.push_back(data.size()); + for (int i = 0; i < count; i++) { + insert_data(seg, temp[i]); + } + } + } + } + + // do static data layout (step 2, part 2) + for (int seg = N_SEG; seg-- > 0;) { + auto& data = m_data_by_seg.at(seg); + for (auto& s : m_static_data_by_seg.at(seg)) { + // align + while (data.size() % s.min_align) { + insert_data(seg, 0); + } + + s.location = data.size(); + + data.insert(data.end(), s.data.begin(), s.data.end()); + } + } + + // step 3, cleaning up things now that we know the memory layout + for (int seg = N_SEG; seg-- > 0;) { + // jumps? + handle_temp_static_type_links(seg); + handle_temp_jump_links(seg); + } + + // actual linking? + for (int seg = N_SEG; seg-- > 0;) { + } + + // emit header + emit_link_tables(); + out.header = generate_header_v3(); + out.segment_data = std::move(m_data_by_seg); + out.link_tables = std::move(m_link_by_seg); + return out; +} + +/*! + * Add a new function to seg, and return a FunctionRecord which can be used to specify this + * new function. + */ +FunctionRecord ObjectGenerator::add_function_to_seg(int seg, int min_align) { + FunctionRecord rec; + rec.seg = seg; + rec.func_id = int(m_function_data_by_seg.at(seg).size()); + m_function_data_by_seg.at(seg).emplace_back(); + m_function_data_by_seg.at(seg).back().min_align = min_align; + return rec; +} + +/*! + * Add a new IR instruction to the function. An IR instruction may contain 0, 1, or multiple + * actual Instructions. These Instructions can be added with add_instruction. The IR_Record + * can be used as a label for jump targets. + */ +IR_Record ObjectGenerator::add_ir(const FunctionRecord& func) { + // verify we aren't adding to an old function. not technically an error, but doesn't make sense + assert(func.func_id == int(m_function_data_by_seg.at(func.seg).size()) - 1); + IR_Record rec; + rec.seg = func.seg; + rec.func_id = func.func_id; + auto& func_data = m_function_data_by_seg.at(rec.seg).at(rec.func_id); + rec.ir_id = int(func_data.ir_to_instruction.size()); + func_data.ir_to_instruction.push_back(int(func_data.instructions.size())); + return rec; +} + +/*! + * Add a new Instruction for the given IR instruction. + */ +InstructionRecord ObjectGenerator::add_instr(Instruction inst, IR_Record ir) { + // verify we aren't adding to an old instruction or function + assert(ir.func_id == int(m_function_data_by_seg.at(ir.seg).size()) - 1); + // only this second condition is an actual error. + assert(ir.ir_id == + int(m_function_data_by_seg.at(ir.seg).at(ir.func_id).ir_to_instruction.size()) - 1); + + InstructionRecord rec; + rec.seg = ir.seg; + rec.func_id = ir.func_id; + rec.ir_id = ir.ir_id; + auto& func_data = m_function_data_by_seg.at(rec.seg).at(rec.func_id); + rec.instr_id = int(func_data.instructions.size()); + func_data.instructions.push_back(inst); + return rec; +} + +/*! + * Create a new static object in the given segment. + */ +StaticRecord ObjectGenerator::add_static_to_seg(int seg, int min_align) { + StaticRecord rec; + rec.seg = seg; + rec.static_id = m_static_data_by_seg.at(seg).size(); + m_static_data_by_seg.at(seg).emplace_back(); + m_static_data_by_seg.at(seg).back().min_align = min_align; + return rec; +} + +/*! + * Add linking data to add a type pointer in rec at offset. + * This will add an entry to the linking data, which will get patched at runtime, during linking. + */ +void ObjectGenerator::link_static_type_ptr(StaticRecord rec, + int offset, + const std::string& type_name) { + StaticTypeLink link; + link.offset = offset; + link.rec = rec; + m_static_type_temp_links_by_seg.at(rec.seg)[type_name].push_back(link); +} + +/*! + * This will patch the jump_instr to jump to destination. This happens during compile time and + * doesn't add anything to the link table. The jump_instr must already be emitted, however the + * destination can be a future IR. To get a reference to a future IR, you must know the index and + * use get_future_ir. + */ +void ObjectGenerator::link_instruction_jump(InstructionRecord jump_instr, IR_Record destination) { + // must jump within our own function. + assert(jump_instr.seg == destination.seg); + assert(jump_instr.func_id == destination.func_id); + m_jump_temp_links_by_seg.at(jump_instr.seg).push_back({jump_instr, destination}); +} + +/*! + * m_static_type_temp_links_by_seg -> m_type_ptr_links_by_seg + */ +void ObjectGenerator::handle_temp_static_type_links(int seg) { + for (const auto& type_links : m_static_type_temp_links_by_seg.at(seg)) { + const auto& type_name = type_links.first; + for (const auto& link : type_links.second) { + assert(seg == link.rec.seg); + const auto& static_object = m_static_data_by_seg.at(seg).at(link.rec.static_id); + int total_offset = static_object.location + link.offset; + m_type_ptr_links_by_seg.at(seg)[type_name].push_back(total_offset); + } + } +} + +/*! + * m_jump_temp_links_by_seg patching + */ +void ObjectGenerator::handle_temp_jump_links(int seg) { + for (const auto& link : m_jump_temp_links_by_seg.at(seg)) { + // we need to compute three offsets, all relative to the start of data. + // 1). the location of the patch (the immediate of the opcode) + // 2). the value of RIP at the jump (the instruction after the jump, on x86) + // 3). the value of RIP we want + const auto& function = m_function_data_by_seg.at(seg).at(link.jump_instr.func_id); + assert(link.jump_instr.func_id == link.dest.func_id); + assert(link.jump_instr.seg == seg); + assert(link.dest.seg == seg); + const auto& jump_instr = function.instructions.at(link.jump_instr.instr_id); + assert(jump_instr.get_imm_size() == 4); + + // 1). patch = instruction location + location of imm in instruction. + int patch_location = function.instruction_to_byte_in_data.at(link.jump_instr.instr_id) + + jump_instr.offset_of_imm(); + + // 2). source rip = jump instr + 1 location + int source_rip = function.instruction_to_byte_in_data.at(link.jump_instr.instr_id + 1); + + // 3). dest rip = first instruction of dest IR + int dest_rip = + function.instruction_to_byte_in_data.at(function.ir_to_instruction.at(link.dest.ir_id)); + + patch_data(seg, patch_location, dest_rip - source_rip); + } +} + +void ObjectGenerator::emit_link_tables() { + // todo! +} + +namespace { +template +uint32_t push_data(const T& data, std::vector& v) { + auto insert = v.size(); + v.resize(insert + sizeof(T)); + memcpy(v.data() + insert, &data, sizeof(T)); + return sizeof(T); +} +} // namespace + +/*! + * Generate linker header. + */ +std::vector ObjectGenerator::generate_header_v3() { + std::vector result; + + // header starts with a "GOAL" magic word + result.push_back('G'); + result.push_back('O'); + result.push_back('A'); + result.push_back('L'); + + u32 offset = 0; // the GOAL doesn't count toward the offset, first 4 bytes are killed. + // then, the version. todo, bump the version once we use this! + offset += push_data(versions::GOAL_VERSION_MAJOR, result); + offset += push_data(versions::GOAL_VERSION_MINOR, result); + + // the object file version + offset += push_data(3, result); + // the segment count + offset += push_data(N_SEG, result); + + offset += sizeof(u32) * N_SEG * 4; // 4 u32's per segment + + struct SizeOffset { + uint32_t offset, size; + }; + + struct SizeOffsetTable { + SizeOffset link_seg[N_SEG]; + SizeOffset code_seg[N_SEG]; + }; + + SizeOffsetTable table; + int total_link_size = 0; + + for (int i = N_SEG; i-- > 0;) { + table.link_seg[i].offset = offset; // start of the link + table.link_seg[i].size = m_link_by_seg[i].size(); // size of the link data + offset += m_link_by_seg[i].size(); // to next link data + total_link_size += m_link_by_seg[i].size(); // need to track this. + } + + offset = 0; + for (int i = N_SEG; i-- > 0;) { + table.code_seg[i].offset = offset; + table.code_seg[i].size = m_data_by_seg[i].size(); + offset += m_data_by_seg[i].size(); + } + + push_data(table, result); + push_data(64 + 4 + total_link_size, result); // todo, make these numbers less magic. + return result; +} +} // namespace emitter \ No newline at end of file diff --git a/goalc/emitter/ObjectGenerator.h b/goalc/emitter/ObjectGenerator.h new file mode 100644 index 0000000000..b9b9de4a9a --- /dev/null +++ b/goalc/emitter/ObjectGenerator.h @@ -0,0 +1,131 @@ +#ifndef JAK_OBJECTGENERATOR_H +#define JAK_OBJECTGENERATOR_H + +#include +#include +#include "ObjectFileData.h" +#include "Instruction.h" + +namespace emitter { + +struct FunctionRecord { + int seg = -1; + int func_id = -1; +}; + +struct IR_Record { + int seg = -1; + int func_id = -1; + int ir_id = -1; +}; + +struct InstructionRecord { + int seg = -1; + int func_id = -1; + int ir_id = -1; + int instr_id = -1; +}; + +struct StaticRecord { + int seg = -1; + int static_id = -1; +}; + +class ObjectGenerator { + public: + ObjectGenerator() = default; + ObjectFileData generate_data_v3(); + + FunctionRecord add_function_to_seg(int seg, + int min_align = 16); // should align and insert function tag + IR_Record add_ir(const FunctionRecord& func); + IR_Record get_future_ir_record(const FunctionRecord& func, int ir_id); + InstructionRecord add_instr(Instruction inst, IR_Record ir); + StaticRecord add_static_to_seg(int seg, int min_align = 16); + void link_instruction_jump(InstructionRecord jump_instr, IR_Record destination); + void link_instruction_symbol(); + void link_instruction_static(); + void link_instruction_to_function(); + void link_static_type_ptr(StaticRecord rec, int offset, const std::string& type_name); + void link_static_sym(); + + void link_type_pointer(const std::string& type_name, const StaticRecord& rec, int offset); + + private: + void handle_temp_static_type_links(int seg); + void handle_temp_jump_links(int seg); + void emit_link_tables(); + std::vector generate_header_v3(); + + template + void insert_data(int seg, const T& x) { + auto& data = m_data_by_seg.at(seg); + auto insert_location = data.size(); + data.resize(insert_location + sizeof(T)); + memcpy(data.data() + insert_location, &x, sizeof(T)); + } + + template + void patch_data(int seg, int offset, const T& x) { + auto& data = m_data_by_seg.at(seg); + assert(offset >= 0); + assert(offset + sizeof(T) <= data.size()); + memcpy(data.data() + offset, &x, sizeof(T)); + } + // jumps, patched before (JumpLink) + // ptrs/access to statics (RipLink) + // type pointers + // symbol links + + struct FunctionData { + std::vector instructions; + std::vector ir_to_instruction; + std::vector instruction_to_byte_in_data; + int min_align = 16; + }; + + struct StaticData { + std::vector data; + int min_align = 16; + int location = -1; + }; + + struct StaticTypeLink { + StaticRecord rec; + int offset = -1; + }; + + // struct SymbolLink { + // + // }; + // + // struct RipLink { + // + // }; + // + struct JumpLink { + InstructionRecord jump_instr; + IR_Record dest; + }; + + // final data + std::array, N_SEG> m_data_by_seg; + std::array, N_SEG> m_link_by_seg; + + // temp data + std::array, N_SEG> m_function_data_by_seg; + std::array, N_SEG> m_static_data_by_seg; + + // temp link stuff + std::array>, N_SEG> + m_static_type_temp_links_by_seg; + std::array, N_SEG> m_jump_temp_links_by_seg; + + // final link stuff + std::array>, N_SEG> m_type_ptr_links_by_seg; + // std::array, N_SEG> m_sym_links_by_seg; + // std::array, N_SEG> m_rip_links_by_seg; +}; +} // namespace emitter + +#endif // JAK_OBJECTGENERATOR_H diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt index 05be29603e..61ad910a94 100644 --- a/test/CMakeLists.txt +++ b/test/CMakeLists.txt @@ -14,4 +14,4 @@ add_executable(goalc-test test_emitter_integer_math.cpp ) -target_link_libraries(goalc-test goos util listener runtime emitter type_system gtest) \ No newline at end of file +target_link_libraries(goalc-test goos util listener runtime compiler type_system gtest) \ No newline at end of file From ff8be70a1078a72d05af95953e443ee923633f9b Mon Sep 17 00:00:00 2001 From: water Date: Fri, 4 Sep 2020 20:43:09 -0400 Subject: [PATCH 02/13] finish v3 generation --- goalc/CMakeLists.txt | 11 +- goalc/compiler/Compiler.cpp | 25 ++++ goalc/compiler/Compiler.h | 5 + goalc/compiler/Env.cpp | 3 + goalc/compiler/Env.h | 8 ++ goalc/compiler/Val.cpp | 3 + goalc/compiler/Val.h | 11 ++ goalc/emitter/Instruction.h | 8 ++ goalc/emitter/ObjectGenerator.cpp | 213 +++++++++++++++++++++++++++++- goalc/emitter/ObjectGenerator.h | 96 ++++++++++---- goalc/logger/CMakeLists.txt | 1 + goalc/logger/Logger.cpp | 67 ++++++++++ goalc/logger/Logger.h | 43 ++++++ goalc/regalloc/IRegister.cpp | 3 + goalc/regalloc/IRegister.h | 24 ++++ 15 files changed, 486 insertions(+), 35 deletions(-) create mode 100644 goalc/compiler/Env.cpp create mode 100644 goalc/compiler/Env.h create mode 100644 goalc/compiler/Val.cpp create mode 100644 goalc/compiler/Val.h create mode 100644 goalc/logger/CMakeLists.txt create mode 100644 goalc/logger/Logger.cpp create mode 100644 goalc/logger/Logger.h create mode 100644 goalc/regalloc/IRegister.cpp create mode 100644 goalc/regalloc/IRegister.h diff --git a/goalc/CMakeLists.txt b/goalc/CMakeLists.txt index 889b959f56..d932421733 100644 --- a/goalc/CMakeLists.txt +++ b/goalc/CMakeLists.txt @@ -1,5 +1,7 @@ add_subdirectory(util) add_subdirectory(goos) +add_subdirectory(logger) + IF (WIN32) # TODO - implement windows listener message("Windows Listener Not Implemented!") @@ -13,13 +15,18 @@ add_library(compiler emitter/ObjectFileData.cpp emitter/ObjectGenerator.cpp emitter/Register.cpp + compiler/Compiler.cpp + compiler/Env.cpp + compiler/Val.cpp + regalloc/IRegister.cpp + compiler/Compiler.cpp ) add_executable(goalc main.cpp - compiler/Compiler.cpp) + ) IF (WIN32) set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON) ENDIF() -target_link_libraries(goalc util goos type_system) +target_link_libraries(goalc util goos type_system compiler logger type_system) diff --git a/goalc/compiler/Compiler.cpp b/goalc/compiler/Compiler.cpp index b1d4ba1350..49e4b2aae1 100644 --- a/goalc/compiler/Compiler.cpp +++ b/goalc/compiler/Compiler.cpp @@ -1 +1,26 @@ #include "Compiler.h" +#include "goalc/logger/Logger.h" + +Compiler::Compiler() { + init_logger(); + m_ts.add_builtin_types(); +} + +void Compiler::execute_repl() { + +} + +Compiler::~Compiler() { + gLogger.close(); +} + +void Compiler::init_logger() { + gLogger.set_file("compiler.txt"); + gLogger.config[MSG_COLOR].kind = LOG_FILE; + gLogger.config[MSG_DEBUG].kind = LOG_IGNORE; + gLogger.config[MSG_TGT].color = COLOR_GREEN; + gLogger.config[MSG_TGT_INFO].color = COLOR_BLUE; + gLogger.config[MSG_WARN].color = COLOR_RED; + gLogger.config[MSG_ICE].color = COLOR_RED; + gLogger.config[MSG_ERR].color = COLOR_RED; +} \ No newline at end of file diff --git a/goalc/compiler/Compiler.h b/goalc/compiler/Compiler.h index c9565aae14..3dd5a4904b 100644 --- a/goalc/compiler/Compiler.h +++ b/goalc/compiler/Compiler.h @@ -5,7 +5,12 @@ class Compiler { public: + Compiler(); + ~Compiler(); + void execute_repl(); private: + void init_logger(); + TypeSystem m_ts; }; diff --git a/goalc/compiler/Env.cpp b/goalc/compiler/Env.cpp new file mode 100644 index 0000000000..785a75a876 --- /dev/null +++ b/goalc/compiler/Env.cpp @@ -0,0 +1,3 @@ + + +#include "Env.h" diff --git a/goalc/compiler/Env.h b/goalc/compiler/Env.h new file mode 100644 index 0000000000..417f99f77b --- /dev/null +++ b/goalc/compiler/Env.h @@ -0,0 +1,8 @@ + + +#ifndef JAK_ENV_H +#define JAK_ENV_H + +class Env {}; + +#endif // JAK_ENV_H diff --git a/goalc/compiler/Val.cpp b/goalc/compiler/Val.cpp new file mode 100644 index 0000000000..b7fee6cae7 --- /dev/null +++ b/goalc/compiler/Val.cpp @@ -0,0 +1,3 @@ + + +#include "Val.h" diff --git a/goalc/compiler/Val.h b/goalc/compiler/Val.h new file mode 100644 index 0000000000..eacdc28a18 --- /dev/null +++ b/goalc/compiler/Val.h @@ -0,0 +1,11 @@ +/*! + * @file Val.h + * The GOAL Value. A value represents a place (where the value is stored) and a type. + */ + +#ifndef JAK_VAL_H +#define JAK_VAL_H + +class Val {}; + +#endif // JAK_VAL_H diff --git a/goalc/emitter/Instruction.h b/goalc/emitter/Instruction.h index cfe7254dfd..cba31f888e 100644 --- a/goalc/emitter/Instruction.h +++ b/goalc/emitter/Instruction.h @@ -149,6 +149,14 @@ struct Instruction { } } + int get_disp_size() const { + if (set_disp_imm) { + return disp.size; + } else { + return 0; + } + } + /*! * Set modrm and rex as needed for two regs. */ diff --git a/goalc/emitter/ObjectGenerator.cpp b/goalc/emitter/ObjectGenerator.cpp index 2b974b2855..988d61bd78 100644 --- a/goalc/emitter/ObjectGenerator.cpp +++ b/goalc/emitter/ObjectGenerator.cpp @@ -19,6 +19,9 @@ namespace emitter { +/*! + * Build an object file with the v3 format. + */ ObjectFileData ObjectGenerator::generate_data_v3() { ObjectFileData out; @@ -70,17 +73,20 @@ ObjectFileData ObjectGenerator::generate_data_v3() { // step 3, cleaning up things now that we know the memory layout for (int seg = N_SEG; seg-- > 0;) { - // jumps? handle_temp_static_type_links(seg); handle_temp_jump_links(seg); + handle_temp_instr_sym_links(seg); + handle_temp_rip_func_links(seg); + handle_temp_rip_data_links(seg); } // actual linking? for (int seg = N_SEG; seg-- > 0;) { + emit_link_table(seg); } // emit header - emit_link_tables(); + out.header = generate_header_v3(); out.segment_data = std::move(m_data_by_seg); out.link_tables = std::move(m_link_by_seg); @@ -117,6 +123,19 @@ IR_Record ObjectGenerator::add_ir(const FunctionRecord& func) { return rec; } +/*! + * Get an IR Record that points to an IR that hasn't been added yet. This can be used to create + * jumps forward to things we haven't seen yet. + */ +IR_Record ObjectGenerator::get_future_ir_record(const FunctionRecord& func, int ir_id) { + assert(func.func_id == int(m_function_data_by_seg.at(func.seg).size()) - 1); + IR_Record rec; + rec.seg = func.seg; + rec.func_id = func.func_id; + rec.ir_id = ir_id; + return rec; +} + /*! * Add a new Instruction for the given IR instruction. */ @@ -176,7 +195,49 @@ void ObjectGenerator::link_instruction_jump(InstructionRecord jump_instr, IR_Rec } /*! + * Patch a load/store instruction to refer to a symbol. This patching will happen at runtime + * linking. The instruction must use 32-bit immediate displacement addressing, relative to the + * symbol table. + */ +void ObjectGenerator::link_instruction_symbol_mem(const InstructionRecord& rec, + const std::string& name) { + m_symbol_instr_temp_links_by_seg.at(rec.seg)[name].push_back({rec, true}); +} + +/*! + * Patch an add instruction to generate a pointer to a symbol. This patching will happen during + * runtime linking. The instruction should be an "add st, imm32". + */ +void ObjectGenerator::link_instruction_symbol_ptr(const InstructionRecord& rec, + const std::string& name) { + m_symbol_instr_temp_links_by_seg.at(rec.seg)[name].push_back({rec, false}); +} + +/*! + * Insert a GOAL pointer to a symbol inside of static data. This patching will happen during runtime + * linking. + */ +void ObjectGenerator::link_static_symbol_ptr(StaticRecord rec, + int offset, + const std::string& name) { + m_static_sym_temp_links_by_seg.at(rec.seg)[name].push_back({rec, offset}); +} + +void ObjectGenerator::link_instruction_static(const InstructionRecord& instr, + const StaticRecord& target_static, + int offset) { + m_rip_data_temp_links_by_seg.at(instr.seg).push_back({instr, target_static, offset}); +} + +void ObjectGenerator::link_instruction_to_function(const InstructionRecord& instr, + const FunctionRecord& target_func) { + m_rip_func_temp_links_by_seg.at(instr.seg).push_back({instr, target_func}); +} + +/*! + * Convert: * m_static_type_temp_links_by_seg -> m_type_ptr_links_by_seg + * after memory layout is done and before link tables are generated */ void ObjectGenerator::handle_temp_static_type_links(int seg) { for (const auto& type_links : m_static_type_temp_links_by_seg.at(seg)) { @@ -191,7 +252,24 @@ void ObjectGenerator::handle_temp_static_type_links(int seg) { } /*! - * m_jump_temp_links_by_seg patching + * Convert: + * m_static_sym_temp_links_by_seg -> m_sym_links_by_seg + * after memory layout is done and before link tables are generated + */ +void ObjectGenerator::handle_temp_static_sym_links(int seg) { + for (const auto& sym_links : m_static_sym_temp_links_by_seg.at(seg)) { + const auto& sym_name = sym_links.first; + for (const auto& link : sym_links.second) { + assert(seg == link.rec.seg); + const auto& static_object = m_static_data_by_seg.at(seg).at(link.rec.static_id); + int total_offset = static_object.location + link.offset; + m_sym_links_by_seg.at(seg)[sym_name].push_back(total_offset); + } + } +} + +/*! + * m_jump_temp_links_by_seg patching after memory layout is done */ void ObjectGenerator::handle_temp_jump_links(int seg) { for (const auto& link : m_jump_temp_links_by_seg.at(seg)) { @@ -221,8 +299,51 @@ void ObjectGenerator::handle_temp_jump_links(int seg) { } } -void ObjectGenerator::emit_link_tables() { - // todo! +/*! + * Convert: + * m_symbol_instr_temp_links_by_seg -> m_sym_links_by_seg + * after memory layout is done and before link tables are generated + */ +void ObjectGenerator::handle_temp_instr_sym_links(int seg) { + for (const auto& links : m_symbol_instr_temp_links_by_seg.at(seg)) { + const auto& sym_name = links.first; + for (const auto& link : links.second) { + assert(seg == link.rec.seg); + const auto& function = m_function_data_by_seg.at(seg).at(link.rec.func_id); + const auto& instruction = function.instructions.at(link.rec.instr_id); + int offset_of_instruction = function.instruction_to_byte_in_data.at(link.rec.instr_id); + int offset_in_instruction = + link.is_mem_access ? instruction.offset_of_disp() : instruction.offset_of_imm(); + if (link.is_mem_access) { + assert(instruction.get_disp_size() == 4); + } else { + assert(instruction.get_imm_size() == 4); + } + m_sym_links_by_seg.at(seg)[sym_name].push_back(offset_of_instruction + offset_in_instruction); + } + } +} + +void ObjectGenerator::handle_temp_rip_func_links(int seg) { + for (const auto& link : m_rip_func_temp_links_by_seg.at(seg)) { + RipLink result; + result.instr = link.instr; + result.target_segment = link.target.seg; + const auto& target_func = m_function_data_by_seg.at(link.target.seg).at(link.target.func_id); + result.offset_in_segment = target_func.instruction_to_byte_in_data.at(0); + m_rip_links_by_seg.at(seg).push_back(result); + } +} + +void ObjectGenerator::handle_temp_rip_data_links(int seg) { + for (const auto& link : m_rip_data_temp_links_by_seg.at(seg)) { + RipLink result; + result.instr = link.instr; + result.target_segment = link.data.seg; + const auto& target = m_static_data_by_seg.at(link.data.seg).at(link.data.static_id); + result.offset_in_segment = target.location + link.offset; + m_rip_links_by_seg.at(seg).push_back(result); + } } namespace { @@ -235,6 +356,88 @@ uint32_t push_data(const T& data, std::vector& v) { } } // namespace +void ObjectGenerator::emit_link_type_pointer(int seg) { + auto& out = m_link_by_seg.at(seg); + for (auto& rec : m_type_ptr_links_by_seg.at(seg)) { + u32 size = rec.second.size(); + if (!size) { + continue; + } + + // start + out.push_back(LINK_TYPE_PTR); + + // name + for (char c : rec.first) { + out.push_back(c); + } + out.push_back(0); + + // method count + out.push_back(0); // todo! + + // number of links + push_data(size, out); + + for (auto& r : rec.second) { + push_data(r, out); + } + } +} + +void ObjectGenerator::emit_link_symbol(int seg) { + auto& out = m_link_by_seg.at(seg); + for (auto& rec : m_sym_links_by_seg.at(seg)) { + out.push_back(LINK_SYMBOL_OFFSET); + for (char c : rec.first) { + out.push_back(c); + } + out.push_back(0); + + // number of links + push_data(rec.second.size(), out); + + for (auto& r : rec.second) { + push_data(r, out); + } + } +} + +void ObjectGenerator::emit_link_rip(int seg) { + auto& out = m_link_by_seg.at(seg); + for (auto& rec : m_rip_links_by_seg.at(seg)) { + // kind (u8) + // target segment (u8) + // offset in current (u32) + // offset into target (u32) + // patch loc (u32) (todo, make this a s8 offset from offset into current?) + + // kind + out.push_back(LINK_DISTANCE_TO_OTHER_SEG_32); + // target segment + out.push_back(rec.target_segment); + // offset into current + const auto& src_func = m_function_data_by_seg.at(rec.instr.seg).at(rec.instr.func_id); + push_data(src_func.instruction_to_byte_in_data.at(rec.instr.instr_id + 1), out); + // offset into target + assert(rec.offset_in_segment >= 0); + push_data(rec.offset_in_segment, out); + // patch location + const auto& src_instr = src_func.instructions.at(rec.instr.instr_id); + assert(src_instr.get_disp_size() == 4); + push_data( + src_func.instruction_to_byte_in_data.at(rec.instr.instr_id) + src_instr.offset_of_disp(), + out); + } +} + +void ObjectGenerator::emit_link_table(int seg) { + emit_link_symbol(seg); + emit_link_type_pointer(seg); + emit_link_rip(seg); + m_link_by_seg.at(seg).push_back(LINK_TABLE_END); +} + /*! * Generate linker header. */ diff --git a/goalc/emitter/ObjectGenerator.h b/goalc/emitter/ObjectGenerator.h index b9b9de4a9a..8a7a70acf4 100644 --- a/goalc/emitter/ObjectGenerator.h +++ b/goalc/emitter/ObjectGenerator.h @@ -31,6 +31,8 @@ struct StaticRecord { int static_id = -1; }; +struct ObjectDebugInfo {}; + class ObjectGenerator { public: ObjectGenerator() = default; @@ -43,18 +45,32 @@ class ObjectGenerator { InstructionRecord add_instr(Instruction inst, IR_Record ir); StaticRecord add_static_to_seg(int seg, int min_align = 16); void link_instruction_jump(InstructionRecord jump_instr, IR_Record destination); - void link_instruction_symbol(); - void link_instruction_static(); - void link_instruction_to_function(); void link_static_type_ptr(StaticRecord rec, int offset, const std::string& type_name); - void link_static_sym(); - void link_type_pointer(const std::string& type_name, const StaticRecord& rec, int offset); + void link_instruction_symbol_mem(const InstructionRecord& rec, const std::string& name); + void link_instruction_symbol_ptr(const InstructionRecord& rec, const std::string& name); + void link_static_symbol_ptr(StaticRecord rec, int offset, const std::string& name); + + void link_instruction_static(const InstructionRecord& instr, + const StaticRecord& target_static, + int offset); + void link_instruction_to_function(const InstructionRecord& instr, + const FunctionRecord& target_func); + + ObjectDebugInfo create_debug_info(); private: void handle_temp_static_type_links(int seg); void handle_temp_jump_links(int seg); - void emit_link_tables(); + void handle_temp_instr_sym_links(int seg); + void handle_temp_static_sym_links(int seg); + void handle_temp_rip_data_links(int seg); + void handle_temp_rip_func_links(int seg); + + void emit_link_table(int seg); + void emit_link_type_pointer(int seg); + void emit_link_symbol(int seg); + void emit_link_rip(int seg); std::vector generate_header_v3(); template @@ -72,10 +88,6 @@ class ObjectGenerator { assert(offset + sizeof(T) <= data.size()); memcpy(data.data() + offset, &x, sizeof(T)); } - // jumps, patched before (JumpLink) - // ptrs/access to statics (RipLink) - // type pointers - // symbol links struct FunctionData { std::vector instructions; @@ -95,36 +107,64 @@ class ObjectGenerator { int offset = -1; }; - // struct SymbolLink { - // - // }; - // - // struct RipLink { - // - // }; - // + struct StaticSymbolLink { + StaticRecord rec; + int offset = -1; + }; + + struct SymbolInstrLink { + InstructionRecord rec; + bool is_mem_access = false; + }; + + struct RipFuncLink { + InstructionRecord instr; + FunctionRecord target; + }; + + struct RipDataLink { + InstructionRecord instr; + StaticRecord data; + int offset = -1; + }; + + struct RipLink { + InstructionRecord instr; + int target_segment = -1; + int offset_in_segment = -1; + }; + struct JumpLink { InstructionRecord jump_instr; IR_Record dest; }; + template + using seg_vector = std::array, N_SEG>; + + template + using seg_map = std::array>, N_SEG>; + // final data - std::array, N_SEG> m_data_by_seg; - std::array, N_SEG> m_link_by_seg; + seg_vector m_data_by_seg; + seg_vector m_link_by_seg; // temp data - std::array, N_SEG> m_function_data_by_seg; - std::array, N_SEG> m_static_data_by_seg; + seg_vector m_function_data_by_seg; + seg_vector m_static_data_by_seg; // temp link stuff - std::array>, N_SEG> - m_static_type_temp_links_by_seg; - std::array, N_SEG> m_jump_temp_links_by_seg; + seg_map m_static_type_temp_links_by_seg; + seg_vector m_jump_temp_links_by_seg; + seg_map m_symbol_instr_temp_links_by_seg; + seg_map m_static_sym_temp_links_by_seg; + seg_vector m_rip_func_temp_links_by_seg; + seg_vector m_rip_data_temp_links_by_seg; // final link stuff - std::array>, N_SEG> m_type_ptr_links_by_seg; - // std::array, N_SEG> m_sym_links_by_seg; - // std::array, N_SEG> m_rip_links_by_seg; + seg_map m_type_ptr_links_by_seg; + seg_map m_sym_links_by_seg; + seg_vector m_rip_links_by_seg; }; } // namespace emitter diff --git a/goalc/logger/CMakeLists.txt b/goalc/logger/CMakeLists.txt new file mode 100644 index 0000000000..e2e0b11dd9 --- /dev/null +++ b/goalc/logger/CMakeLists.txt @@ -0,0 +1 @@ +add_library(logger SHARED Logger.cpp) \ No newline at end of file diff --git a/goalc/logger/Logger.cpp b/goalc/logger/Logger.cpp new file mode 100644 index 0000000000..563bde1ef7 --- /dev/null +++ b/goalc/logger/Logger.cpp @@ -0,0 +1,67 @@ +#include "Logger.h" + +void Logger::close() { + if (fp) { + fclose(fp); + } +} + +void Logger::set_file(std::string filename) { + if (fp) { + fclose(fp); + } + + fp = fopen(filename.c_str(), "w"); + if (!fp) { + throw std::runtime_error("invalid file name " + filename + " in logger"); + } +} + +void Logger::log(LoggerMessageKind kind, const char* format, ...) { + FILE* dest = nullptr; + auto& settings = config[kind]; + switch (settings.kind) { + case LOG_STDERR: + dest = stderr; + break; + case LOG_STDOUT: + dest = stdout; + break; + case LOG_IGNORE: + dest = nullptr; + break; + case LOG_FILE: + dest = fp; + break; + default: + throw std::runtime_error("unknown log destination in log"); + } + + if (!dest) + return; + + if (!settings.prefix.empty()) { + fprintf(dest, "%s", settings.prefix.c_str()); + } + + if (settings.color != COLOR_NORMAL) { + const char* color_codes[] = {"", "[0;31m", "[0;32m", "[0;36m"}; + printf("\033%s", color_codes[settings.color]); + } + + va_list arglist; + va_start(arglist, format); + vfprintf(dest, format, arglist); + va_end(arglist); + + if (settings.color != COLOR_NORMAL) { + printf("\033[0m"); + } + + // todo, does this make things slow? + if (settings.kind == LOG_FILE) { + fflush(fp); + } +} + +Logger gLogger; \ No newline at end of file diff --git a/goalc/logger/Logger.h b/goalc/logger/Logger.h new file mode 100644 index 0000000000..301e8a0912 --- /dev/null +++ b/goalc/logger/Logger.h @@ -0,0 +1,43 @@ +#ifndef JAK_LOGGER_H +#define JAK_LOGGER_H + +#include +#include +#include + +enum LoggerColor { COLOR_NORMAL, COLOR_RED, COLOR_GREEN, COLOR_BLUE }; + +enum LoggerDestKind { LOG_STDOUT, LOG_STDERR, LOG_FILE, LOG_IGNORE }; + +struct LoggerDest { + LoggerDestKind kind = LOG_STDOUT; + LoggerColor color = COLOR_NORMAL; + std::string prefix; +}; + +enum LoggerMessageKind { + MSG_GOAL, + MSG_ICE, + MSG_ERR, + MSG_COLOR, + MSG_EMIT, + MSG_DEBUG, + MSG_WARN, + MSG_TGT, + MSG_TGT_INFO, +}; + +class Logger { + public: + void set_file(std::string filename); + void log(LoggerMessageKind kind, const char* format, ...); + std::unordered_map config; + void close(); + + private: + FILE* fp = nullptr; +}; + +extern Logger gLogger; + +#endif // JAK_LOGGER_H diff --git a/goalc/regalloc/IRegister.cpp b/goalc/regalloc/IRegister.cpp new file mode 100644 index 0000000000..919e856169 --- /dev/null +++ b/goalc/regalloc/IRegister.cpp @@ -0,0 +1,3 @@ + + +#include "IRegister.h" diff --git a/goalc/regalloc/IRegister.h b/goalc/regalloc/IRegister.h new file mode 100644 index 0000000000..de27b86ace --- /dev/null +++ b/goalc/regalloc/IRegister.h @@ -0,0 +1,24 @@ +/*! + * IRegister is the Register for the Intermediate Representation. + */ + +#ifndef JAK_IREGISTER_H +#define JAK_IREGISTER_H + +#include + +class IRegister { + public: + enum Kind { + GPR, + XMM_FLOAT, + XMM_INT, + XMM_VF + }; + + int id = -1; + + std::string to_string(bool with_constraints = false); +}; + +#endif // JAK_IREGISTER_H From 2eeaa70c761c4f7d96d0b9d2f7079bee561c4222 Mon Sep 17 00:00:00 2001 From: water Date: Sat, 5 Sep 2020 11:30:38 -0400 Subject: [PATCH 03/13] add analysis for register allocator --- goalc/CMakeLists.txt | 4 +- goalc/emitter/Register.cpp | 17 ++ goalc/emitter/Register.h | 18 +- goalc/regalloc/Allocator.cpp | 321 +++++++++++++++++++++++++++++++++++ goalc/regalloc/Allocator.h | 43 +++++ goalc/regalloc/Assignment.h | 48 ++++++ goalc/regalloc/IRegister.cpp | 18 +- goalc/regalloc/IRegister.h | 24 +-- goalc/regalloc/LiveInfo.h | 162 ++++++++++++++++++ goalc/regalloc/StackOp.h | 46 +++++ goalc/regalloc/allocate.cpp | 152 +++++++++++++++++ goalc/regalloc/allocate.h | 66 +++++++ test/CMakeLists.txt | 4 +- 13 files changed, 906 insertions(+), 17 deletions(-) create mode 100644 goalc/regalloc/Allocator.cpp create mode 100644 goalc/regalloc/Allocator.h create mode 100644 goalc/regalloc/Assignment.h create mode 100644 goalc/regalloc/LiveInfo.h create mode 100644 goalc/regalloc/StackOp.h create mode 100644 goalc/regalloc/allocate.cpp create mode 100644 goalc/regalloc/allocate.h diff --git a/goalc/CMakeLists.txt b/goalc/CMakeLists.txt index d932421733..e43a3d77cc 100644 --- a/goalc/CMakeLists.txt +++ b/goalc/CMakeLists.txt @@ -19,6 +19,8 @@ add_library(compiler compiler/Env.cpp compiler/Val.cpp regalloc/IRegister.cpp + regalloc/Allocator.cpp + regalloc/allocate.cpp compiler/Compiler.cpp ) @@ -29,4 +31,4 @@ IF (WIN32) set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON) ENDIF() -target_link_libraries(goalc util goos type_system compiler logger type_system) +target_link_libraries(goalc util goos type_system compiler type_system logger) diff --git a/goalc/emitter/Register.cpp b/goalc/emitter/Register.cpp index 30cac4b18f..0afeda5bda 100644 --- a/goalc/emitter/Register.cpp +++ b/goalc/emitter/Register.cpp @@ -30,4 +30,21 @@ RegisterInfo RegisterInfo::make_register_info() { return info; } +RegisterInfo gRegInfo = RegisterInfo::make_register_info(); + +std::string to_string(RegKind kind){ + switch(kind) { + case RegKind::GPR: + return "gpr"; + case RegKind::XMM: + return "xmm"; + default: + assert(false); + } +} + +std::string Register::print() const { + return gRegInfo.get_info(*this).name; +} + } // namespace emitter \ No newline at end of file diff --git a/goalc/emitter/Register.h b/goalc/emitter/Register.h index 1e46393bb3..efeccb9e0b 100644 --- a/goalc/emitter/Register.h +++ b/goalc/emitter/Register.h @@ -13,8 +13,16 @@ namespace emitter { +enum class RegKind : u8 { + GPR, + XMM, + INVALID +}; + +std::string to_string(RegKind kind); + // registers by name -enum X86_REG : u8 { +enum X86_REG : s8 { RAX, // return, temp RCX, // arg 3, temp RDX, // arg 2, temp @@ -48,7 +56,7 @@ enum X86_REG : u8 { XMM12, XMM13, XMM14, - XMM15 + XMM15, }; class Register { @@ -83,8 +91,10 @@ class Register { bool operator!=(const Register& x) const { return m_id != x.m_id; } + std::string print() const; + private: - u8 m_id = 0xff; + s8 m_id = -1; }; class RegisterInfo { @@ -129,6 +139,8 @@ class RegisterInfo { std::array m_saved_xmms; }; +extern RegisterInfo gRegInfo; + } // namespace emitter #endif // JAK_REGISTER_H diff --git a/goalc/regalloc/Allocator.cpp b/goalc/regalloc/Allocator.cpp new file mode 100644 index 0000000000..a10c620f7e --- /dev/null +++ b/goalc/regalloc/Allocator.cpp @@ -0,0 +1,321 @@ +#include +#include "Allocator.h" +#include "LiveInfo.h" + +void find_basic_blocks(RegAllocCache* cache, const AllocationInput& in) { + std::vector dividers; + + dividers.push_back(0); + dividers.push_back(in.instructions.size()); + + // loop over instructions, finding jump targets + for (uint32_t i = 0; i < in.instructions.size(); i++) { + const auto& instr = in.instructions[i]; + if (!instr.jumps.empty()) { + dividers.push_back(i + 1); + for (auto dest : instr.jumps) { + dividers.push_back(dest); + } + } + } + + // sort dividers, and make blocks + std::sort(dividers.begin(), dividers.end(), [](int a, int b) { return a < b; }); + + for (uint32_t i = 0; i < dividers.size() - 1; i++) { + if (dividers[i] != dividers[i + 1]) { + // new basic block! + RegAllocBasicBlock block; + for (int j = dividers[i]; j < dividers[i + 1]; j++) { + block.instr_idx.push_back(j); + } + + block.idx = cache->basic_blocks.size(); + cache->basic_blocks.push_back(block); + } + } + + if (!cache->basic_blocks.empty()) { + cache->basic_blocks.front().is_entry = true; + cache->basic_blocks.back().is_exit = true; + } + + auto find_basic_block_to_target = [&](int instr) { + bool found = false; + uint32_t result = -1; + for (uint32_t i = 0; i < cache->basic_blocks.size(); i++) { + if (!cache->basic_blocks[i].instr_idx.empty() && + cache->basic_blocks[i].instr_idx.front() == instr) { + assert(!found); + found = true; + result = i; + } + } + if (!found) { + printf("[RegAlloc Error] couldn't find basic block beginning with instr %d of %ld\n", instr, + in.instructions.size()); + } + assert(found); + return result; + }; + + // link blocks + for (auto& block : cache->basic_blocks) { + assert(!block.instr_idx.empty()); + auto& last_instr = in.instructions.at(block.instr_idx.back()); + if (last_instr.fallthrough) { + // try to link to next block: + int next_idx = block.idx + 1; + if (next_idx < (int)cache->basic_blocks.size()) { + cache->basic_blocks.at(next_idx).pred.push_back(block.idx); + block.succ.push_back(next_idx); + } + } + for (auto target : last_instr.jumps) { + cache->basic_blocks.at(find_basic_block_to_target(target)).pred.push_back(block.idx); + block.succ.push_back(find_basic_block_to_target(target)); + } + } +} + +namespace { +void compute_live_ranges(RegAllocCache* cache, const AllocationInput& in) { + // then resize live ranges to the correct size + cache->live_ranges.resize(cache->max_var, LiveInfo(in.instructions.size(), 0)); + + // now compute the ranges + for (auto& block : cache->basic_blocks) { + // from var use + for (auto instr_id : block.instr_idx) { + auto& inst = in.instructions.at(instr_id); + + for (auto& lst : {inst.read, inst.write}) { + for (auto& x : lst) { + cache->live_ranges.at(x.id).add_live_instruction(instr_id); + } + } + } + + // and liveliness analysis + assert(block.live.size() == block.instr_idx.size()); + for (uint32_t i = 0; i < block.live.size(); i++) { + for (auto& x : block.live[i]) { + cache->live_ranges.at(x).add_live_instruction(block.instr_idx.at(i)); + } + } + } + + // make us alive at any constrained instruction. todo, if this happens is this a sign of an issue + for (auto& con : in.constraints) { + cache->live_ranges.at(con.ireg.id).add_live_instruction(con.instr_idx); + } +} +} // namespace + +void analyze_liveliness(RegAllocCache* cache, const AllocationInput& in) { + cache->max_var = in.max_vars; + cache->was_colored.resize(cache->max_var, false); + cache->iregs.resize(cache->max_var); + + for (auto& instr : in.instructions) { + for (auto& wr : instr.write) { + cache->iregs.at(wr.id) = wr; + } + + for (auto& rd : instr.read) { + cache->iregs.at(rd.id) = rd; + } + } + + // phase 1 + for (auto& block : cache->basic_blocks) { + block.live.resize(block.instr_idx.size()); + block.dead.resize(block.instr_idx.size()); + block.analyze_liveliness_phase1(in.instructions); + } + + // phase 2 + bool changed = false; + do { + changed = false; + for (auto& block : cache->basic_blocks) { + if (block.analyze_liveliness_phase2(cache->basic_blocks, in.instructions)) { + changed = true; + } + } + } while (changed); + + // phase 3 + for (auto& block : cache->basic_blocks) { + block.analyze_liveliness_phase3(cache->basic_blocks, in.instructions); + } + + // phase 4 + compute_live_ranges(cache, in); + + // final setup! + for (size_t i = 0; i < cache->live_ranges.size(); i++) { + cache->live_ranges.at(i).prepare_for_allocation(i); + } + cache->stack_ops.resize(in.instructions.size()); +} + +namespace { +template +bool in_set(std::set& set, const T& obj) { + return set.find(obj) != set.end(); +} +} // namespace + +void RegAllocBasicBlock::analyze_liveliness_phase1(const std::vector& instructions) { + for (int i = instr_idx.size(); i-- > 0;) { + auto ii = instr_idx.at(i); + auto& instr = instructions.at(ii); + auto& lv = live.at(i); + auto& dd = dead.at(i); + + // make all read live out + lv.clear(); + for (auto& x : instr.read) { + lv.insert(x.id); + } + + // kill things which are overwritten + dd.clear(); + for (auto& x : instr.write) { + if (!in_set(lv, x.id)) { + dd.insert(x.id); + } + } + + // b.use = i.liveout + std::set use_old = use; + use.clear(); + for (auto& x : lv) { + use.insert(x); + } + // | (bu.use & !i.dead) + for (auto& x : use_old) { + if (!in_set(dd, x)) { + use.insert(x); + } + } + + // b.defs = i.dead + std::set defs_old = defs; + defs.clear(); + for (auto& x : dd) { + defs.insert(x); + } + // | b.defs & !i.lv + for (auto& x : defs_old) { + if (!in_set(lv, x)) { + defs.insert(x); + } + } + } +} + +bool RegAllocBasicBlock::analyze_liveliness_phase2(std::vector& blocks, + const std::vector& instructions) { + (void)instructions; + bool changed = false; + auto out = defs; + + for (auto s : succ) { + for (auto in : blocks.at(s).input) { + out.insert(in); + } + } + + std::set in = use; + for (auto x : out) { + if (!in_set(defs, x)) { + in.insert(x); + } + } + + if (in != input || out != output) { + changed = true; + input = in; + output = out; + } + + return changed; +} + +void RegAllocBasicBlock::analyze_liveliness_phase3(std::vector& blocks, + const std::vector& instructions) { + (void)instructions; + std::set live_local; + for (auto s : succ) { + for (auto i : blocks.at(s).input) { + live_local.insert(i); + } + } + + for (int i = instr_idx.size(); i-- > 0;) { + auto& lv = live.at(i); + auto& dd = dead.at(i); + + std::set new_live = lv; + for (auto x : live_local) { + if (!in_set(dd, x)) { + new_live.insert(x); + } + } + lv = live_local; + live_local = new_live; + } +} + +std::string RegAllocBasicBlock::print_summary() const { + std::string result = "block " + std::to_string(idx) + "\nsucc: "; + for (auto s : succ) { + result += std::to_string(s) + " "; + } + result += "\npred: "; + for (auto p : pred) { + result += std::to_string(p) + " "; + } + result += "\nuse: "; + for (auto x : use) { + result += std::to_string(x) + " "; + } + result += "\ndef: "; + for (auto x : defs) { + result += std::to_string(x) + " "; + } + result += "\ninput: "; + for (auto x : input) { + result += std::to_string(x) + " "; + } + result += "\noutput: "; + for (auto x : output) { + result += std::to_string(x) + " "; + } + + return result; +} + +std::string RegAllocBasicBlock::print(const std::vector& insts) const { + std::string result = print_summary() + "\n"; + int k = 0; + for (auto instr : instr_idx) { + std::string line = insts.at(instr).print(); + constexpr int pad_len = 30; + if (line.length() < pad_len) { + // line.insert(line.begin(), pad_len - line.length(), ' '); + line.append(pad_len - line.length(), ' '); + } + + result += " " + line + " live: "; + for (auto j : live.at(k)) { + result += std::to_string(j) + " "; + } + result += "\n"; + + k++; + } + return result; +} \ No newline at end of file diff --git a/goalc/regalloc/Allocator.h b/goalc/regalloc/Allocator.h new file mode 100644 index 0000000000..55b5ea476e --- /dev/null +++ b/goalc/regalloc/Allocator.h @@ -0,0 +1,43 @@ + + +#ifndef JAK_ALLOCATOR_H +#define JAK_ALLOCATOR_H + +#include +#include +#include "IRegister.h" +#include "allocate.h" +#include "LiveInfo.h" +#include "StackOp.h" + +struct RegAllocBasicBlock { + std::vector instr_idx; + std::vector succ; + std::vector pred; + std::vector> live, dead; + std::set use, defs, input, output; + bool is_entry = false; + bool is_exit = false; + int idx = -1; + void analyze_liveliness_phase1(const std::vector& instructions); + bool analyze_liveliness_phase2(std::vector& blocks, + const std::vector& instructions); + void analyze_liveliness_phase3(std::vector& blocks, + const std::vector& instructions); + std::string print(const std::vector& insts) const; + std::string print_summary() const; +}; + +struct RegAllocCache { + std::vector basic_blocks; + std::vector live_ranges; + int max_var = -1; + std::vector was_colored; + std::vector iregs; + std::vector stack_ops; +}; + +void find_basic_blocks(RegAllocCache* cache, const AllocationInput& in); +void analyze_liveliness(RegAllocCache* cache, const AllocationInput& in); + +#endif // JAK_ALLOCATOR_H diff --git a/goalc/regalloc/Assignment.h b/goalc/regalloc/Assignment.h new file mode 100644 index 0000000000..f4e12ac2f9 --- /dev/null +++ b/goalc/regalloc/Assignment.h @@ -0,0 +1,48 @@ +#ifndef JAK_ASSIGNMENT_H +#define JAK_ASSIGNMENT_H + +#include "third-party/fmt/core.h" +#include "goalc/emitter/Register.h" + +/*! + * The assignment of an IRegister to a real Register. + * For a single IR Instruction. + */ +struct Assignment { + enum class Kind { STACK, REGISTER, UNASSIGNED } kind = Kind::UNASSIGNED; + emitter::Register reg = -1; //! where the IRegister is now + int stack_slot = -1; //! index of the slot, if we are ever spilled + bool spilled = false; //! are we ever spilled + + std::string to_string() const { + std::string result; + if (spilled) { + result += "*"; + } + switch (kind) { + case Kind::STACK: + result += fmt::format("stack[{:2d}]", stack_slot); + break; + case Kind::REGISTER: + result += emitter::gRegInfo.get_info(reg).name; + break; + case Kind::UNASSIGNED: + result += "unassigned"; + break; + default: + assert(false); + } + + return result; + } + + bool occupies_same_reg(const Assignment& other) const { + return other.reg == reg && (reg != -1); + } + + bool is_assigned() const { + return kind != Kind::UNASSIGNED; + } +}; + +#endif // JAK_ASSIGNMENT_H diff --git a/goalc/regalloc/IRegister.cpp b/goalc/regalloc/IRegister.cpp index 919e856169..6e7a1c5809 100644 --- a/goalc/regalloc/IRegister.cpp +++ b/goalc/regalloc/IRegister.cpp @@ -1,3 +1,19 @@ +#include "third-party/fmt/core.h" +#include "IRegister.h" +std::string IRegister::to_string() const { + // if (with_constraints) { + // std::string result = fmt::format("i{}-{}\n", emitter::to_string(kind), id); + // for (const auto& x : constraints) { + // result += fmt::format(" [{:3d] in {}\n", x.instr_idx, + // emitter::gRegInfo.get_info(x.desired_register).name); + // } + // return result; + // } else { + return fmt::format("i{}-{}", emitter::to_string(kind), id); + // } +} -#include "IRegister.h" +std::string IRegConstraint::to_string() const { + return fmt::format("[{:3d}] {} in {}", instr_idx, ireg.to_string(), desired_register.print()); +} \ No newline at end of file diff --git a/goalc/regalloc/IRegister.h b/goalc/regalloc/IRegister.h index de27b86ace..71fefeea5c 100644 --- a/goalc/regalloc/IRegister.h +++ b/goalc/regalloc/IRegister.h @@ -6,19 +6,23 @@ #define JAK_IREGISTER_H #include +#include +#include "goalc/emitter/Register.h" -class IRegister { - public: - enum Kind { - GPR, - XMM_FLOAT, - XMM_INT, - XMM_VF - }; - +struct IRegister { + emitter::RegKind kind = emitter::RegKind::INVALID; int id = -1; + std::string to_string() const; + struct hash { + auto operator()(const IRegister& r) const { return std::hash()(r.id); } + }; +}; - std::string to_string(bool with_constraints = false); +struct IRegConstraint { + IRegister ireg; + int instr_idx = -1; + emitter::Register desired_register; + std::string to_string() const; }; #endif // JAK_IREGISTER_H diff --git a/goalc/regalloc/LiveInfo.h b/goalc/regalloc/LiveInfo.h new file mode 100644 index 0000000000..c652576fe3 --- /dev/null +++ b/goalc/regalloc/LiveInfo.h @@ -0,0 +1,162 @@ + + +#ifndef JAK_LIVEINFO_H +#define JAK_LIVEINFO_H + +#include +#include +#include +#include "Assignment.h" + +// with this on, gaps in usage of registers allow other variables to steal registers. +// this reduces stack spills/moves, but may make register allocation slower. +constexpr bool enable_fancy_coloring = true; + +// Indication of where a variable is live and what assignment it has at each point in the range. +struct LiveInfo { + public: + LiveInfo(int start, int end) : min(start), max(end) { } + // min, max are inclusive. + // meaning the variable written for the first time at min, and read for the last time at max. + int min, max; + + std::vector is_alive; + std::vector indices_of_alive; + + // which variable is this? + int var = -1; + + // have we actually seen this variable in the code? + bool seen = false; + + // does this variable have a constraint? + bool has_constraint = false; + + // the assignment of this variable at each instruction in [min, max] + std::vector assignment; + + // a hint on where to put this variable. + Assignment best_hint; + + /*! + * Add an instruction id where this variable is live. + */ + void add_live_instruction(int value) { + if(value > max) max = value; + if(value < min) min = value; + indices_of_alive.push_back(value); + // remember that this variable is actually used + seen = true; + } + + /*! + * Is the given instruction contained in the live range? + */ + bool is_live_at_instr(int value) { + if(value >= min && value <= max) { + if(enable_fancy_coloring) { + return is_alive.at(value - min); + } else { + return true; + } + } + return false; + } + + /*! + * Are we alive at idx, but not alive at idx - 1 (or idx - 1 doesn't exist) + */ + bool becomes_live_at_instr(int idx) { + if(enable_fancy_coloring) { + if(idx == min) return true; + if(idx < min || idx > max) return false; + assert(idx > min); + return is_alive.at(idx - min) && !is_alive.at(idx - min - 1); + } else { + return idx == min; + } + } + + /*! + * Are we alive at idx, but not alive at idx + 1 (or idx + 1 doesn't exist) + */ + bool dies_next_at_instr(int idx) { + if(enable_fancy_coloring) { + if(idx == max) return true; + if(idx < min || idx > max) return false; + assert(idx < max); + return is_alive.at(idx - min) && !is_alive.at(idx - min + 1); + } else { + return idx == max; + } + } + + /*! + * Resize Live Range after instructions have been added. Do this before assigning. + */ + void prepare_for_allocation(int id) { + var = id; + if(!seen) return; // don't do any prep for a variable which isn't used. + assert(max - min >= 0); + assignment.resize(max - min + 1); + is_alive.resize(max - min + 1); + for(auto& x : indices_of_alive) { + is_alive.at(x - min) = true; + } + } + + /*! + * Lock an assignment at a given instruction. + * Will overwrite any previous assignment here + * Will set best_hint to this assignment. + */ + void constrain_at_one(int id, Assignment ass) { + assert(id >= min && id <= max); + assignment.at(id - min) = ass; + has_constraint = true; + best_hint = ass; + } + + /*! + * At the given instruction, does the given assignment conflict with this one? + */ + bool conflicts_at(int id, Assignment ass) { + assert(id >= min && id <= max); + return assignment.at(id - min).occupies_same_reg(ass); + } + + /*! + * Assign variable to the given assignment at all instructions + * Throws if this would require modifying a currently set assignment. + */ + void assign_no_overwrite(Assignment ass) { + assert(seen); + assert(ass.is_assigned()); + for(int i = min; i <= max; i++) { + auto& a = assignment.at(i - min); + if(a.is_assigned() && !(a.occupies_same_reg(ass))) { + throw std::runtime_error("assign_no_overwrite failed!"); + } else { + a = ass; + } + } + } + + /*! + * Get the assignment at the given instruction. + */ + const Assignment& get(int id) { + assert(id >= min && id <= max); + return assignment.at(id - min); + } + + std::string print_assignment() { + std::string result = "Assignment for var " + std::to_string(var) + "\n"; + for(uint32_t i = 0; i < assignment.size(); i++) { + result += fmt::format("i[{:3d}] {}\n", i+min, assignment.at(i).to_string()); + } + return result; + } +}; + +#endif // JAK_LIVEINFO_H diff --git a/goalc/regalloc/StackOp.h b/goalc/regalloc/StackOp.h new file mode 100644 index 0000000000..90e21c82ed --- /dev/null +++ b/goalc/regalloc/StackOp.h @@ -0,0 +1,46 @@ +/*! + * @file StackOp.h + * An operation that's added to an Instruction so that it loads/stores things from the stack + */ + +#ifndef JAK_STACKOP_H +#define JAK_STACKOP_H + +#include +#include "third-party/fmt/core.h" +#include "goalc/emitter/Register.h" + +struct StackOp { + struct Op { + int slot = -1; + emitter::Register reg; + bool load = false; // load from reg before instruction? + bool store = false; // store into reg after instruction? + }; + + std::vector ops; + + std::string print() { + std::string result; + bool added = false; + for (const auto& op : ops) { + if (op.load) { + result += fmt::format("{} <- [{:2d}], ", emitter::gRegInfo.get_info(op.reg).name, op.slot); + added = true; + } + if (op.store) { + result += fmt::format("{} -> [{:2d}], ", emitter::gRegInfo.get_info(op.reg).name, op.slot); + added = true; + } + } + + if (added) { + result.pop_back(); + result.pop_back(); + } + + return result; + } +}; + +#endif // JAK_STACKOP_H diff --git a/goalc/regalloc/allocate.cpp b/goalc/regalloc/allocate.cpp new file mode 100644 index 0000000000..f902ed09c8 --- /dev/null +++ b/goalc/regalloc/allocate.cpp @@ -0,0 +1,152 @@ +#include "third-party/fmt/core.h" +#include "allocate.h" +#include "Allocator.h" + +namespace { +void print_allocate_input(const AllocationInput& in) { + fmt::print("[RegAlloc] Debug Input:\n"); + if (in.instructions.size() == in.debug_instruction_names.size()) { + for (size_t i = 0; i < in.instructions.size(); i++) { + fmt::print(" [{:3d}] {:30} -> {:30}\n", in.debug_instruction_names.at(i), + in.instructions.at(i).print()); + } + } else { + for (const auto& instruction : in.instructions) { + fmt::print(" [{:3d}] {}\n", instruction.print()); + } + } + for (const auto& c : in.constraints) { + fmt::print(" {}\n", c.to_string()); + } + + fmt::print(" aligned_stack? {}\n", in.requires_aligned_stack); + fmt::print("\n"); +} + +void print_analysis(const AllocationInput& in, RegAllocCache* cache) { + fmt::print("[RegAlloc] Basic Blocks\n"); + fmt::print("-----------------------------------------------------------------\n"); + for(auto& b : cache->basic_blocks) { + fmt::print("{}\n", b.print(in.instructions)); + } + + printf("[RegAlloc] Alive Info\n"); + printf("-----------------------------------------------------------------\n"); + // align to where we start putting live stuff + printf(" %30s ", ""); + for(int i = 0; i < cache->max_var; i++) { + printf("%2d ", i); + } + printf("\n"); + printf("_________________________________________________________________\n"); + for(uint32_t i = 0; i < in.instructions.size(); i++) { + std::vector ids_live; + std::string lives; + + ids_live.resize(cache->max_var, false); + + for(int j = 0; j < cache->max_var; j++) { + if(cache->live_ranges.at(j).is_live_at_instr(i)) { + ids_live.at(j) = true; + } + } + + for(uint32_t j = 0; j < ids_live.size(); j++) { + if(ids_live[j]) { + char buff[256]; + sprintf(buff, "%2d ", j); + lives.append(buff); + } else { + lives.append(".. "); + } + } + + if(in.debug_instruction_names.size() == in.instructions.size()) { + std::string code_str = in.debug_instruction_names.at(i); + if(code_str.length() >= 50) { + code_str = code_str.substr(0, 48); + code_str.push_back('~'); + } + printf("[%03d] %30s -> %s\n", i, code_str.c_str(), lives.c_str()); + } else { + printf("[%03d] %30s -> %s\n", i, "???", lives.c_str()); + } + + } +} +} // namespace + +/*! + * The top-level register allocation algorithm! + */ +AllocationResult allocate_registers(const AllocationInput& input) { + AllocationResult result; + RegAllocCache cache; + + // if desired, print input for debugging. + if (input.debug_settings.print_input) { + print_allocate_input(input); + } + + // first step is analysis + find_basic_blocks(&cache, input); + analyze_liveliness(&cache, input); + if(input.debug_settings.print_analysis) { + print_analysis(input, &cache); + } + + + return result; +} + +std::string RegAllocInstr::print() const { + bool first = true; + std::string result = "("; + + if (!write.empty()) { + first = false; + result += "(write"; + for (auto& i : write) { + result += " " + i.to_string(); + } + result += ")"; + } + + if (!read.empty()) { + if (!first) { + result += " "; + } + first = false; + result += "(read"; + for (auto& i : read) { + result += " " + i.to_string(); + } + result += ")"; + } + + if (!clobber.empty()) { + if (!first) { + result += " "; + } + first = false; + result += "(clobber"; + for (auto& i : clobber) { + result += " " + i.print(); + } + result += ")"; + } + + if (!jumps.empty()) { + if (!first) { + result += " "; + } + first = false; + result += "(jumps"; + for (auto& i : jumps) { + result += " " + std::to_string(i); + } + result += ")"; + } + result += ")"; + return result; +} \ No newline at end of file diff --git a/goalc/regalloc/allocate.h b/goalc/regalloc/allocate.h new file mode 100644 index 0000000000..cd13e43e9f --- /dev/null +++ b/goalc/regalloc/allocate.h @@ -0,0 +1,66 @@ +/*! + * @file allocate.h + * Interface for the register allocator. + * + * The IR is translated to RegAllocInstrs, which are added to a RegAllocFunc. + * These, plus any additional info are put in a AllocationInput, which is processed by the + * allocate_registers algorithm. + */ + +#ifndef JAK_ALLOCATE_H +#define JAK_ALLOCATE_H + +#include +#include "goalc/emitter/Register.h" +#include "IRegister.h" + +/*! + * Information about an instruction needed for register allocation. + * The model is this: + * instruction reads all read registers + * instruction writes junk into all clobber registers + * instruction writes all write registers + * + * The "exclude" registers cannot be used at any time during this instruction, for any reason. + * Possibly because the actual implementation requires using it. + */ +struct RegAllocInstr { + std::vector clobber; + std::vector exclude; + std::vector write; + std::vector read; + std::vector jumps; + bool fallthrough = true; + bool is_move = false; + std::string print() const; +}; + + +struct AllocationResult { + bool ok = false; +}; + +struct AllocationInput { + std::vector instructions; + std::vector constraints; + bool requires_aligned_stack = false; + int max_vars = -1; + + std::vector debug_instruction_names; // optional + + + + int add_instruction(const RegAllocInstr& instr) { + instructions.push_back(instr); + return int(instructions.size()) - 1; + } + + struct { + bool print_input = false; + bool print_analysis = false; + } debug_settings; +}; + +AllocationResult allocate_registers(const AllocationInput& input); + +#endif // JAK_ALLOCATE_H diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt index 8173c9820c..656c7fe8a2 100644 --- a/test/CMakeLists.txt +++ b/test/CMakeLists.txt @@ -20,7 +20,7 @@ IF (WIN32) set(gtest_force_shared_crt ON CACHE BOOL "" FORCE) # TODO - implement windows listener message("Windows Listener Not Implemented!") - target_link_libraries(goalc-test mman goos util runtime compiler type_system gtest) + target_link_libraries(goalc-test mman goos util runtime compiler type_system gtest logger) ELSE() - target_link_libraries(goalc-test goos util listener runtime compiler type_system gtest) + target_link_libraries(goalc-test goos util listener runtime compiler type_system gtest logger) ENDIF() From c304fabf72a40a7ad3fd1fdbbd4ee43b66740836 Mon Sep 17 00:00:00 2001 From: water Date: Sat, 5 Sep 2020 13:49:23 -0400 Subject: [PATCH 04/13] add register allocator --- goalc/compiler/Compiler.cpp | 4 +- goalc/compiler/Compiler.h | 1 + goalc/emitter/Register.cpp | 20 +- goalc/emitter/Register.h | 22 +- goalc/regalloc/Allocator.cpp | 549 +++++++++++++++++++++++++++++++++++ goalc/regalloc/Allocator.h | 7 + goalc/regalloc/Assignment.h | 10 +- goalc/regalloc/LiveInfo.h | 64 ++-- goalc/regalloc/StackOp.h | 5 +- goalc/regalloc/allocate.cpp | 125 +++++++- goalc/regalloc/allocate.h | 80 +++-- 11 files changed, 817 insertions(+), 70 deletions(-) diff --git a/goalc/compiler/Compiler.cpp b/goalc/compiler/Compiler.cpp index 49e4b2aae1..e1353df059 100644 --- a/goalc/compiler/Compiler.cpp +++ b/goalc/compiler/Compiler.cpp @@ -6,9 +6,7 @@ Compiler::Compiler() { m_ts.add_builtin_types(); } -void Compiler::execute_repl() { - -} +void Compiler::execute_repl() {} Compiler::~Compiler() { gLogger.close(); diff --git a/goalc/compiler/Compiler.h b/goalc/compiler/Compiler.h index 3dd5a4904b..12781a408d 100644 --- a/goalc/compiler/Compiler.h +++ b/goalc/compiler/Compiler.h @@ -8,6 +8,7 @@ class Compiler { Compiler(); ~Compiler(); void execute_repl(); + private: void init_logger(); diff --git a/goalc/emitter/Register.cpp b/goalc/emitter/Register.cpp index 0afeda5bda..55ba17e9bd 100644 --- a/goalc/emitter/Register.cpp +++ b/goalc/emitter/Register.cpp @@ -27,13 +27,29 @@ RegisterInfo RegisterInfo::make_register_info() { info.m_saved_xmms = std::array({XMM8, XMM9, XMM10, XMM11, XMM12, XMM13, XMM14, XMM15}); + for (size_t i = 0; i < N_SAVED_GPRS; i++) { + info.m_saved_all[i] = info.m_saved_gprs[i]; + } + for (size_t i = 0; i < N_SAVED_XMMS; i++) { + info.m_saved_all[i + N_SAVED_GPRS] = info.m_saved_xmms[i]; + } + + // todo - experiment with better orders for allocation. + info.m_gpr_alloc_order = {RAX, RCX, RDX, RBX, RBP, RSI, RDI, R8, R9, R10, R11}; // arbitrary + info.m_xmm_alloc_order = {XMM0, XMM1, XMM2, XMM3, XMM4, XMM5, XMM6, XMM7, + XMM8, XMM9, XMM10, XMM11, XMM12, XMM13, XMM14}; + + info.m_gpr_spill_temp_alloc_order = {RAX, RCX, RDX, RBX, RBP, RSI, + RDI, R8, R9, R10, R11, R12}; // arbitrary + info.m_xmm_spill_temp_alloc_order = {XMM0, XMM1, XMM2, XMM3, XMM4, XMM5, XMM6, XMM7, + XMM8, XMM9, XMM10, XMM11, XMM12, XMM13, XMM14, XMM15}; return info; } RegisterInfo gRegInfo = RegisterInfo::make_register_info(); -std::string to_string(RegKind kind){ - switch(kind) { +std::string to_string(RegKind kind) { + switch (kind) { case RegKind::GPR: return "gpr"; case RegKind::XMM: diff --git a/goalc/emitter/Register.h b/goalc/emitter/Register.h index efeccb9e0b..de087b1fad 100644 --- a/goalc/emitter/Register.h +++ b/goalc/emitter/Register.h @@ -9,15 +9,12 @@ #include #include #include +#include #include "common/common_types.h" namespace emitter { -enum class RegKind : u8 { - GPR, - XMM, - INVALID -}; +enum class RegKind : u8 { GPR, XMM, INVALID }; std::string to_string(RegKind kind); @@ -131,12 +128,27 @@ class RegisterInfo { Register get_ret_reg() const { return RAX; } + const std::vector& get_gpr_alloc_order() { return m_gpr_alloc_order; } + + const std::vector& get_xmm_alloc_order() { return m_xmm_alloc_order; } + + const std::vector& get_gpr_spill_alloc_order() { return m_gpr_spill_temp_alloc_order; } + + const std::vector& get_xmm_spill_alloc_order() { return m_xmm_spill_temp_alloc_order; } + + const std::array& get_all_saved() { return m_saved_all; } + private: RegisterInfo() = default; std::array m_info; std::array m_arg_regs; std::array m_saved_gprs; std::array m_saved_xmms; + std::array m_saved_all; + std::vector m_gpr_alloc_order; + std::vector m_xmm_alloc_order; + std::vector m_gpr_spill_temp_alloc_order; + std::vector m_xmm_spill_temp_alloc_order; }; extern RegisterInfo gRegInfo; diff --git a/goalc/regalloc/Allocator.cpp b/goalc/regalloc/Allocator.cpp index a10c620f7e..1b9cfdde64 100644 --- a/goalc/regalloc/Allocator.cpp +++ b/goalc/regalloc/Allocator.cpp @@ -1,7 +1,15 @@ +/*! + * @file Allocator.cpp + * Implementation of register allocation algorithms + */ + #include #include "Allocator.h" #include "LiveInfo.h" +/*! + * Find basic blocks and add block link info. + */ void find_basic_blocks(RegAllocCache* cache, const AllocationInput& in) { std::vector dividers; @@ -79,6 +87,10 @@ void find_basic_blocks(RegAllocCache* cache, const AllocationInput& in) { } namespace { + +/*! + * Setup live_ranges. Must have found where iregs are live first. + */ void compute_live_ranges(RegAllocCache* cache, const AllocationInput& in) { // then resize live ranges to the correct size cache->live_ranges.resize(cache->max_var, LiveInfo(in.instructions.size(), 0)); @@ -112,6 +124,9 @@ void compute_live_ranges(RegAllocCache* cache, const AllocationInput& in) { } } // namespace +/*! + * Analysis pass to find out where registers are live. Must have found basic blocks first. + */ void analyze_liveliness(RegAllocCache* cache, const AllocationInput& in) { cache->max_var = in.max_vars; cache->was_colored.resize(cache->max_var, false); @@ -318,4 +333,538 @@ std::string RegAllocBasicBlock::print(const std::vector& insts) c k++; } return result; +} + +/*! + * Assign registers which are constrained. If constraints are inconsistent, won't succeed in + * satisfying them (of course), and won't error either. Use check_constrained_alloc to confirm + * that all constraints are then satisfied. + */ +void do_constrained_alloc(RegAllocCache* cache, const AllocationInput& in, bool trace_debug) { + for (auto& constr : in.constraints) { + auto var_id = constr.ireg.id; + if (trace_debug) { + fmt::print("[RA] Apply constraint {}\n", constr.to_string()); + } + cache->live_ranges.at(var_id).constrain_at_one(constr.instr_idx, constr.desired_register); + } +} + +/*! + * Check to run after do_constrained_alloc to see if the constraints could actually be satisfied. + */ +bool check_constrained_alloc(RegAllocCache* cache, const AllocationInput& in) { + bool ok = true; + for (auto& constr : in.constraints) { + if (!cache->live_ranges.at(constr.ireg.id) + .conflicts_at(constr.instr_idx, constr.desired_register)) { + fmt::print("[RegAlloc Error] There are conflicting constraints on {}: {} and {}\n", + constr.ireg.to_string(), constr.desired_register.print(), + cache->live_ranges.at(constr.ireg.id).get(constr.instr_idx).to_string()); + ok = false; + } + } + + for (uint32_t i = 0; i < in.instructions.size(); i++) { + for (auto& lr1 : cache->live_ranges) { + if (!lr1.seen || !lr1.is_live_at_instr(i)) + continue; + for (auto& lr2 : cache->live_ranges) { + if (!lr2.seen || !lr2.is_live_at_instr(i) || (&lr1 == &lr2)) + continue; + // if lr1 is assigned... + auto& ass1 = lr1.get(i); + if (ass1.kind != Assignment::Kind::UNASSIGNED) { + auto& ass2 = lr2.get(i); + if (ass1.occupies_same_reg(ass2)) { + // todo, this error won't be helpful + fmt::print( + "[RegAlloc Error] Cannot satisfy constraints at instruction {} due to constraints " + "on {} and {}\n", + i, lr1.var, lr2.var); + ok = false; + } + } + } + } + } + return ok; +} + +namespace { + +/*! + * Assign variable to register. Don't check if its safe. If it's already assigned, and this would + * change that assignment, throw. + */ +void assign_var_no_check(int var, Assignment ass, RegAllocCache* cache) { + cache->live_ranges.at(var).assign_no_overwrite(ass); +} + +/*! + * Can var be assigned to ass? + */ +bool can_var_be_assigned(int var, + Assignment ass, + RegAllocCache* cache, + const AllocationInput& in, + int debug_trace) { + // our live range: + auto& lr = cache->live_ranges.at(var); + // check against all other live ranges: + for (auto& other_lr : cache->live_ranges) { + if (other_lr.var == var /*|| !other_lr.seen*/) + continue; // but not us! + for (int instr = lr.min; instr <= lr.max; instr++) { + if (other_lr.is_live_at_instr(instr)) { + // LR's overlap + if (/*(instr != other_lr.max) && */ other_lr.conflicts_at(instr, ass)) { + bool allowed_by_move_eliminator = false; + if (move_eliminator) { + if (enable_fancy_coloring) { + if (lr.dies_next_at_instr(instr) && other_lr.becomes_live_at_instr(instr) && + in.instructions.at(instr).is_move) { + allowed_by_move_eliminator = true; + } + + if (lr.becomes_live_at_instr(instr) && other_lr.dies_next_at_instr(instr) && + in.instructions.at(instr).is_move) { + allowed_by_move_eliminator = true; + } + } else { + // case to allow rename (from us to them) + if (instr == lr.max && instr == other_lr.min && in.instructions.at(instr).is_move) { + allowed_by_move_eliminator = true; + } + + if (instr == lr.min && instr == other_lr.min && in.instructions.at(instr).is_move) { + allowed_by_move_eliminator = true; + } + } + } + + if (!allowed_by_move_eliminator) { + if (debug_trace >= 2) { + printf("at idx %d, %s conflicts\n", instr, other_lr.print_assignment().c_str()); + } + + return false; + } + } + } + } + } + + // can clobber on the last one or first one - check that we don't interfere with a clobber + for (int instr = lr.min + 1; instr <= lr.max - 1; instr++) { + for (auto clobber : in.instructions.at(instr).clobber) { + if (ass.occupies_reg(clobber)) { + if (debug_trace >= 2) { + printf("at idx %d clobber\n", instr); + } + + return false; + } + } + } + + for (int instr = lr.min; instr <= lr.max; instr++) { + for (auto exclusive : in.instructions.at(instr).exclude) { + if (ass.occupies_reg(exclusive)) { + if (debug_trace >= 2) { + printf("at idx %d exclusive conflict\n", instr); + } + + return false; + } + } + } + + // check we don't violate any others. + for (int instr = lr.min; instr <= lr.max; instr++) { + if (lr.has_constraint && lr.assignment.at(instr - lr.min).is_assigned()) { + if (!(ass.occupies_same_reg(lr.assignment.at(instr - lr.min)))) { + if (debug_trace >= 2) { + printf("at idx %d self bad\n", instr); + } + + return false; + } + } + } + + return true; +} + +bool assignment_ok_at(int var, + int idx, + Assignment ass, + RegAllocCache* cache, + const AllocationInput& in, + int debug_trace) { + auto& lr = cache->live_ranges.at(var); + for (auto& other_lr : cache->live_ranges) { + if (other_lr.var == var /*|| !other_lr.seen*/) + continue; + if (other_lr.is_live_at_instr(idx)) { + if (/*(idx != other_lr.max) &&*/ other_lr.conflicts_at(idx, ass)) { + bool allowed_by_move_eliminator = false; + if (move_eliminator) { + if (enable_fancy_coloring) { + if (lr.dies_next_at_instr(idx) && other_lr.becomes_live_at_instr(idx) && + in.instructions.at(idx).is_move) { + allowed_by_move_eliminator = true; + } + + if (lr.becomes_live_at_instr(idx) && other_lr.dies_next_at_instr(idx) && + in.instructions.at(idx).is_move) { + allowed_by_move_eliminator = true; + } + } else { + // case to allow rename (from us to them) + if (idx == lr.max && idx == other_lr.min && in.instructions.at(idx).is_move) { + allowed_by_move_eliminator = true; + } + + if (idx == lr.min && idx == other_lr.min && in.instructions.at(idx).is_move) { + allowed_by_move_eliminator = true; + } + } + } + + if (!allowed_by_move_eliminator) { + if (debug_trace >= 2) { + printf("at idx %d, %s conflicts\n", idx, other_lr.print_assignment().c_str()); + } + return false; + } + } + } + } + + // check we aren't violating a clobber + if (idx != lr.min && idx != lr.max) { + for (auto clobber : in.instructions.at(idx).clobber) { + if (ass.occupies_reg(clobber)) { + if (debug_trace >= 2) { + printf("at idx %d clobber\n", idx); + } + + return false; + } + } + } + + for (auto exclusive : in.instructions.at(idx).exclude) { + if (ass.occupies_reg(exclusive)) { + if (debug_trace >= 2) { + printf("at idx %d exclusive conflict\n", idx); + } + + return false; + } + } + + // check we aren't violating ourselves + if (lr.assignment.at(idx - lr.min).is_assigned()) { + if (!(ass.occupies_same_reg(lr.assignment.at(idx - lr.min)))) { + if (debug_trace >= 2) { + printf("at idx %d self bad\n", idx); + } + + return false; + } + } + + return true; +} + +bool try_assignment_for_var(int var, + Assignment ass, + RegAllocCache* cache, + const AllocationInput& in, + int debug_trace) { + if (can_var_be_assigned(var, ass, cache, in, debug_trace)) { + assign_var_no_check(var, ass, cache); + return true; + } + return false; +} + +int get_stack_slot_for_var(int var, RegAllocCache* cache) { + auto kv = cache->var_to_stack_slot.find(var); + if (kv == cache->var_to_stack_slot.end()) { + auto slot = cache->current_stack_slot++; + cache->var_to_stack_slot[var] = slot; + return slot; + } else { + return kv->second; + } +} + +const std::vector& get_default_alloc_order_for_var_spill(int v, + RegAllocCache* cache) { + auto& info = cache->iregs.at(v); + assert(info.kind != emitter::RegKind::INVALID); + if (info.kind == emitter::RegKind::GPR) { + return emitter::gRegInfo.get_gpr_spill_alloc_order(); + } else if (info.kind == emitter::RegKind::XMM) { + return emitter::gRegInfo.get_xmm_spill_alloc_order(); + } else { + assert(false); + } +} + +const std::vector& get_default_alloc_order_for_var(int v, RegAllocCache* cache) { + auto& info = cache->iregs.at(v); + assert(info.kind != emitter::RegKind::INVALID); + if (info.kind == emitter::RegKind::GPR) { + return emitter::gRegInfo.get_gpr_alloc_order(); + } else if (info.kind == emitter::RegKind::XMM) { + return emitter::gRegInfo.get_xmm_alloc_order(); + } else { + assert(false); + } +} + +bool try_spill_coloring(int var, RegAllocCache* cache, const AllocationInput& in, int debug_trace) { + // todo, reject flagged "unspillables" + if (debug_trace >= 1) { + printf("---- SPILL VAR %d ----\n", var); + } + + auto& lr = cache->live_ranges.at(var); + + // possibly get a hint assignment + Assignment hint_assignment; + hint_assignment.kind = Assignment::Kind::UNASSIGNED; + + // loop over live range + for (int instr = lr.min; instr <= lr.max; instr++) { + // bonus_instructions.at(instr).clear(); + StackOp::Op bonus; + + // we may have a constaint in here + auto& current_assignment = lr.assignment.at(instr - lr.min); + + auto& op = in.instructions.at(instr); + bool is_read = op.reads(var); + bool is_written = op.writes(var); + + // we have a constraint! + if (current_assignment.is_assigned()) { + if (debug_trace >= 2) { + printf(" [%02d] already assigned %s\n", instr, current_assignment.to_string().c_str()); + } + + // remember this assignment as a hint for later + hint_assignment = current_assignment; + // check that this assignment is ok + if (!assignment_ok_at(var, instr, current_assignment, cache, in, debug_trace)) { + // this shouldn't be possible with feasible constraints + printf("-- SPILL FAILED -- IMPOSSIBLE CONSTRAINT @ %d %s. This is likely a RegAlloc bug!\n", + instr, current_assignment.to_string().c_str()); + assert(false); + return false; + } + + // flag it as spilled, but currently in a GPR. + current_assignment.spilled = true; + bonus.reg = current_assignment.reg; + } else { + // not assigned. + if (debug_trace >= 1) { + printf(" [%02d] nya rd? %d wr? %d\n", instr, is_read, is_written); + } + + // We'd like to keep it on the stack if possible + Assignment spill_assignment; + spill_assignment.spilled = true; + spill_assignment.kind = Assignment::Kind::STACK; + spill_assignment.reg = -1; // for now + + // needs a temp register + if (is_read || is_written) { + // we need to put it in a register here! + // first check if the hint works? + // todo floats? + if (hint_assignment.kind == Assignment::Kind::REGISTER) { + if (debug_trace >= 2) { + printf(" try hint %s\n", hint_assignment.to_string().c_str()); + } + + if (assignment_ok_at(var, instr, hint_assignment, cache, in, debug_trace)) { + // it's ok! + if (debug_trace >= 2) { + printf(" it worked!\n"); + } + spill_assignment.reg = hint_assignment.reg; + } + } + + // hint didn't work + // auto reg_order = get_default_reg_alloc_order(); + auto reg_order = get_default_alloc_order_for_var_spill(var, cache); + if (spill_assignment.reg == -1) { + for (auto reg : reg_order) { + Assignment ass; + ass.kind = Assignment::Kind::REGISTER; + ass.reg = reg; + if (debug_trace >= 2) { + printf(" try %s\n", ass.to_string().c_str()); + } + + if (assignment_ok_at(var, instr, ass, cache, in, debug_trace)) { + if (debug_trace >= 2) { + printf(" it worked!\n"); + } + spill_assignment.reg = ass.reg; + break; + } + } + } + + if (spill_assignment.reg == -1) { + printf("SPILLING FAILED BECAUSE WE COULDN'T FIND A TEMP REGISTER!\n"); + assert(false); + // std::vector can_try_spilling; + // for(uint32_t other_spill = 0; other_spill < was_colored.size(); other_spill++) + // { + // if((int)other_spill != var && was_colored.at(other_spill)) { + // LOG("TRY SPILL %d?\n", other_spill); + // if(try_spill_coloring(other_spill)) { + // LOG("SPILL OK.\n"); + // if(try_spill_coloring(var)) { + // return true; + // } + // } else { + // LOG("SPILL %d failed.\n", other_spill); + // } + // } + // } + return false; + } + + // mark that it's in a GPR! + spill_assignment.kind = Assignment::Kind::REGISTER; + } // end need temp reg + spill_assignment.stack_slot = get_stack_slot_for_var(var, cache); + lr.assignment.at(instr - lr.min) = spill_assignment; + bonus.reg = spill_assignment.reg; + bonus.slot = spill_assignment.stack_slot; + } // end not constrained + + bonus.slot = get_stack_slot_for_var(var, cache); + bonus.load = is_read; + bonus.store = is_written; + cache->stack_ops.at(instr).ops.push_back(bonus); + } + return true; +} + +template +bool in_vec(const std::vector& vec, const T& obj) { + for (const auto& x : vec) { + if (x == obj) + return true; + } + return false; +} + +bool do_allocation_for_var(int var, + RegAllocCache* cache, + const AllocationInput& in, + int debug_trace) { + // first, let's see if there's a hint... + auto& lr = cache->live_ranges.at(var); + bool colored = false; + if (lr.best_hint.is_assigned()) { + colored = try_assignment_for_var(var, lr.best_hint, cache, in, debug_trace); + if (debug_trace >= 2) { + printf("var %d reg %s ? %d\n", var, lr.best_hint.to_string().c_str(), colored); + } + } + + auto reg_order = get_default_alloc_order_for_var(var, cache); + + // todo, try other regs.. + if (!colored && move_eliminator) { + auto& first_instr = in.instructions.at(lr.min); + auto& last_instr = in.instructions.at(lr.max); + + if (first_instr.is_move) { + auto& possible_coloring = cache->live_ranges.at(first_instr.read.front().id).get(lr.min); + if (possible_coloring.is_assigned() && in_vec(reg_order, possible_coloring.reg)) { + colored = try_assignment_for_var(var, possible_coloring, cache, in, debug_trace); + } + } + + if (!colored && last_instr.is_move) { + auto& possible_coloring = cache->live_ranges.at(last_instr.write.front().id).get(lr.max); + if (possible_coloring.is_assigned() && in_vec(reg_order, possible_coloring.reg)) { + colored = try_assignment_for_var(var, possible_coloring, cache, in, debug_trace); + } + } + } + + // auto reg_order = get_default_reg_alloc_order(); + + for (auto reg : reg_order) { + if (colored) + break; + Assignment ass; + ass.kind = Assignment::Kind::REGISTER; + ass.reg = reg; + colored = try_assignment_for_var(var, ass, cache, in, debug_trace); + if (debug_trace >= 1) { + printf("var %d reg %s ? %d\n", var, ass.to_string().c_str(), colored); + } + } + + if (!colored) { + colored = try_spill_coloring(var, cache, in, debug_trace); + if (colored) { + cache->used_stack = true; + } + } + + // todo, try spilling + if (!colored) { + printf("[ERROR] var %d could not be colored:\n%s\n", var, + cache->live_ranges.at(var).print_assignment().c_str()); + + return false; + } else { + if (debug_trace >= 2) { + printf("Colored var %d\n", var); + } + + cache->was_colored.at(var) = true; + return true; + } +} + +} // namespace + +bool run_allocator(RegAllocCache* cache, const AllocationInput& in, int debug_trace) { + // here we allocate + std::vector allocation_order; + for (uint32_t i = 0; i < cache->live_ranges.size(); i++) { + if (cache->live_ranges.at(i).seen && cache->live_ranges.at(i).has_constraint) { + allocation_order.push_back(i); + } + } + + for (uint32_t i = 0; i < cache->live_ranges.size(); i++) { + if (cache->live_ranges.at(i).seen && !cache->live_ranges.at(i).has_constraint) { + allocation_order.push_back(i); + } + } + + for (int var : allocation_order) { + if (!do_allocation_for_var(var, cache, in, debug_trace)) { + return false; + } + } + return true; } \ No newline at end of file diff --git a/goalc/regalloc/Allocator.h b/goalc/regalloc/Allocator.h index 55b5ea476e..f6d0540a22 100644 --- a/goalc/regalloc/Allocator.h +++ b/goalc/regalloc/Allocator.h @@ -5,6 +5,7 @@ #include #include +#include #include "IRegister.h" #include "allocate.h" #include "LiveInfo.h" @@ -35,9 +36,15 @@ struct RegAllocCache { std::vector was_colored; std::vector iregs; std::vector stack_ops; + std::unordered_map var_to_stack_slot; + int current_stack_slot = 0; + bool used_stack = false; }; void find_basic_blocks(RegAllocCache* cache, const AllocationInput& in); void analyze_liveliness(RegAllocCache* cache, const AllocationInput& in); +void do_constrained_alloc(RegAllocCache* cache, const AllocationInput& in, bool trace_debug); +bool check_constrained_alloc(RegAllocCache* cache, const AllocationInput& in); +bool run_allocator(RegAllocCache* cache, const AllocationInput& in, int debug_trace); #endif // JAK_ALLOCATOR_H diff --git a/goalc/regalloc/Assignment.h b/goalc/regalloc/Assignment.h index f4e12ac2f9..9752d6b073 100644 --- a/goalc/regalloc/Assignment.h +++ b/goalc/regalloc/Assignment.h @@ -36,13 +36,11 @@ struct Assignment { return result; } - bool occupies_same_reg(const Assignment& other) const { - return other.reg == reg && (reg != -1); - } + bool occupies_same_reg(const Assignment& other) const { return other.reg == reg && (reg != -1); } - bool is_assigned() const { - return kind != Kind::UNASSIGNED; - } + bool occupies_reg(emitter::Register other_reg) const { return reg == other_reg && (reg != -1); } + + bool is_assigned() const { return kind != Kind::UNASSIGNED; } }; #endif // JAK_ASSIGNMENT_H diff --git a/goalc/regalloc/LiveInfo.h b/goalc/regalloc/LiveInfo.h index c652576fe3..588a97304e 100644 --- a/goalc/regalloc/LiveInfo.h +++ b/goalc/regalloc/LiveInfo.h @@ -12,10 +12,13 @@ // this reduces stack spills/moves, but may make register allocation slower. constexpr bool enable_fancy_coloring = true; +// will attempt to allocate in a way to reduce the number of moves. +constexpr bool move_eliminator = true; + // Indication of where a variable is live and what assignment it has at each point in the range. struct LiveInfo { public: - LiveInfo(int start, int end) : min(start), max(end) { } + LiveInfo(int start, int end) : min(start), max(end) {} // min, max are inclusive. // meaning the variable written for the first time at min, and read for the last time at max. int min, max; @@ -42,8 +45,10 @@ struct LiveInfo { * Add an instruction id where this variable is live. */ void add_live_instruction(int value) { - if(value > max) max = value; - if(value < min) min = value; + if (value > max) + max = value; + if (value < min) + min = value; indices_of_alive.push_back(value); // remember that this variable is actually used seen = true; @@ -52,9 +57,9 @@ struct LiveInfo { /*! * Is the given instruction contained in the live range? */ - bool is_live_at_instr(int value) { - if(value >= min && value <= max) { - if(enable_fancy_coloring) { + bool is_live_at_instr(int value) const { + if (value >= min && value <= max) { + if (enable_fancy_coloring) { return is_alive.at(value - min); } else { return true; @@ -67,9 +72,11 @@ struct LiveInfo { * Are we alive at idx, but not alive at idx - 1 (or idx - 1 doesn't exist) */ bool becomes_live_at_instr(int idx) { - if(enable_fancy_coloring) { - if(idx == min) return true; - if(idx < min || idx > max) return false; + if (enable_fancy_coloring) { + if (idx == min) + return true; + if (idx < min || idx > max) + return false; assert(idx > min); return is_alive.at(idx - min) && !is_alive.at(idx - min - 1); } else { @@ -81,9 +88,11 @@ struct LiveInfo { * Are we alive at idx, but not alive at idx + 1 (or idx + 1 doesn't exist) */ bool dies_next_at_instr(int idx) { - if(enable_fancy_coloring) { - if(idx == max) return true; - if(idx < min || idx > max) return false; + if (enable_fancy_coloring) { + if (idx == max) + return true; + if (idx < min || idx > max) + return false; assert(idx < max); return is_alive.at(idx - min) && !is_alive.at(idx - min + 1); } else { @@ -96,11 +105,12 @@ struct LiveInfo { */ void prepare_for_allocation(int id) { var = id; - if(!seen) return; // don't do any prep for a variable which isn't used. + if (!seen) + return; // don't do any prep for a variable which isn't used. assert(max - min >= 0); assignment.resize(max - min + 1); is_alive.resize(max - min + 1); - for(auto& x : indices_of_alive) { + for (auto& x : indices_of_alive) { is_alive.at(x - min) = true; } } @@ -110,8 +120,11 @@ struct LiveInfo { * Will overwrite any previous assignment here * Will set best_hint to this assignment. */ - void constrain_at_one(int id, Assignment ass) { + void constrain_at_one(int id, emitter::Register reg) { assert(id >= min && id <= max); + Assignment ass; + ass.reg = reg; + ass.kind = Assignment::Kind::REGISTER; assignment.at(id - min) = ass; has_constraint = true; best_hint = ass; @@ -125,6 +138,17 @@ struct LiveInfo { return assignment.at(id - min).occupies_same_reg(ass); } + /*! + * At the given instruction, does the given assignment conflict with this one? + */ + bool conflicts_at(int id, emitter::Register reg) { + assert(id >= min && id <= max); + Assignment ass; + ass.reg = reg; + ass.kind = Assignment::Kind::REGISTER; + return assignment.at(id - min).occupies_same_reg(ass); + } + /*! * Assign variable to the given assignment at all instructions * Throws if this would require modifying a currently set assignment. @@ -132,9 +156,9 @@ struct LiveInfo { void assign_no_overwrite(Assignment ass) { assert(seen); assert(ass.is_assigned()); - for(int i = min; i <= max; i++) { + for (int i = min; i <= max; i++) { auto& a = assignment.at(i - min); - if(a.is_assigned() && !(a.occupies_same_reg(ass))) { + if (a.is_assigned() && !(a.occupies_same_reg(ass))) { throw std::runtime_error("assign_no_overwrite failed!"); } else { a = ass; @@ -145,15 +169,15 @@ struct LiveInfo { /*! * Get the assignment at the given instruction. */ - const Assignment& get(int id) { + const Assignment& get(int id) const { assert(id >= min && id <= max); return assignment.at(id - min); } std::string print_assignment() { std::string result = "Assignment for var " + std::to_string(var) + "\n"; - for(uint32_t i = 0; i < assignment.size(); i++) { - result += fmt::format("i[{:3d}] {}\n", i+min, assignment.at(i).to_string()); + for (uint32_t i = 0; i < assignment.size(); i++) { + result += fmt::format("i[{:3d}] {}\n", i + min, assignment.at(i).to_string()); } return result; } diff --git a/goalc/regalloc/StackOp.h b/goalc/regalloc/StackOp.h index 90e21c82ed..fff5b7940f 100644 --- a/goalc/regalloc/StackOp.h +++ b/goalc/regalloc/StackOp.h @@ -1,6 +1,7 @@ /*! * @file StackOp.h - * An operation that's added to an Instruction so that it loads/stores things from the stack + * An operation that's added to an Instruction so that it loads/stores things from the stack if + * needed for spilling. */ #ifndef JAK_STACKOP_H @@ -20,7 +21,7 @@ struct StackOp { std::vector ops; - std::string print() { + std::string print() const { std::string result; bool added = false; for (const auto& op : ops) { diff --git a/goalc/regalloc/allocate.cpp b/goalc/regalloc/allocate.cpp index f902ed09c8..8e6bf798ea 100644 --- a/goalc/regalloc/allocate.cpp +++ b/goalc/regalloc/allocate.cpp @@ -1,8 +1,16 @@ +/*! + * @file allocate.cpp + * Runs the register allocator. + */ + #include "third-party/fmt/core.h" #include "allocate.h" #include "Allocator.h" namespace { +/*! + * Print out the input data for debugging. + */ void print_allocate_input(const AllocationInput& in) { fmt::print("[RegAlloc] Debug Input:\n"); if (in.instructions.size() == in.debug_instruction_names.size()) { @@ -19,14 +27,16 @@ void print_allocate_input(const AllocationInput& in) { fmt::print(" {}\n", c.to_string()); } - fmt::print(" aligned_stack? {}\n", in.requires_aligned_stack); fmt::print("\n"); } +/*! + * Print out the state of the RegAllocCache after doing analysis. + */ void print_analysis(const AllocationInput& in, RegAllocCache* cache) { fmt::print("[RegAlloc] Basic Blocks\n"); fmt::print("-----------------------------------------------------------------\n"); - for(auto& b : cache->basic_blocks) { + for (auto& b : cache->basic_blocks) { fmt::print("{}\n", b.print(in.instructions)); } @@ -34,25 +44,25 @@ void print_analysis(const AllocationInput& in, RegAllocCache* cache) { printf("-----------------------------------------------------------------\n"); // align to where we start putting live stuff printf(" %30s ", ""); - for(int i = 0; i < cache->max_var; i++) { + for (int i = 0; i < cache->max_var; i++) { printf("%2d ", i); } printf("\n"); printf("_________________________________________________________________\n"); - for(uint32_t i = 0; i < in.instructions.size(); i++) { + for (uint32_t i = 0; i < in.instructions.size(); i++) { std::vector ids_live; std::string lives; ids_live.resize(cache->max_var, false); - for(int j = 0; j < cache->max_var; j++) { - if(cache->live_ranges.at(j).is_live_at_instr(i)) { + for (int j = 0; j < cache->max_var; j++) { + if (cache->live_ranges.at(j).is_live_at_instr(i)) { ids_live.at(j) = true; } } - for(uint32_t j = 0; j < ids_live.size(); j++) { - if(ids_live[j]) { + for (uint32_t j = 0; j < ids_live.size(); j++) { + if (ids_live[j]) { char buff[256]; sprintf(buff, "%2d ", j); lives.append(buff); @@ -61,9 +71,9 @@ void print_analysis(const AllocationInput& in, RegAllocCache* cache) { } } - if(in.debug_instruction_names.size() == in.instructions.size()) { + if (in.debug_instruction_names.size() == in.instructions.size()) { std::string code_str = in.debug_instruction_names.at(i); - if(code_str.length() >= 50) { + if (code_str.length() >= 50) { code_str = code_str.substr(0, 48); code_str.push_back('~'); } @@ -71,7 +81,38 @@ void print_analysis(const AllocationInput& in, RegAllocCache* cache) { } else { printf("[%03d] %30s -> %s\n", i, "???", lives.c_str()); } + } +} + +/*! + * Print the result of register allocation for debugging. + */ +void print_result(const AllocationInput& in, const AllocationResult& result) { + printf("[RegAlloc] result:\n"); + printf("-----------------------------------------------------------------\n"); + for (uint32_t i = 0; i < in.instructions.size(); i++) { + std::vector ids_live; + std::string lives; + + ids_live.resize(in.max_vars, false); + + for (int j = 0; j < in.max_vars; j++) { + if (result.ass_as_ranges.at(j).is_live_at_instr(i)) { + lives += std::to_string(j) + " " + result.ass_as_ranges.at(j).get(i).to_string() + " "; + } + } + std::string code_str; + if (in.debug_instruction_names.size() == in.instructions.size()) { + code_str = in.debug_instruction_names.at(i); + } + + if (code_str.length() >= 50) { + code_str = code_str.substr(0, 48); + code_str.push_back('~'); + } + printf("[%03d] %30s | %30s | %30s\n", i, code_str.c_str(), lives.c_str(), + result.stack_ops.at(i).print().c_str()); } } } // namespace @@ -91,14 +132,76 @@ AllocationResult allocate_registers(const AllocationInput& input) { // first step is analysis find_basic_blocks(&cache, input); analyze_liveliness(&cache, input); - if(input.debug_settings.print_analysis) { + if (input.debug_settings.print_analysis) { print_analysis(input, &cache); } + // do constraints first, to get them out of the way + do_constrained_alloc(&cache, input, input.debug_settings.trace_debug_constraints); + // the user may have specified invalid constraints, so we should attempt to find conflicts now + // rather than having the register allocation mysteriously fail later on or silently ignore a + // constraint. + if (!check_constrained_alloc(&cache, input)) { + result.ok = false; + fmt::print("[RegAlloc Error] Register allocation has failed due to bad constraints.\n"); + return result; + } + + // do the allocations! + if (!run_allocator(&cache, input, input.debug_settings.allocate_log_level)) { + result.ok = false; + fmt::print("[RegAlloc Error] Register allocation has failed.\n"); + return result; + } + + // prepare the result + result.ok = true; + result.needs_aligned_stack_for_spills = cache.used_stack; + result.stack_slots = cache.current_stack_slot; + + // copy over the assignment result + result.assignment.resize(cache.max_var); + for (size_t i = 0; i < result.assignment.size(); i++) { + auto& x = result.assignment[i]; + x.resize(input.instructions.size()); + const auto& lr = cache.live_ranges.at(i); + for (int j = lr.min; j <= lr.max; j++) { + x.at(j) = lr.get(j); + } + } + + // check for use of saved registers + for (auto sr : emitter::gRegInfo.get_all_saved()) { + bool uses_sr = false; + for (auto& lr : cache.live_ranges) { + for (int instr_idx = lr.min; instr_idx <= lr.max; instr_idx++) { + if (lr.get(instr_idx).reg == sr) { + uses_sr = true; + break; + } + } + if (uses_sr) { + break; + } + } + if (uses_sr) { + result.used_saved_regs.push_back(sr); + } + } + result.ass_as_ranges = std::move(cache.live_ranges); + result.stack_ops = std::move(cache.stack_ops); + + // final result print + if (input.debug_settings.print_result) { + print_result(input, result); + } return result; } +/*! + * Print for debugging + */ std::string RegAllocInstr::print() const { bool first = true; std::string result = "("; diff --git a/goalc/regalloc/allocate.h b/goalc/regalloc/allocate.h index cd13e43e9f..46c3423cf4 100644 --- a/goalc/regalloc/allocate.h +++ b/goalc/regalloc/allocate.h @@ -13,6 +13,9 @@ #include #include "goalc/emitter/Register.h" #include "IRegister.h" +#include "StackOp.h" +#include "Assignment.h" +#include "LiveInfo.h" /*! * Information about an instruction needed for register allocation. @@ -25,40 +28,75 @@ * Possibly because the actual implementation requires using it. */ struct RegAllocInstr { - std::vector clobber; - std::vector exclude; - std::vector write; - std::vector read; - std::vector jumps; - bool fallthrough = true; - bool is_move = false; + std::vector clobber; // written, but safe to use as input/output + std::vector exclude; // written, unsafe to use for input/output + std::vector write; // results go in here + std::vector read; // inputs go in here + std::vector jumps; // RegAllocInstr indexes of possible jumps + bool fallthrough = true; // can it fall through to the next instruction + bool is_move = false; // is this a move? std::string print() const; -}; + /*! + * Does this read IReg id? + */ + bool reads(int id) const { + for (const auto& x : read) { + if (x.id == id) + return true; + } + return false; + } + + /*! + * Does this write IReg id? + */ + bool writes(int id) const { + for (const auto& x : write) { + if (x.id == id) + return true; + } + return false; + } +}; +/*! + * Result of the allocate_registers algorithm + */ struct AllocationResult { - bool ok = false; + bool ok = false; // did it work? + std::vector> assignment; // variable, instruction + std::vector ass_as_ranges; // another format, maybe easier? + std::vector used_saved_regs; // which saved regs get clobbered? + int stack_slots = 0; // how many space on the stack do we need? + std::vector stack_ops; // additional instructions to spill/restore + bool needs_aligned_stack_for_spills = false; }; +/*! + * Input to the allocate_registers algorithm + */ struct AllocationInput { - std::vector instructions; - std::vector constraints; - bool requires_aligned_stack = false; - int max_vars = -1; - - std::vector debug_instruction_names; // optional - + std::vector instructions; // all instructions in the function + std::vector constraints; // all register constraints + int max_vars = -1; // maximum register id. + std::vector debug_instruction_names; // optional, for debug prints + struct { + bool print_input = false; + bool print_analysis = false; + bool trace_debug_constraints = false; + int allocate_log_level = 0; + bool print_result = false; + } debug_settings; + /*! + * Add instruction and return its idx. + */ int add_instruction(const RegAllocInstr& instr) { instructions.push_back(instr); return int(instructions.size()) - 1; } - - struct { - bool print_input = false; - bool print_analysis = false; - } debug_settings; }; AllocationResult allocate_registers(const AllocationInput& input); From d40c9be3455cf443034033b72485fb668f16b781 Mon Sep 17 00:00:00 2001 From: water Date: Sat, 5 Sep 2020 15:11:49 -0400 Subject: [PATCH 05/13] fix const --- common/type_system/TypeSystem.cpp | 2 +- common/type_system/TypeSystem.h | 2 +- goalc/CMakeLists.txt | 2 + goalc/compiler/CodeGenerator.cpp | 3 ++ goalc/compiler/CodeGenerator.h | 8 ++++ goalc/compiler/Env.h | 12 ++++++ goalc/compiler/IR.cpp | 3 ++ goalc/compiler/IR.h | 21 +++++++++ goalc/compiler/Val.cpp | 21 ++++++++- goalc/compiler/Val.h | 71 ++++++++++++++++++++++++++++++- 10 files changed, 141 insertions(+), 4 deletions(-) create mode 100644 goalc/compiler/CodeGenerator.cpp create mode 100644 goalc/compiler/CodeGenerator.h create mode 100644 goalc/compiler/IR.cpp create mode 100644 goalc/compiler/IR.h diff --git a/common/type_system/TypeSystem.cpp b/common/type_system/TypeSystem.cpp index 3b61135bf5..308128dce3 100644 --- a/common/type_system/TypeSystem.cpp +++ b/common/type_system/TypeSystem.cpp @@ -145,7 +145,7 @@ DerefInfo TypeSystem::get_deref_info(const TypeSpec& ts) { * Create a simple typespec. The type must be defined or forward declared for this to work. * If you really need a TypeSpec which refers to a non-existent type, just construct your own. */ -TypeSpec TypeSystem::make_typespec(const std::string& name) { +TypeSpec TypeSystem::make_typespec(const std::string& name) const { if (m_types.find(name) != m_types.end() || m_forward_declared_types.find(name) != m_forward_declared_types.end()) { return TypeSpec(name); diff --git a/common/type_system/TypeSystem.h b/common/type_system/TypeSystem.h index 722ca515ae..b5612992ec 100644 --- a/common/type_system/TypeSystem.h +++ b/common/type_system/TypeSystem.h @@ -37,7 +37,7 @@ class TypeSystem { DerefInfo get_deref_info(const TypeSpec& ts); - TypeSpec make_typespec(const std::string& name); + TypeSpec make_typespec(const std::string& name) const; TypeSpec make_function_typespec(const std::vector& arg_types, const std::string& return_type); diff --git a/goalc/CMakeLists.txt b/goalc/CMakeLists.txt index e43a3d77cc..1d42dcf820 100644 --- a/goalc/CMakeLists.txt +++ b/goalc/CMakeLists.txt @@ -18,6 +18,8 @@ add_library(compiler compiler/Compiler.cpp compiler/Env.cpp compiler/Val.cpp + compiler/IR.cpp + compiler/CodeGenerator.cpp regalloc/IRegister.cpp regalloc/Allocator.cpp regalloc/allocate.cpp diff --git a/goalc/compiler/CodeGenerator.cpp b/goalc/compiler/CodeGenerator.cpp new file mode 100644 index 0000000000..2f78d6224d --- /dev/null +++ b/goalc/compiler/CodeGenerator.cpp @@ -0,0 +1,3 @@ + + +#include "CodeGenerator.h" diff --git a/goalc/compiler/CodeGenerator.h b/goalc/compiler/CodeGenerator.h new file mode 100644 index 0000000000..6d1ca719f7 --- /dev/null +++ b/goalc/compiler/CodeGenerator.h @@ -0,0 +1,8 @@ + + +#ifndef JAK_CODEGENERATOR_H +#define JAK_CODEGENERATOR_H + +class CodeGenerator {}; + +#endif // JAK_CODEGENERATOR_H diff --git a/goalc/compiler/Env.h b/goalc/compiler/Env.h index 417f99f77b..b8281b3382 100644 --- a/goalc/compiler/Env.h +++ b/goalc/compiler/Env.h @@ -5,4 +5,16 @@ class Env {}; +// global +// noemit +// objectfile +// configuration +// function +// block +// lexical +// label +// symbolmacro +// get parent env of type. + + #endif // JAK_ENV_H diff --git a/goalc/compiler/IR.cpp b/goalc/compiler/IR.cpp new file mode 100644 index 0000000000..479292ef2e --- /dev/null +++ b/goalc/compiler/IR.cpp @@ -0,0 +1,3 @@ + + +#include "IR.h" diff --git a/goalc/compiler/IR.h b/goalc/compiler/IR.h new file mode 100644 index 0000000000..f379466466 --- /dev/null +++ b/goalc/compiler/IR.h @@ -0,0 +1,21 @@ +#ifndef JAK_IR_H +#define JAK_IR_H + +#include +#include "CodeGenerator.h" +#include "goalc/regalloc/allocate.h" + +class IR { + public: + virtual std::string print() = 0; + virtual RegAllocInstr to_rai() = 0; + virtual void do_codegen(CodeGenerator* gen) = 0; +}; + +class IR_Set : public IR { + public: + std::string print() override; + RegAllocInstr to_rai() override; +}; + +#endif // JAK_IR_H diff --git a/goalc/compiler/Val.cpp b/goalc/compiler/Val.cpp index b7fee6cae7..28046ac206 100644 --- a/goalc/compiler/Val.cpp +++ b/goalc/compiler/Val.cpp @@ -1,3 +1,22 @@ +#include "Val.h" +/*! + * Fallback to_gpr if a more optimized one is not provided. + */ +RegVal Val::to_gpr(FunctionEnv* fe) const { + (void)fe; + throw std::runtime_error("Val::to_gpr NYI"); +} -#include "Val.h" +/*! + * Fallback to_xmm if a more optimized one is not provided. + */ +RegVal Val::to_xmm(FunctionEnv* fe) const { + (void)fe; + throw std::runtime_error("Val::to_xmm NYI"); +} + +RegVal None::to_reg(FunctionEnv* fe) const { + (void)fe; + throw std::runtime_error("Cannot put None into a register."); +} \ No newline at end of file diff --git a/goalc/compiler/Val.h b/goalc/compiler/Val.h index eacdc28a18..031526e6f5 100644 --- a/goalc/compiler/Val.h +++ b/goalc/compiler/Val.h @@ -6,6 +6,75 @@ #ifndef JAK_VAL_H #define JAK_VAL_H -class Val {}; +#include +#include +#include "third-party/fmt/core.h" +#include "common/type_system/TypeSystem.h" +#include "goalc/regalloc/IRegister.h" + + +class RegVal; +class FunctionEnv; + +/*! + * Parent class for every Val. + */ +class Val { + public: + explicit Val(TypeSpec ts) : m_ts(std::move(ts)) {} + + virtual bool is_register() const { return false; } + + virtual IRegister ireg() const { + throw std::runtime_error("get_ireg called on invalid Val: " + print()); + } + + virtual std::string print() const = 0; + virtual RegVal to_reg(FunctionEnv* fe) const = 0; + virtual RegVal to_gpr(FunctionEnv* fe) const; + virtual RegVal to_xmm(FunctionEnv* fe) const; + + const TypeSpec& type() const { return m_ts; } + + protected: + TypeSpec m_ts; +}; + +/*! + * Special None Val used for the value of anything returning (none). + */ +class None : public Val { + explicit None(TypeSpec _ts) : Val(std::move(_ts)) {} + explicit None(const TypeSystem& _ts) : Val(_ts.make_typespec("none")) {} + std::string print() const override { return "none"; } + RegVal to_reg(FunctionEnv* fe) const override; +}; + +/*! + * A Val stored in a register. + */ +class RegVal : public Val { + public: + RegVal(IRegister ireg, TypeSpec ts) : Val(std::move(ts)), m_ireg(ireg) {} + bool is_register() const override { return true; } + IRegister ireg() const override { return m_ireg; } + std::string print() const override { return m_ireg.to_string(); }; + + protected: + IRegister m_ireg; +}; + + +// Symbol +// Lambda +// Static +// MemOffConstant +// MemOffVar +// MemDeref +// PairEntry +// Alias +// IntegerConstant +// FloatConstant +// Bitfield #endif // JAK_VAL_H From 4589c260af6af2b63e61b2c613afd4076b63658a Mon Sep 17 00:00:00 2001 From: water Date: Sat, 5 Sep 2020 15:37:20 -0400 Subject: [PATCH 06/13] fix build --- goalc/compiler/Val.cpp | 9 +++++---- goalc/compiler/Val.h | 11 +++++++---- 2 files changed, 12 insertions(+), 8 deletions(-) diff --git a/goalc/compiler/Val.cpp b/goalc/compiler/Val.cpp index 28046ac206..b510a5da5b 100644 --- a/goalc/compiler/Val.cpp +++ b/goalc/compiler/Val.cpp @@ -3,7 +3,7 @@ /*! * Fallback to_gpr if a more optimized one is not provided. */ -RegVal Val::to_gpr(FunctionEnv* fe) const { +RegVal* Val::to_gpr(FunctionEnv* fe) const { (void)fe; throw std::runtime_error("Val::to_gpr NYI"); } @@ -11,12 +11,13 @@ RegVal Val::to_gpr(FunctionEnv* fe) const { /*! * Fallback to_xmm if a more optimized one is not provided. */ -RegVal Val::to_xmm(FunctionEnv* fe) const { +RegVal* Val::to_xmm(FunctionEnv* fe) const { (void)fe; throw std::runtime_error("Val::to_xmm NYI"); } -RegVal None::to_reg(FunctionEnv* fe) const { +RegVal* None::to_reg(FunctionEnv* fe) const { (void)fe; throw std::runtime_error("Cannot put None into a register."); -} \ No newline at end of file +} + diff --git a/goalc/compiler/Val.h b/goalc/compiler/Val.h index 031526e6f5..9754e500e4 100644 --- a/goalc/compiler/Val.h +++ b/goalc/compiler/Val.h @@ -30,9 +30,9 @@ class Val { } virtual std::string print() const = 0; - virtual RegVal to_reg(FunctionEnv* fe) const = 0; - virtual RegVal to_gpr(FunctionEnv* fe) const; - virtual RegVal to_xmm(FunctionEnv* fe) const; + virtual RegVal* to_reg(FunctionEnv* fe) const = 0; + virtual RegVal* to_gpr(FunctionEnv* fe) const; + virtual RegVal* to_xmm(FunctionEnv* fe) const; const TypeSpec& type() const { return m_ts; } @@ -47,7 +47,7 @@ class None : public Val { explicit None(TypeSpec _ts) : Val(std::move(_ts)) {} explicit None(const TypeSystem& _ts) : Val(_ts.make_typespec("none")) {} std::string print() const override { return "none"; } - RegVal to_reg(FunctionEnv* fe) const override; + RegVal* to_reg(FunctionEnv* fe) const override; }; /*! @@ -59,6 +59,9 @@ class RegVal : public Val { bool is_register() const override { return true; } IRegister ireg() const override { return m_ireg; } std::string print() const override { return m_ireg.to_string(); }; + RegVal* to_reg(FunctionEnv* fe) const override; + RegVal* to_gpr(FunctionEnv* fe) const override; + RegVal* to_xmm(FunctionEnv* fe) const override; protected: IRegister m_ireg; From cfa9401587900ae3209979971e5722de226d6af5 Mon Sep 17 00:00:00 2001 From: water Date: Sat, 5 Sep 2020 15:46:55 -0400 Subject: [PATCH 07/13] fix formatting for clang-format --- goalc/compiler/Env.h | 1 - goalc/compiler/Val.cpp | 1 - goalc/compiler/Val.h | 2 -- 3 files changed, 4 deletions(-) diff --git a/goalc/compiler/Env.h b/goalc/compiler/Env.h index b8281b3382..e7c85e61d9 100644 --- a/goalc/compiler/Env.h +++ b/goalc/compiler/Env.h @@ -16,5 +16,4 @@ class Env {}; // symbolmacro // get parent env of type. - #endif // JAK_ENV_H diff --git a/goalc/compiler/Val.cpp b/goalc/compiler/Val.cpp index b510a5da5b..ef5ba521fd 100644 --- a/goalc/compiler/Val.cpp +++ b/goalc/compiler/Val.cpp @@ -20,4 +20,3 @@ RegVal* None::to_reg(FunctionEnv* fe) const { (void)fe; throw std::runtime_error("Cannot put None into a register."); } - diff --git a/goalc/compiler/Val.h b/goalc/compiler/Val.h index 9754e500e4..2fe374d7f3 100644 --- a/goalc/compiler/Val.h +++ b/goalc/compiler/Val.h @@ -12,7 +12,6 @@ #include "common/type_system/TypeSystem.h" #include "goalc/regalloc/IRegister.h" - class RegVal; class FunctionEnv; @@ -67,7 +66,6 @@ class RegVal : public Val { IRegister m_ireg; }; - // Symbol // Lambda // Static From 14ae4ee812db2c05507ae925ed2bcecda195c7e2 Mon Sep 17 00:00:00 2001 From: water Date: Sat, 5 Sep 2020 15:49:53 -0400 Subject: [PATCH 08/13] attempt to fix windows build --- goalc/emitter/ObjectGenerator.h | 1 + 1 file changed, 1 insertion(+) diff --git a/goalc/emitter/ObjectGenerator.h b/goalc/emitter/ObjectGenerator.h index 8a7a70acf4..c7ee449d25 100644 --- a/goalc/emitter/ObjectGenerator.h +++ b/goalc/emitter/ObjectGenerator.h @@ -3,6 +3,7 @@ #include #include +#include #include "ObjectFileData.h" #include "Instruction.h" From 92a38ed89df13075d7cf30e57cdc045946e718f4 Mon Sep 17 00:00:00 2001 From: water Date: Sat, 5 Sep 2020 15:52:33 -0400 Subject: [PATCH 09/13] windows 2 --- goalc/compiler/Val.h | 1 + 1 file changed, 1 insertion(+) diff --git a/goalc/compiler/Val.h b/goalc/compiler/Val.h index 2fe374d7f3..a95f9f285f 100644 --- a/goalc/compiler/Val.h +++ b/goalc/compiler/Val.h @@ -8,6 +8,7 @@ #include #include +#include #include "third-party/fmt/core.h" #include "common/type_system/TypeSystem.h" #include "goalc/regalloc/IRegister.h" From 5ff22da84d613b6cefeef25f5e2a05f6ff1ae979 Mon Sep 17 00:00:00 2001 From: water Date: Sat, 5 Sep 2020 15:55:05 -0400 Subject: [PATCH 10/13] windows 3 --- goalc/regalloc/LiveInfo.h | 1 + 1 file changed, 1 insertion(+) diff --git a/goalc/regalloc/LiveInfo.h b/goalc/regalloc/LiveInfo.h index 588a97304e..2af1007534 100644 --- a/goalc/regalloc/LiveInfo.h +++ b/goalc/regalloc/LiveInfo.h @@ -6,6 +6,7 @@ #include #include #include +#include #include "Assignment.h" // with this on, gaps in usage of registers allow other variables to steal registers. From eebad42497526a9c92d8592876bbed55f8da59f7 Mon Sep 17 00:00:00 2001 From: water Date: Sat, 5 Sep 2020 16:04:41 -0400 Subject: [PATCH 11/13] windows 4 --- goalc/CMakeLists.txt | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/goalc/CMakeLists.txt b/goalc/CMakeLists.txt index 1d42dcf820..0c29c448ac 100644 --- a/goalc/CMakeLists.txt +++ b/goalc/CMakeLists.txt @@ -31,6 +31,7 @@ add_executable(goalc main.cpp IF (WIN32) set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON) + target_link_libraries(compiler util goos type_system logger mman) ENDIF() -target_link_libraries(goalc util goos type_system compiler type_system logger) +target_link_libraries(goalc util goos type_system compiler logger) From 399258b0245e54ecfa9c9cf6d077dd9cdca03ba0 Mon Sep 17 00:00:00 2001 From: water Date: Sat, 5 Sep 2020 16:08:25 -0400 Subject: [PATCH 12/13] windows 5 --- goalc/logger/Logger.cpp | 1 + 1 file changed, 1 insertion(+) diff --git a/goalc/logger/Logger.cpp b/goalc/logger/Logger.cpp index 563bde1ef7..684d406a6a 100644 --- a/goalc/logger/Logger.cpp +++ b/goalc/logger/Logger.cpp @@ -1,3 +1,4 @@ +#include #include "Logger.h" void Logger::close() { From 63697a73e07dc35a451eacb97b13e55b7d8eb0f8 Mon Sep 17 00:00:00 2001 From: water Date: Sat, 5 Sep 2020 16:27:07 -0400 Subject: [PATCH 13/13] windows 6 --- common/type_system/CMakeLists.txt | 1 + goalc/CMakeLists.txt | 6 +++--- goalc/logger/CMakeLists.txt | 1 - test/CMakeLists.txt | 4 ++-- 4 files changed, 6 insertions(+), 6 deletions(-) delete mode 100644 goalc/logger/CMakeLists.txt diff --git a/common/type_system/CMakeLists.txt b/common/type_system/CMakeLists.txt index 97a6aeff65..d8db0b46ec 100644 --- a/common/type_system/CMakeLists.txt +++ b/common/type_system/CMakeLists.txt @@ -1,4 +1,5 @@ add_library(type_system + SHARED TypeSystem.cpp Type.cpp TypeSpec.cpp) diff --git a/goalc/CMakeLists.txt b/goalc/CMakeLists.txt index 0c29c448ac..21b3fcb92c 100644 --- a/goalc/CMakeLists.txt +++ b/goalc/CMakeLists.txt @@ -1,6 +1,5 @@ add_subdirectory(util) add_subdirectory(goos) -add_subdirectory(logger) IF (WIN32) # TODO - implement windows listener @@ -20,6 +19,7 @@ add_library(compiler compiler/Val.cpp compiler/IR.cpp compiler/CodeGenerator.cpp + logger/Logger.cpp regalloc/IRegister.cpp regalloc/Allocator.cpp regalloc/allocate.cpp @@ -31,7 +31,7 @@ add_executable(goalc main.cpp IF (WIN32) set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON) - target_link_libraries(compiler util goos type_system logger mman) + target_link_libraries(compiler util goos type_system mman) ENDIF() -target_link_libraries(goalc util goos type_system compiler logger) +target_link_libraries(goalc util goos compiler type_system) diff --git a/goalc/logger/CMakeLists.txt b/goalc/logger/CMakeLists.txt deleted file mode 100644 index e2e0b11dd9..0000000000 --- a/goalc/logger/CMakeLists.txt +++ /dev/null @@ -1 +0,0 @@ -add_library(logger SHARED Logger.cpp) \ No newline at end of file diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt index 656c7fe8a2..8173c9820c 100644 --- a/test/CMakeLists.txt +++ b/test/CMakeLists.txt @@ -20,7 +20,7 @@ IF (WIN32) set(gtest_force_shared_crt ON CACHE BOOL "" FORCE) # TODO - implement windows listener message("Windows Listener Not Implemented!") - target_link_libraries(goalc-test mman goos util runtime compiler type_system gtest logger) + target_link_libraries(goalc-test mman goos util runtime compiler type_system gtest) ELSE() - target_link_libraries(goalc-test goos util listener runtime compiler type_system gtest logger) + target_link_libraries(goalc-test goos util listener runtime compiler type_system gtest) ENDIF()