PAL jak 1 decompiler profile

.vs/launch.vs.json

@@ -76,6 +76,13 @@
       "name" : "Run - Decompiler - Jak 1",
       "args" : [ "${workspaceRoot}/decompiler/config/jak1_ntsc_black_label.jsonc", "${workspaceRoot}/iso_data/jak1", "${workspaceRoot}/decompiler_out/jak1"]
     },
+    {
+      "type" : "default",
+      "project" : "CMakeLists.txt",
+      "projectTarget" : "decompiler.exe (bin\\decompiler.exe)",
+      "name" : "Run - Decompiler - Jak 1 PAL",
+      "args" : [ "${workspaceRoot}/decompiler/config/jak1_pal.jsonc", "${workspaceRoot}/iso_data/jak1pal", "${workspaceRoot}/decompiler_out/jak1pal"]
+    },
     {
       "type" : "default",
       "project" : "CMakeLists.txt",

CMakeLists.txt

@@ -31,7 +31,7 @@ if(UNIX)
         -fdiagnostics-color=always")
 else()
     set(CMAKE_CXX_FLAGS "/EHsc")
-    set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} /STACK:10000000")
+    set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} /STACK:16000000,16384")
 endif(UNIX)
 
 if(WIN32)

README.md

@@ -185,7 +185,7 @@ Place a copy of the game's files in `iso_data`, then run the decompiler with the
 Run the OpenGOAL compiler `build/goalc/goalc`.  Enter `(mi)` to build the `"iso"` target, which contains everything we have so far.
 
 ### Run Game
-In a separate terminal, start the runtime with `build/game/gk -fakeiso -debug`.  Then, in the OpenGOAL window, run `(lt)` to connect, `(lg)` to load the game engine `(test-play)` to start the game engine.  If it all works right, it will look something like this:
+In a separate terminal, start the runtime with `build/game/gk -fakeiso -debug`.  Then, in the OpenGOAL window, run `(mi)` to create the data for the game and give the REPL information for running code, `(lt)` to connect, `(lg)` to load the game engine and `(test-play)` to start the game engine.  If it all works right, it will look something like this:
 ```
 g > (lt)
 [Listener] Socket connected established! (took 0 tries). Waiting for version...
@@ -210,7 +210,7 @@ There are four main components to the project.
 
 The first is `goalc`, which is a GOAL compiler for x86-64. Our implementation of GOAL is called OpenGOAL. All of the compiler source code is in `goalc`. To run the compiler on Linux, there is a script `gc.sh`.  The compiler is controlled through a prompt which can be used to enter commands to compile, connect to a running GOAL program for interaction, run the OpenGOAL debugger, or, if you are connected to a running GOAL program, can be used as a REPL to run code interactively. In addition to compiling code files, the compiler has features to pack and build data files.
 
-The second component to the project is the decompiler. You must have a copy of the PS2 game and place all files from the DVD into the `iso_data` folder. Then run `decomp.sh` to run the decompiler. The decompile will extract assets to the `assets` folder. These assets will be used by the compiler when building the port. The decompiler will output code and other data intended to be inspected by humans in the `decompiler_out` folder. Stuff in this folder will not be used by the compiler.
+The second component to the project is the decompiler. You must have a copy of the PS2 game and place all files from the DVD into the `iso_data` folder. Then run `decomp.sh` (Linux) to run the decompiler. For Windows, it is the `decomp-jak1.bat` file, and it wants your game's DVD files in a `jak1` folder inside `iso_data`. The decompile will extract assets to the `assets` folder. These assets will be used by the compiler when building the port, and you may want to turn asset extraction off after running it once. The decompiler will output code and other data intended to be inspected by humans in the `decompiler_out` folder. Stuff in this folder will not be used by the compiler.
 
 The third is the game source code, written in OpenGOAL. This is located in `goal_src`. All GOAL and GOOS code should be in this folder.  Right now most of this is placeholders, but you can take a look at `kernel/gcommon.gc` or `goal-lib.gc` to see some in-progress source code.
 
@@ -278,20 +278,20 @@ The final component is the "runtime", located in `game`. This is the part of the
   - `iso`: Final outputs that are used by the game.
   - `obj`: Object files generated by the compiler.
 - `resources`: To be removed. Contains fake versions of some files required to get things booting.
-- `scripts`: Utility scripts.
+- `scripts`: Utility scripts. Windows-specific batch files are in a `batch` folder while Unix shell scripts are in a `shell` folder.
 - `test`: Unit tests (run on GitHub Actions)
 - `third-party`: Third party libraries
   - CMake Code Coverage. For code coverage statistics on GitHub builds
   - `fmt`. String formatting library
   - `googletest`: Test framework
   - `inja`: templating library used for generating test code for compiler tests
-  - `minilzo`: decompression code for Jak 2 and later DGOs
+  - `lzokay`: decompression code for Jak 2 and later DGOs
   - `mman`: Windows library used to emulate `mmap` on Linux
   - `run-clang-format`: Utility to check and enforce code formatting
   - `run-clang-tidy`
   - `zydis`: x86-64 disassembler used in the OpenGOAL debugger
   - `json`: A JSON library
-  - `replxx`: Used for the REPL input. Support history and useful editing shortcuts.
+  - `replxx`: Used for the REPL input. Supports history and useful editing shortcuts.
   - `svpng`: Save a PNG file
 
 

decompiler/ObjectFile/ObjectFileDB.cpp

@@ -213,8 +213,20 @@ void ObjectFileDB::get_objs_from_dgo(const std::string& filename, const Config&
   // get all obj files...
   for (uint32_t i = 0; i < header.object_count; i++) {
     auto obj_header = reader.read<DgoHeader>();
-    assert(reader.bytes_left() >= obj_header.object_count);
     assert_string_empty_after(obj_header.name, 60);
+    if (i == header.object_count - 1) {
+      if (reader.bytes_left() == obj_header.object_count - 0x30) {
+        if (config.is_pal) {
+          lg::warn("Skipping {} because it is a broken PAL object", obj_header.name);
+          reader.ffwd(reader.bytes_left());
+          continue;
+        } else {
+          assert(false);
+        }
+      }
+    } else {
+      assert(reader.bytes_left() >= obj_header.object_count);
+    }
 
     if (std::string(obj_header.name).find("-ag") != std::string::npos) {
       lg::error(

decompiler/ObjectFile/ObjectFileDB.h

@@ -125,6 +125,7 @@ class ObjectFileDB {
     assert(obj_files_by_name.size() == obj_file_order.size());
     for (const auto& name : obj_file_order) {
       for (auto& obj : obj_files_by_name.at(name)) {
+        // lg::info("{}...", name);
         f(obj);
       }
     }

decompiler/config.cpp

@@ -54,6 +54,7 @@ Config read_config_file(const std::string& path_to_config_file) {
   config.dump_objs = cfg.at("dump_objs").get<bool>();
   config.print_cfgs = cfg.at("print_cfgs").get<bool>();
   config.generate_symbol_definition_map = cfg.at("generate_symbol_definition_map").get<bool>();
+  config.is_pal = cfg.at("is_pal").get<bool>();
 
   auto allowed = cfg.at("allowed_objects").get<std::vector<std::string>>();
   for (const auto& x : allowed) {

decompiler/config.h

@@ -106,6 +106,8 @@ struct Config {
 
   bool generate_symbol_definition_map = false;
 
+  bool is_pal = false;
+
   std::unordered_set<std::string> allowed_objects;
   std::unordered_set<std::string> banned_objects;
   std::unordered_map<std::string, std::unordered_map<int, std::vector<RegisterTypeCast>>>

decompiler/config/all-types.gc

@@ -593,6 +593,7 @@
   (shutdown)
   (list-sounds)
   (unload-music)
+  (set-fps)
   )
 
 ;; - Enums
@@ -1058,7 +1059,7 @@
   (ogre-end #x600)
   (ogre-buzzer #x601)
   (ogre-boss #x603)
-  
+
   (assistant-voicebox-intro-ogre-race #x605)
 
   (sidekick-speech-hint-ogre-race #x61c)
@@ -2887,6 +2888,14 @@
   :flag-assert         #x900000004
   )
 
+(deftype sound-rpc-set-fps (sound-rpc-cmd)
+  ((fps  uint8  :offset-assert 4)
+   )
+  :method-count-assert 9
+  :size-assert         #x5
+  :flag-assert         #x900000005
+  )
+
 (deftype sound-rpc-list-sounds (sound-rpc-cmd)
   ()
   :method-count-assert 9
@@ -2922,6 +2931,7 @@
    (set-reverb        sound-rpc-set-reverb           :offset 0)
    (set-ear-trans     sound-rpc-set-ear-trans        :offset 0)
    (set-flava         sound-rpc-set-flava            :offset 0)
+   (set-fps           sound-rpc-set-fps              :offset 0)
    (shutdown          sound-rpc-shutdown             :offset 0)
    (list-sounds       sound-rpc-list-sounds          :offset 0)
    (unload-music      sound-rpc-unload-music         :offset 0)
@@ -11750,11 +11760,12 @@
    (fp-hud              uint64              :offset-assert 560)
    (no-load-wait        uint64              :offset-assert 568)
    (no-look-around-wait uint64              :offset-assert 576)
+   ;;(burn-proc           handle              :offset-assert 584) ;; PAL only
    )
   :heap-base #x1e0
   :method-count-assert 21
-  :size-assert         #x248
-  :flag-assert         #x1501e00248
+  :size-assert         #x248 ;;#x250
+  :flag-assert         #x1501e00248 ;;#x1501e00250
   ;; inherited inspect of process-drawable
   (:methods
     (dummy-20 (_type_ collide-cache) none 20)
@@ -13488,7 +13499,7 @@
 
 (defenum sp-field-id
   :type uint16
-  
+
   (misc-fields-start 0)
   (spt-texture 1)
   (spt-anim 2)
@@ -13616,6 +13627,7 @@
   (bit1 1)
   (start-dead 2)
   (launch-asap 3)
+  (bit6 6)
   )
 
 (deftype sparticle-group-item (structure)
@@ -15186,6 +15198,7 @@
 (define-extern sound-set-sound-falloff (function sound-name int int int int))
 (define-extern sound-set-flava (function uint int))
 (define-extern sound-volume-off (function int))
+(define-extern sound-set-fps (function uint int))
 (define-extern show-iop-info (function dma-buffer int))
 (define-extern show-iop-memory (function dma-buffer int))
 (define-extern make-sqrt-table (function int))
@@ -20021,6 +20034,7 @@
 (define-extern *master-exit* symbol)
 (define-extern *last-master-mode* symbol)
 (define-extern *screen-filter* screen-filter)
+(define-extern *first-boot* symbol) ;; PAL
 
 ;; - Unknowns
 

decompiler/config/jak1_ntsc_black_label.jsonc

@@ -44,7 +44,7 @@
 
   // generate the symbol_map.json file.
   // this is a guess at where each symbol is first defined/used.
-  "generate_symbol_definition_map": true,
+  "generate_symbol_definition_map": false,
 
   // debug option for instruction decoder
   "write_hex_near_instructions": false,
@@ -60,6 +60,9 @@
   // print control flow graph
   "print_cfgs": false,
 
+  // set to true for PAL versions. this will forcefully skip files that have some data missing at the end.
+  "is_pal": false,
+
   ////////////////////////////
   // CONFIG FILES
   ////////////////////////////

decompiler/config/jak1_ntsc_black_label/inputs.jsonc

@@ -263,6 +263,7 @@
   "audio_dir_file_name": "",
 
   "streamed_audio_file_names": [
-    "VAGWAD.ENG"
+    "VAGWAD.ENG",
+    "VAGWAD.JAP"
   ]
 }

decompiler/config/jak1_pal.jsonc

@@ -0,0 +1,81 @@
+{
+  "game_version": 1,
+
+  // if you want to filter to only some object names.
+  // it will make the decompiler much faster.
+  "allowed_objects": [],
+  "banned_objects": [],
+
+  ////////////////////////////
+  // CODE ANALYSIS OPTIONS
+  ////////////////////////////
+
+  // set to true to generate plain .asm files with MIPS disassembly, with no fancy decompilation.
+  // this is fast and should succeed 100% of the time.
+  "disassemble_code": false,
+
+  // Run the decompiler
+  "decompile_code": true,
+
+  ////////////////////////////
+  // DATA ANALYSIS OPTIONS
+  ////////////////////////////
+
+  // set to true to generate plain .asm files for data files.
+  // this will display most data as hex, but will add labels/references/type pointers/strings
+  // this generates a huge amount of output if you run it on the entire game.
+  "disassemble_data": false,
+
+  // unpack textures to assets folder
+  "process_tpages": false,
+  // unpack game text to assets folder
+  "process_game_text": false,
+  // unpack game count to assets folder
+  "process_game_count": false,
+
+  ///////////////////////////
+  // WEIRD OPTIONS
+  ///////////////////////////
+
+  // these options are used rarely and should usually be left at false
+
+  // output a file type_defs.gc which is used for the types part of all-types.gc
+  "regenerate_all_types": false,
+
+  // generate the symbol_map.json file.
+  // this is a guess at where each symbol is first defined/used.
+  "generate_symbol_definition_map": false,
+
+  // debug option for instruction decoder
+  "write_hex_near_instructions": false,
+
+  // experimental tool to extract linked lists used for region scripting in Jak 2 and Jak 3.
+  "write_scripts": false,
+
+  // hex dump of code/data files.
+  "hexdump_code": false,
+  "hexdump_data": false,
+  // dump raw obj files
+  "dump_objs": false,
+  // print control flow graph
+  "print_cfgs": false,
+
+  // set to true for PAL versions. this will forcefully skip files that have some data missing at the end.
+  "is_pal": true,
+
+  ////////////////////////////
+  // CONFIG FILES
+  ////////////////////////////
+
+  "type_casts_file": "decompiler/config/jak1_pal/type_casts.jsonc",
+  "anonymous_function_types_file": "decompiler/config/jak1_pal/anonymous_function_types.jsonc",
+  "var_names_file": "decompiler/config/jak1_pal/var_names.jsonc",
+  "label_types_file": "decompiler/config/jak1_pal/label_types.jsonc",
+  "stack_structures_file": "decompiler/config/jak1_pal/stack_structures.jsonc",
+  "hacks_file": "decompiler/config/jak1_pal/hacks.jsonc",
+  "inputs_file": "decompiler/config/jak1_pal/inputs.jsonc",
+
+  // optional: a predetermined object file name map from a file.
+  // this will make decompilation naming consistent even if you only run on some objects.
+  "obj_file_name_map_file": ""
+}