Process Text Files in Decompiler

[water111]

It turns out the game text files (TEXT/XCOMMON.TXT) are GOAL object files. They break some assumptions I made, like

• They are the only v2 GOAL file, everything else is v3 or v4. The GOAL disassembler doesn't support v2 properly.
• They contain GOAL strings with non-printable characters. The GOAL disassembler doesn't work on these properly.
• They are located outside of a DGO file. This breaks the naming convention and object file map system.

This PR will fix these issues.

To Do before merging

• Support v2 in klink.cpp
• Confirm we can load and link text files.
• Test compiling/loading/inspecting fake text data in a test
• Document text stuff
• Add a test for accessing inline arrays.

[coveralls]

Pull Request Test Coverage Report for Build 373530819

• 512 of 658 (77.81%) changed or added relevant lines in 13 files are covered.
• 1 unchanged line in 1 file lost coverage.
• Overall coverage increased (+0.3%) to 81.558%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
goalc/compiler/compilation/CompilerControl.cpp	7	9	77.78%
goalc/compiler/Compiler.cpp	0	4	0.0%
game/sce/sif_ee.cpp	37	51	72.55%
goalc/data_compiler/game_text.cpp	89	105	84.76%
goalc/data_compiler.cpp	0	25	0.0%
goalc/data_compiler/DataObjectGenerator.cpp	112	137	81.75%
game/kernel/klink.cpp	145	205	70.73%

Files with Coverage Reduction	New Missed Lines	%
game/overlord/iso.cpp	1	77.62%

Totals
Change from base Build 368892222:	0.3%
Covered Lines:	16593
Relevant Lines:	20345

---

💛 - Coveralls

[water111]

The extracted text looks something like this:

The special characters are a mess of GOAL escape codes and non-printable characters that get turned into \c escape sequences. None of them are standard (I think Japanese is just indices into a table of selected characters) so I am just leaving it as is for now. It's not nice to look at but the game will know how to print these to the screen when needed.

[water111]

We are now able to load text files!

The example works like this:

• First, run the decompiler. Modify assets/game_text.txt to modify text if desired.
• Run (build-data) at the compiler to build the data files
• Run check.sh if you didn't modify the text to verify the text data file exactly matches the game.
• Start gk, connect the listener with (lt) and run (blg) to build and load all code
• Run (hack-load-text-test) to load english text.

(defun hack-load-text-test ()
  "Test to try loading game text"
  (set! *text* (the game-text-info (load "$JAK-PROJECT/out/0COMMON.TXT" *common-text-heap*)))
  )

• Run (inspect *text*) or (print *text*)

gc > (inspect *text*)
[0016a084] game-text-info
	length: 617
	data[617]: @ #x16a090
	  [0] #x103 "~Y~22L<~Z~Y~27L*~Z~Y~1L>~Z~Y~23L[~Z~+26H = YES, ~Y~22L<~Z~Y~26L;~Z~Y~1L>~Z~Y~23L[~Z~+26H = NO"
	  [1] #x104 "PRESS ~Y~22L<~Z~Y~25L@~Z~Y~1L>~Z~Y~23L[~Z~+26H TO TALK."
	  [2] #x105 "PRESS ~Y~22L<~Z~Y~25L@~Z~Y~1L>~Z~Y~23L[~Z~+26H TO USE."
	  [3] #x106 "DO YOU WANT TO PLAY?"
	  [4] #x107 "DO YOU WANT TO PLAY AGAIN?"
	  [5] #x108 "QUIT"
	  [6] #x109 "PAUSE"
...

This still needs some clean up and adding tests before merging, but a huge number of things are working to make this possible!