Decompiling a function

Function decompilation (decomp) is one of the core areas of a decomp project. In the ROM, functions are encoded as ARM (or THUMB) assembly code, and the goal is to transform this assembly into C code that produces the exact same lines of assembly when compiled (often called "matching"). C code is easier to read and modify than assembly, making a decompiled C function easier to hack or research with.

To decompile a function, you need to know both ARM assembly and C. It is also helpful (but not required) to use a reverse engineering tool like Ghidra or IDA.

If you are not familiar with ARM assembly or Ghidra, you can check out Reverse Engineering a DS Game for a primer on reverse engineering, including steps to set up Ghidra with EoS symbols and an introduction to reading ARM assembly. You can also look at Whirlwind Tour of ARM Assembly for a more thorough ARM assembly reference.

Setting up a function for decomp

The first order of business is to pick a function to decompile. The functions are located in .s files within the asm directory, surrounded by an arm_func_start and arm_func_end (or their THUMB equivalents). As for which function to pick, this is up to you: perhaps you are new to function decomp and want a small function to ease into the process, or you are a hacker who wants to decomp a specific function to edit that function as C code instead of assembly, or you don't mind either way and just pick the first function in a file.

Once you pick a function, you'll need a workflow where you can write some C code, compile it to assembly, and compare the compiled assembly code with the original assembly code to see if they match. A common website for this is decomp.me.

Click "Start decomping" to begin setting up a function decomp environment ("scratch"). You can optionally sign into your GitHub account in decomp.me to keep track of all scratches you've created.

You'll be prompted to create a new scratch by filling in a couple of fields.

• Choose the DS (ARMv5TE) platform and the Pokémon Mystery Dungeon: Explorers of Sky preset. This will set up the compiler and flags to match the compiler used by the EoS decomp.
• In "Diff label", enter the name of the function you plan to decompile.
• In "Target assembly", place the entire function from the .s file, including the arm_func_start and arm_func_end.
• "Context" can contain definitions such as typedefs, structs, enums, and extern functions. It is technically not required, but using it will keep function source code clean when working on the scratch. You can grab a default context from nitro/types.h; exclude all the #ifdefs and take the typedefs along with these three #defines:

#define TRUE 1
#define FALSE 0
#define NULL ((void *)0)

If you filled in all fields correctly, "Create scratch" will create the scratch. The creation may fail if there are errors parsing the target assembly code, in which case you should review the parsing errors and the above instructions to see what went wrong. When the scratch is created, you'll be taken to the screen below.

You'll see an empty C function on the left and the assembly comparison on the right, including the target assembly you inputted during setup. With an empty C function, the compiled (current) assembly is only a bx lr to return from the function.

Decompiling a function

At this point, you can begin decompiling the function. A common approach is to start with the output from an automated decompiler, like the ones in Ghidra or IDA, and clean up the code from there. Alternatively, you can write C code from scratch by looking at the target assembly. If you haven't decompiled before, I recommend starting from scratch to learn the function decompiling process. You can then try using an automated decompiler on later functions to see if you prefer this approach.

If you want to follow along with the function used in this guide, here is the target assembly:

	arm_func_start ov29_022E0354
ov29_022E0354: ; 0x022E0354
	cmp r0, #0
	moveq r0, #0
	bxeq lr
	ldr r0, [r0]
	cmp r0, #0
	movne r0, #1
	moveq r0, #0
	and r0, r0, #0xff
	bx lr
	arm_func_end ov29_022E0354

Alternatively, if you are having trouble with the setup process, you can use this scratch to see how a scratch looks like when set up, including the target assembly, context, and compiler options.

Decompiling from scratch

When decompiling from scratch, you'll be reading through the target assembly and translating this to C code. This section will step through that process.

Let's start with the first three lines of assembly.

cmp r0, #0
moveq r0, #0
bxeq lr

r0 is immediately used without being assigned to, so it is a parameter to the function. You can arbitrarily pick a type (say, s32) to start with. Note that you should use the typedefs in the context instead of primitive types like int and long.

void ov29_022E0354(s32 param_0)

The parameter is compared to 0. If it is 0, then return 0 from the function.

if (param_0 == 0)
{
    return 0;
}

The function's return type can also be updated. Again, you can pick a type arbitrarily for now.

s32 ov29_022E0354(s32 param_0)

The cmp and moveq lines are now matched. bxeq isn't matched yet, but that's not surprising because there is no logic outside of the if statement yet to produce branching logic.

The next line of assembly is:

ldr r0, [r0]

The ldr indicates that r0 has an address to load from. This means that param_0 is probably a pointer.

s32 ov29_022E0354(s32 *param_0)

And now to add the load to the function body.

s32 param_0_value = *param_0;

Note that the line of code above is currently optimized out of the compiled assembly, since param_0_value is loaded by not used. Don't worry, it will be used by the time the end of the function is reached.

Now for the next lines of assembly:

cmp r0, #0
movne r0, #1
moveq r0, #0

r0 is now the dereferenced value of param_0. That value is compared to 0, outputting a 1 if the value is nonzero and a 0 if the value is 0.

s32 param_0_result;
if (param_0_value != 0)
{
    param_0_result = 1;
}
else
{
    param_0_result = 0;
}

This code can be simplified to:

s32 param_0_result = param_0_value != 0;

The code above remains optimized out of the compiled assembly. Even though param_0_value is used to assign param_0_result, param_0_result is not used, so this section of code is still considered unused. Since param_0_result is now used

The next line of assembly is:

and r0, r0, #0xff

Taken literally, this would be the following line of code:

param_0_result &= 0xFF;

We will revisit this later. For now, let's continue to the last line of assembly:

bx lr

The function returns after assigning a value to r0, which means the current value of r0 at this point is the return value. In this case, that would be param_0_result.

return param_0_result;

Now that param_0_result is used, the assembly for the above code is now generated.

Cool, a match! Technically this could be considered a stopping point, but let's look back at the code and clean it up a bit now that it matches.

The first item of note is the &= 0xFF. A bitwise and with 0xFF is special, as it takes the 8 least significant bits of the number. This indicates that param_0_result is likely a u8, with the and being an automatic cast added by the compiler. This means it is not necessary to add the &= manually, and the previous line can instead be:

u8 param_0_result = param_0_value != 0;

If you make this change, the compiled assembly still matches. This demonstrates an important point: there are often multiple ways to write C code that all produce the same assembly.

Note that the type is specifically an unsigned u8 type rather than a signed s8 type. A signed type often produces different assembly, as the signed bit needs to be handled specially. For example, if you change the u8 to an s8 here, the assembly will use lsl and asr to cast the value rather than and.

Since the returned value is a u8, the function's return type can be changed to that as well.

u8 ov29_022E0354(s32 *param_0)

Back in the if statement, param_0 is a pointer, so it can be compared to the NULL macro instead of 0 for clarity.

if (param_0 == NULL)

Note that the function only returns 1 or 0, which indicates that the return type is boolean. There is no specific bool8 type, so the u8 type will suffice here. However, the return 0 can be changed to use the boolean macros, turning into return FALSE. Remember to use the special boolean macros (TRUE and FALSE) instead of the regular boolean keywords (true and false).

Now for some more standard code cleanup. All of this:

s32 param_0_value = *param_0;
u8 param_0_result = param_0_value != 0;
return param_0_result;

can be simplified to:

return *param_0 != 0;

Finally, if you already know what the function does in the context of game functionality, or if you want to research the game to learn this, you can name the function and its variables.

The function is ready to add back to the decomp. Here is the completed scratch. Though before getting to that, let's go over the other decomp approach using an automated decompiler.

Starting with automated decompiler output

If you haven't already set up Ghidra, follow this guide to do so. Once Ghidra is set up, choose the overlay of the function you're decompiling and find the function within the overlay. Copy the decompiler output into decomp.me as a starting point.

Decompiled function in Ghidra

decomp.me with the Ghidra decompiler's output

Ghidra uses primitive C types, but the decomp uses custom typedefs for its types, so the primitive types should be converted to the custom types. For example, int becomes s32 and bool becomes u8. Also, use the macros FALSE and TRUE for booleans instead of false and true. Here's what the function looks like after cleaning up these types and macros, along with indentation and newlines.

The function now compiles successfully, but the compiled assembly does not match the target assembly. In the vast majority of cases, the automated decompiler will not produce matching output. You'll have to read the target assembly and the mismatches and see what changes can be made to the C code to possibly produce a match.

Breaking down the diff, the target assembly has the following:

cmp r0, #0
moveq r0, #0
bxeq lr

If r0 is 0, it is assigned to 0 as a return value, and the function exits.

Meanwhile, the current assembly has the following instead:

cmp r0, #0
beq 20
...
20: mov r0, #0

The logic is the same, but the mov r0, #0 operation is at the end of the function instead of right after the cmp. The two branches in this function (return *param_1 != 0 and return FALSE) are swapped in the assembly.

One way to change the compiled assembly is to flip the branches in the C code. Instead of this:

if (param_1 != (s32 *)0x0)
{
    return *param_1 != 0;
}
return FALSE;

Invert the if statement and swap the branching logic accordingly:

if (param_1 == (s32 *)0x0)
{
    return FALSE;
}
return *param_1 != 0;

That did the trick! The compiled and target assembly are now matching.

Note that not all functions will be this simple to match with automated decompiler output. Longer functions and more complicated logic give automated decompilers more trouble, and will take more tweaks and possibly large refactors to match. Some people prefer to avoid automated decompilers and stick to writing the function from scratch, and it is up to you to decide which approach you prefer.

Inserting a decompiled function

Now that the function has been decompiled, you'll need to add it into the decomp project and remove the corresponding raw assembly code. The repo has an extract_function script to move the function from assembly into C, or you can manually move it as a fallback.

You can check out this sample PR for an example of the changes needed to add a decompiled function to the decomp.

Using extract_function

To use extract_function, you will need Python 3 installed. The script will move a function from its assembly file to C files (.h and .c), splitting the assembly file in two if needed.

1. Make sure to have no uncommitted changes, in case something goes wrong with the script that needs a revert.
2. Run this command from the project root.

   python tools/extract_function/extract_function.py <asm_file> <function_header>
   

   • asm_file is the the assembly file containing the function you decompiled.
   • function_header is the C function header of the function you decompiled. Since it has spaces, remember to surround it in quotes.
3. The script will either create new C files for the function or add the function to an existing C file, depending on its location within the assembly file. Go to the .c file to find an empty function with the header you specified, and add the decompiled function code to this function.
4. Search for any externs in other files that reference the newly decompiled function. These externs can be removed and replaced with an #include to the new .h file.
5. Run make tidy and make to ensure that the project compiles and produces a matching ROM. If the ROM doesn't match, you can compare the mismatched files with the asmdiff tool or a hex editor to troubleshoot the issue.
6. If you want to decompile more functions, repeat the decomp process by finding a new function and creating a new scratch. If you are done, make a PR to the main pmd-sky repo.

Manually inserting a function

If extract_function doesn't work for you, or if you want to manually move the function from assembly to C for any other reason, here are the steps to do so. The end result is the same as using extract_function.

1. Create a .c in the src folder and a corresponding .h file in include.
   • Alternatively, if the function is at the beginning or end of the .s file, you may be able to add it to an existing C file. Check main.lsf to see which C (.o) file is right before/after the .s file.
2. Add the decompiled function to the new .c file, along with its corresponding header in the .h file.
3. Search for any externs in other files that reference the newly decompiled function. These externs can be removed and replaced with an #include to the new .h file.
4. Remove the function's assembly code from the .s file.
5. Split the .s file in two at the location where the function's assembly code was. The new .s file should be named according the the offset of the first function in the new file (e.g., overlay_29_022E0378.s). If the function you decompiled was at the beginning or end of the file, you can skip this step and step 6.
6. Split the corresponding .inc file (in asm/include) to the .s file you split.
7. Find the split file in main.lsf and add two files after it: the corresponding .o to the new .c file, and the newly split off .s file.
8. Run make tidy and make to ensure that the project compiles and produces a matching ROM. If the ROM doesn't match, you can compare the mismatched files with the asmdiff tool or a hex editor to troubleshoot the issue.
9. If you want to decompile more functions, repeat the decomp process by finding a new function and creating a new scratch. If you are done, make a PR to the main pmd-sky repo.

Function decomp tips

The example function in this guide shows the overall process of decompiling a function, though it doesn't cover every situation you may encounter. While it is impractical to go over every possible assembly construct and its C equivalent, here are some assorted tips.

• When working in decomp.me, any calls to other functions can be represented as extern functions, even if they are already decompiled in the decomp project. When adding the function to the decomp, you can replace these extern functions with #includes as needed, or leave the externs there for functions that have not been decompiled yet.
• Keep in mind some of the more eclectic C constructs like (static) inline functions, ternary statements, gotos, and C library functions like memcpy(). All of these may produce different assembly compared to more basic C constructs.
• If there are multiple places in a function with the same C code, the compiler may merge them into a single block of assembly and use unconditional branches to connect the different places to this block. This is known as a tail merge.
• There are times where you'll match everything in the function aside from which registers are used. For example, two variables are assigned to registers r4 and r5 respectively, but the target assembly assigns the first variable to r5 and the second to r4 instead. This is known in the decomp community as a regswap (register swap) or regalloc (register allocation) issue, and is one of the more frustrating issues to run into. There are a number of possible code changes to try and fix a regswap. Note that this list is not exhaustive.
  • Ensure that the register use is actually identical in functionality. It is easy to dismiss a difference as a regswap when it is actually a value being assigned incorrectly.
  • Reuse a local variable in multiple places, or split a local variable into multiple variables.
  • Move a local variable definition elsewhere in the function.
  • Add or collapse struct/array accesses with local variables.
  • Assign macros and enums to local variables.
  • Play with the structure of conditionals and loops.
  • Surround parts of the function with no-op do while(0) loops.

Permuter

Another option for dealing with regswaps is the decomp permuter. This program will randomly change a function with some of the regswap tricks above to try matching the function.

For pmd-sky, there are a couple of steps to set up the permuter.

1. Clone the permuter repo and follow the README.md to install the required dependencies. The permuter is written in Python, so you'll need to have Python installed too.
2. Create a directory in the repo to contain the function you want to permute.
3. Add a base C file (e.g., base.c) to the new folder with the context and function code that you have so far. Note that the permuter is a little finicky and doesn't recognize certain C constructs like comments and complex casts, so you may need to make some code changes later.
4. Add a target assembly file (e.g., target.s) with the function ASM, excluding arm_func_start and arm_func_end.
5. Add a function.txt file containing the name of the function you are decompiling.
6. Add a compile.sh Bash script with the following contents. This will tell the permuter how to run the compiler (mwccarm) on the permuted functions it generates.

cd <decomp directory>
wine ./tools/mwccarm/2.0/sp2p2/mwccarm.exe -O4,s -DPM_KEEP_ASSERTS -DSDK_ARM9 -DSDK_CODE_ARM -DSDK_FINALROM -enum int -lang c99 -Cpp_exceptions off -gccext,on -proc arm946e -msgstyle gcc -gccinc -interworking -inline on,noauto -char signed -W all -W pedantic -W noimpl_signedunsigned -W noimplicitconv -W nounusedarg -W nomissingreturn -W error -gccdep -MD -c -o $3 $1

7. Run the compile.sh script on the base C file with the arguments <base C file> -o <base object file>. Name the object file the same as the base C file, except with a .o extension (e.g., base.o). Since the script has a cd command in it (required by mwccarm), pass in the absolute paths of both the .c and .o file.
8. Assemble the target assembly file with the command rm-none-eabi-as -mthumb -march=armv5te <target assembly file> -o <target object file>. Like with the compile command, name the target object file the same as the target assembly file, except with .o (e.g., target.o).
9. Run the permuter with permuter.py <directory> --stop-on-zero, where <directory> is the directory you created in step 2.

The permuter will run until it finds a function permutation that matches the target assembly. It will also output any permutations that are a closer match to the target than your base C code. There is no guarantee that it will find a match, but it is worth giving a shot if you are having trouble with a regswap or any other ASM difference that is functionally equivalent.

Note that the permuter only makes changes that are functionally equivalent to the base function. If your base function code has a bug that produces different behavior to the ASM you are matching, the permuter will not fix it.

Asking for help

If you have trouble matching a function, you can ask for help on the pret Discord's #asm2c channel. Post the link to your decomp.me scratch on the channel, and other people can fork the scratch to experiment on their own. You'll see a notification on decomp.me if anyone successfully matches your function. You can also browse through previously matched functions for inspiration on tricks used by others to produce matching assembly. As you continue to decompile more, you can try helping others in the channel, which in turn will help you practice and gain exposure to the nuances of the decompilation process.

If all else fails, leave the function in assembly within the decomp project and add a comment linking to your decomp.me scratch. This provides others with a starting point to try matching the function later on.