modbase merge (#9)

* Add more autosplit points for Jak 2 Any% (#3639) Goes with open-goal/speedrunning#21 Based on some of the autosplit points from the [comgold sheet](https://docs.google.com/spreadsheets/d/1ZtceX0ZxCLkufVFQuCgVxptvmVgp6deHwPYWxUPi258/edit?gid=0#gid=0), plus one after talking to samos to give a clean split for timing the deload. ``` (tomb-poles-poles uint8) ;; left tomb/daxter boulder start (fortress-save-friends-introduction uint8) ;; talk to torn (before rescue friends) (sewer-escort-get-gun uint8) ;; get peacemaker (forest-protect-introduction uint8) ;; talk to onin (protect samos) (forest-protect-meeting uint8) ;; talk to samos (protect samos) ``` * ci: simplify release workflow (#3642) * Update cut-release.yaml * fix stop-sound-file (and split out stop-all-sounds) (OpenGOAL-Mods#104) * fix bad map usage * binaries to be safe * play audio file relative to custom_assets/<game>/audio/ * fix main music and things * modversion jak 2 * Add a has-landed to SR text (OpenGOAL-Mods#103) * Add a has-landed to SR text * Needs cleanup but pushing to test * cleanup * cleanup --------- Co-authored-by: dallmeyer <2515356+dallmeyer@users.noreply.github.com> * change has-landed logic * Update .gitignore * CI: Periodic Controller Database Update (#3647) Updating Controller Database Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com> * Update cut-release.yaml * build(deps): bump hendrikmuhs/ccache-action from 1.2.13 to 1.2.14 (#3650) Bumps [hendrikmuhs/ccache-action](https://github.com/hendrikmuhs/ccache-action) from 1.2.13 to 1.2.14. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/hendrikmuhs/ccache-action/releases">hendrikmuhs/ccache-action's releases</a>.</em></p> <blockquote> <h2>v1.2.14</h2> <h2>What's Changed</h2> <ul> <li>Add sccache to PATH after installation by <a href="https://github.com/kendalharland"><code>@kendalharland</code></a> in <a href="https://redirect.github.com/hendrikmuhs/ccache-action/pull/204">hendrikmuhs/ccache-action#204</a></li> <li>Make ccache-action respect environment variables by <a href="https://github.com/TrentHouliston"><code>@TrentHouliston</code></a> in <a href="https://redirect.github.com/hendrikmuhs/ccache-action/pull/217">hendrikmuhs/ccache-action#217</a></li> <li>updates</li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/kendalharland"><code>@kendalharland</code></a> made their first contribution in <a href="https://redirect.github.com/hendrikmuhs/ccache-action/pull/204">hendrikmuhs/ccache-action#204</a></li> <li><a href="https://github.com/cclauss"><code>@cclauss</code></a> made their first contribution in <a href="https://redirect.github.com/hendrikmuhs/ccache-action/pull/213">hendrikmuhs/ccache-action#213</a></li> <li><a href="https://github.com/TrentHouliston"><code>@TrentHouliston</code></a> made their first contribution in <a href="https://redirect.github.com/hendrikmuhs/ccache-action/pull/217">hendrikmuhs/ccache-action#217</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/hendrikmuhs/ccache-action/compare/v1...v1.2.14">https://github.com/hendrikmuhs/ccache-action/compare/v1...v1.2.14</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/hendrikmuhs/ccache-action/commit/ed74d11c0b343532753ecead8a951bb09bb34bc9"><code>ed74d11</code></a> Bump <code>@types/node</code> from 22.0.0 to 22.1.0 (<a href="https://redirect.github.com/hendrikmuhs/ccache-action/issues/222">#222</a>)</li> <li><a href="https://github.com/hendrikmuhs/ccache-action/commit/a92dd99d2cf20a1db8898b00bb383b234fb1cf15"><code>a92dd99</code></a> Bump <code>@types/node</code> from 20.14.11 to 22.0.0 (<a href="https://redirect.github.com/hendrikmuhs/ccache-action/issues/220">#220</a>)</li> <li><a href="https://github.com/hendrikmuhs/ccache-action/commit/aa7d29411285c29f578109e54b7a8d8155c2fbb3"><code>aa7d294</code></a> Bump typescript from 5.5.3 to 5.5.4 (<a href="https://redirect.github.com/hendrikmuhs/ccache-action/issues/218">#218</a>)</li> <li><a href="https://github.com/hendrikmuhs/ccache-action/commit/6f0874030891bf49d844fff92b862568f093dabe"><code>6f08740</code></a> Make ccache-action respect environment variables (<a href="https://redirect.github.com/hendrikmuhs/ccache-action/issues/217">#217</a>)</li> <li><a href="https://github.com/hendrikmuhs/ccache-action/commit/ed979992cda44142d976add1d5a7d6f39f7e8b67"><code>ed97999</code></a> Bump <code>@types/node</code> from 20.14.10 to 20.14.11 (<a href="https://redirect.github.com/hendrikmuhs/ccache-action/issues/216">#216</a>)</li> <li><a href="https://github.com/hendrikmuhs/ccache-action/commit/ca1e5062f3378412bbfeb780d1ebe3c2a4913081"><code>ca1e506</code></a> Bump actions/checkout from 2 to 4 (<a href="https://redirect.github.com/hendrikmuhs/ccache-action/issues/214">#214</a>)</li> <li><a href="https://github.com/hendrikmuhs/ccache-action/commit/069136ab7ab2267ea6624fde73f80d7d472d323e"><code>069136a</code></a> Bump <code>@types/node</code> from 20.14.9 to 20.14.10 (<a href="https://redirect.github.com/hendrikmuhs/ccache-action/issues/212">#212</a>)</li> <li><a href="https://github.com/hendrikmuhs/ccache-action/commit/3cf745af56c860cc76c89ffd830efec6aef03b56"><code>3cf745a</code></a> Bump typescript from 5.5.2 to 5.5.3 (<a href="https://redirect.github.com/hendrikmuhs/ccache-action/issues/211">#211</a>)</li> <li><a href="https://github.com/hendrikmuhs/ccache-action/commit/9a0cc152966f2c3f3df86a6e0364da1608924006"><code>9a0cc15</code></a> Keep GitHub Actions up to date with GitHub's Dependabot (<a href="https://redirect.github.com/hendrikmuhs/ccache-action/issues/213">#213</a>)</li> <li><a href="https://github.com/hendrikmuhs/ccache-action/commit/b7c0e162a73e852cdd80bd368aa77e7801fce009"><code>b7c0e16</code></a> Bump <code>@types/node</code> from 20.14.8 to 20.14.9 (<a href="https://redirect.github.com/hendrikmuhs/ccache-action/issues/210">#210</a>)</li> <li>Additional commits viewable in <a href="https://github.com/hendrikmuhs/ccache-action/compare/v1.2.13...v1.2.14">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=hendrikmuhs/ccache-action&package-manager=github_actions&previous-version=1.2.13&new-version=1.2.14)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Create `abutton-idx` enum, replace magic numbers across codebase (#3646) Found out the hard way that the `abutton` array for pressure sensitivity uses a different order than the `pad-buttons` enum, so I created a new enum for abutton indexing. I replaced any magic numbers across the codebase where it made sense, e.g. `(-> *cpad-list* cpads (-> self control unknown-cpad-info00 number) abutton 6)` becomes `(-> *cpad-list* cpads (-> self control unknown-cpad-info00 number) abutton (abutton-idx x))` * jak3: fix vagdir bit-field for msvc + clang on windows (#3643) * add stellars normals fix for merc_replacements * Update default-menu.gc flooooot * format before vanilla merge * regen binaries * support common_art_groups --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Tyler Wilding <xTVaser@users.noreply.github.com> Co-authored-by: ZedB0T <89345505+Zedb0T@users.noreply.github.com> Co-authored-by: OpenGOAL Bot <99294829+OpenGOALBot@users.noreply.github.com> Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Dillon Pentz <dillon@vodbox.io>
OpenGOAL-Mods · Sep 2, 2024 · 6eb2354 · 6eb2354
1 parent ab54335
commit 6eb2354
Show file tree

Hide file tree

Showing 90 changed files with 2,213 additions and 2,002 deletions.
diff --git a/.github/workflows/compiler-output-check.yaml b/.github/workflows/compiler-output-check.yaml
@@ -26,7 +26,7 @@ jobs:
             libxi-dev zip ninja-build libgl1-mesa-dev libssl-dev
 
       - name: Setup sccache
-        uses: hendrikmuhs/ccache-action@v1.2.13
+        uses: hendrikmuhs/ccache-action@v1.2.14
         with:
           variant: sccache
           key: linux-ubuntu-20.04--Release-linux-clang-asan-${{ github.sha }}

diff --git a/.github/workflows/cut-release.yaml b/.github/workflows/cut-release.yaml
@@ -28,10 +28,7 @@ jobs:
       metadataAuthors: "zed,barg,MikeGamePro"
       metadataTags: "speedrunning,challenge,gameplay-mod,rng"
       metadataWebsiteUrl: "https://github.com/OpenGOAL-Mods/OG-Randomizer/blob/main/README.md"
-      skipMacOS: true
-      skipLinux: true
-      toolingBinaryDir: "out/build/Release/bin"
+      toolingRepo: "OpenGOAL-Mods/OG-Mod-Base"
       gameAssetsDir: "game/assets"
-      copyEntireBinaryDir: true
     secrets:
       token: ${{ secrets.GITHUB_TOKEN }}
diff --git a/.github/workflows/linux-build-clang.yaml b/.github/workflows/linux-build-clang.yaml
@@ -35,7 +35,7 @@ jobs:
             libudev-dev libopenal-dev libflac-dev libogg-dev libvorbis-dev
 
       - name: Setup sccache
-        uses: hendrikmuhs/ccache-action@v1.2.13
+        uses: hendrikmuhs/ccache-action@v1.2.14
         with:
           variant: sccache
           key: linux-ubuntu-20.04-${{ inputs.cachePrefix }}-${{ inputs.cmakePreset }}-${{ github.sha }}

diff --git a/.github/workflows/linux-build-gcc.yaml b/.github/workflows/linux-build-gcc.yaml
@@ -34,7 +34,7 @@ jobs:
             sudo update-alternatives --set g++ /usr/bin/g++-10
 
       - name: Setup sccache
-        uses: hendrikmuhs/ccache-action@v1.2.13
+        uses: hendrikmuhs/ccache-action@v1.2.14
         with:
           variant: sccache
           key: linux-ubuntu-20.04-${{ inputs.cachePrefix }}-${{ inputs.cmakePreset }}-${{ github.sha }}

diff --git a/.github/workflows/macos-build-arm.yaml b/.github/workflows/macos-build-arm.yaml
@@ -27,7 +27,7 @@ jobs:
         run: arch -arm64 brew install cmake ninja
 
       - name: Setup sccache
-        uses: hendrikmuhs/ccache-action@v1.2.13
+        uses: hendrikmuhs/ccache-action@v1.2.14
         with:
           variant: sccache
           key: macos-12-${{ inputs.cachePrefix }}-${{ inputs.cmakePreset }}-${{ github.sha }}

diff --git a/.github/workflows/macos-build.yaml b/.github/workflows/macos-build.yaml
@@ -28,7 +28,7 @@ jobs:
         run: brew install cmake nasm ninja
 
       - name: Setup sccache
-        uses: hendrikmuhs/ccache-action@v1.2.13
+        uses: hendrikmuhs/ccache-action@v1.2.14
         with:
           variant: sccache
           key: macos-12-${{ inputs.cachePrefix }}-${{ inputs.cmakePreset }}-${{ github.sha }}

diff --git a/.github/workflows/windows-build-clang.yaml b/.github/workflows/windows-build-clang.yaml
@@ -36,7 +36,7 @@ jobs:
           }
 
       - name: Setup sccache
-        uses: hendrikmuhs/ccache-action@v1.2.13
+        uses: hendrikmuhs/ccache-action@v1.2.14
         with:
           variant: sccache
           key: windows-2022-${{ inputs.cachePrefix }}-${{ inputs.cmakePreset }}-${{ github.sha }}

diff --git a/.github/workflows/windows-build-msvc.yaml b/.github/workflows/windows-build-msvc.yaml
@@ -31,7 +31,7 @@ jobs:
           }
 
       - name: Setup sccache
-        uses: hendrikmuhs/ccache-action@v1.2.13
+        uses: hendrikmuhs/ccache-action@v1.2.14
         with:
           variant: sccache
           key: windows-2022-${{ inputs.cachePrefix }}-${{ inputs.cmakePreset }}-${{ github.sha }}

diff --git a/.gitignore b/.gitignore
@@ -62,19 +62,9 @@ node_modules/
 # custom_assets/jak3/texture_replacements/*
 
 # merc replacements
-custom_assets/jak1/merc_replacements/*
-custom_assets/jak2/merc_replacements/*
-custom_assets/jak3/merc_replacements/*
-
-# merc replacements
-custom_assets/jak1/merc_replacements/*
-custom_assets/jak2/merc_replacements/*
-custom_assets/jak3/merc_replacements/*
-
-# merc replacements
-custom_assets/jak1/merc_replacements/*
-custom_assets/jak2/merc_replacements/*
-custom_assets/jak3/merc_replacements/*
+# custom_assets/jak1/merc_replacements/*
+# custom_assets/jak2/merc_replacements/*
+# custom_assets/jak3/merc_replacements/*
 
 # generated cmake files
 svnrev.h
@@ -113,4 +103,4 @@ unifont-15.0.03.ttf
 out/jak2/fr3.rar
 
 *.diff
-goalc-report.html
+goalc-report.html
diff --git a/decompiler/config.cpp b/decompiler/config.cpp
@@ -329,6 +329,10 @@ Config make_config_via_json(nlohmann::json& json) {
         inputs_json.at("animated_textures").get<std::unordered_set<std::string>>();
   }
 
+  if (json.contains("common_art_groups")) {
+    config.common_art_groups = json.at("common_art_groups").get<std::unordered_set<std::string>>();
+  }
+
   if (inputs_json.contains("common_tpages")) {
     config.common_tpages = inputs_json.at("common_tpages").get<std::unordered_set<int>>();
   }

diff --git a/decompiler/config.h b/decompiler/config.h
@@ -168,6 +168,7 @@ struct Config {
   std::unordered_map<std::string, int> bad_format_strings;
 
   std::unordered_set<std::string> animated_textures;
+  std::unordered_set<std::string> common_art_groups;
   std::unordered_set<int> common_tpages;
 
   std::vector<std::string> levels_to_extract;

diff --git a/decompiler/config/jak1/all-types.gc b/decompiler/config/jak1/all-types.gc
@@ -287,25 +287,41 @@
   )
 
 (defenum pad-buttons
-    :bitfield #t
-    :type uint32
-    (select   0)
-    (l3       1)
-    (r3       2)
-    (start    3)
-    (up       4)
-    (right    5)
-    (down     6)
-    (left     7)
-    (l2       8)
-    (r2       9)
-    (l1       10)
-    (r1       11)
-    (triangle 12)
-    (circle   13)
-    (x        14)
-    (square   15)
-    )
+  :bitfield #t
+  :type uint32
+  (select   0)
+  (l3       1)
+  (r3       2)
+  (start    3)
+  (up       4)
+  (right    5)
+  (down     6)
+  (left     7)
+  (l2       8)
+  (r2       9)
+  (l1       10)
+  (r1       11)
+  (triangle 12)
+  (circle   13)
+  (x        14)
+  (square   15)
+  )
+
+(defenum abutton-idx
+  :type uint8
+  (right 0)
+  (left 1)
+  (up 2)
+  (down 3)
+  (triangle 4)
+  (circle 5)
+  (x 6)
+  (square 7)
+  (l1 8)
+  (r1 9)
+  (l2 10)
+  (r2 11)
+  )
 
 (defenum gs-psm
     :bitfield #f

diff --git a/decompiler/config/jak1/jak1_config.jsonc b/decompiler/config/jak1/jak1_config.jsonc
@@ -125,6 +125,12 @@
   // whether or not to dump out streamed audio files to decompiler_out/<game>/audio
   "rip_streamed_audio": false,
 
+  // art groups that should always be possible to access, e.g. "common_art_groups": ["eichar-racer+0-ag", "racer-ag", "ef-plane-ag"]
+  "common_art_groups": [],
+
+  // tpages that should always be possible to access, e.g. "common_tpages": [1119],
+  "common_tpages": [],
+
   ////////////////////////////
   // PATCHING OPTIONS
   ////////////////////////////

diff --git a/decompiler/config/jak2/all-types.gc b/decompiler/config/jak2/all-types.gc
@@ -4202,6 +4202,22 @@ and the state of the VU memory buffers at the end of the bucket."
     (confirm 24)
     )
 
+(defenum abutton-idx
+  :type uint8
+  (right 0)
+  (left 1)
+  (up 2)
+  (down 3)
+  (triangle 4)
+  (circle 5)
+  (x 6)
+  (square 7)
+  (l1 8)
+  (r1 9)
+  (l2 10)
+  (r2 11)
+  )
+
 (defenum mouse-buttons
     :bitfield #t
     :type uint32

diff --git a/decompiler/config/jak3/all-types.gc b/decompiler/config/jak3/all-types.gc
@@ -4603,6 +4603,24 @@
   )
 ;; ---pad:pad-buttons
 
+;; +++pad:abutton-idx
+(defenum abutton-idx
+  :type uint8
+  (right 0)
+  (left 1)
+  (up 2)
+  (down 3)
+  (triangle 4)
+  (circle 5)
+  (x 6)
+  (square 7)
+  (l1 8)
+  (r1 9)
+  (l2 10)
+  (r2 11)
+  )
+;; ---pad:abutton-idx
+
 ;; +++pad:mouse-buttons
 (defenum mouse-buttons
   :bitfield #t

diff --git a/decompiler/data/streamed_audio.cpp b/decompiler/data/streamed_audio.cpp
@@ -164,13 +164,16 @@ AudioDir read_audio_dir(const decompiler::Config& config, const fs::path& path)
         u64 data;
         struct {
           u64 name : 42;
-          bool stereo : 1;
-          bool international : 1;
-          u8 param : 4;
+          u64 stereo : 1;
+          u64 international : 1;
+          u64 param : 4;
           u64 offset : 16;
         };
       };
     };
+
+    static_assert(sizeof(DirEntryJak3) == sizeof(u64));
+
     dir = reader.read<VagDirJak3>();
     ASSERT(dir.id[0] == 0x41574756);
     ASSERT(dir.id[1] == 0x52494444);

diff --git a/decompiler/level_extractor/extract_level.cpp b/decompiler/level_extractor/extract_level.cpp
@@ -257,7 +257,8 @@ void extract_common(const ObjectFileDB& db,
                     const TextureDB& tex_db,
                     const std::string& dgo_name,
                     const fs::path& output_folder,
-                    const Config& config) {
+                    const Config& config,
+                    const std::vector<std::string>& all_dgo_names) {
   if (db.obj_files_by_dgo.count(dgo_name) == 0) {
     lg::warn("Skipping common extract for {} because the DGO was not part of the input", dgo_name);
     return;
@@ -278,6 +279,35 @@ void extract_common(const ObjectFileDB& db,
   add_all_textures_from_level(tfrag_level, "ARTSPOOL", tex_db);
   extract_art_groups_from_level(db, tex_db, {}, "ARTSPOOL", tfrag_level, art_group_data);
 
+  // copy in any art groups that were requested to be common
+  if (config.common_art_groups.size() > 0) {
+    std::unordered_set<std::string> art_groups_made_common;
+    for (const std::string& lvl_dgo_name : all_dgo_names) {
+      // exit early if we've found everything
+      if (config.common_art_groups.size() == art_groups_made_common.size()) {
+        lg::info("Found all requested art groups to be made common!");
+        break;
+      }
+
+      lg::info("Looking for common art groups in {}", lvl_dgo_name);
+      auto tex_remap = extract_tex_remap(db, lvl_dgo_name);
+      if (db.obj_files_by_dgo.count(lvl_dgo_name)) {
+        const auto& files = db.obj_files_by_dgo.at(lvl_dgo_name);
+        for (const auto& file : files) {
+          if (!art_groups_made_common.contains(file.name) && config.common_art_groups.contains(file.name)) {
+            lg::info("Art group {} was requested to be made common, we found it in {}!", file.name,
+                     lvl_dgo_name);
+            const auto& ag_file = db.lookup_record(file);
+            extract_merc(ag_file, tex_db, db.dts, tex_remap, tfrag_level, false, db.version());
+            extract_joint_group(ag_file, db.dts, db.version(), art_group_data);
+            // track found art groups so we don't borther re-processing in a later level
+            art_groups_made_common.insert(file.name);
+          }
+        }
+      }
+    }
+  }
+
   std::set<std::string> textures_we_have;
 
   // put _all_ index textures in common.
@@ -378,7 +408,7 @@ void extract_all_levels(const ObjectFileDB& db,
                         const std::string& common_name,
                         const Config& config,
                         const fs::path& output_path) {
-  extract_common(db, tex_db, common_name, output_path, config);
+  extract_common(db, tex_db, common_name, output_path, config, dgo_names);
   auto entities_dir = file_util::get_jak_project_dir() / "decompiler_out" /
                       game_version_names[config.game_version] / "entities";
   file_util::create_dir_if_needed(entities_dir);

diff --git a/decompiler/level_extractor/merc_replacement.cpp b/decompiler/level_extractor/merc_replacement.cpp
@@ -131,9 +131,9 @@ void merc_convert_replacement(MercSwapData& out,
     x.pos[0] = y.x;
     x.pos[1] = y.y;
     x.pos[2] = y.z;
-    x.normal[0] = copy_from.normal[0];
-    x.normal[1] = copy_from.normal[1];
-    x.normal[2] = copy_from.normal[2];
+    x.normal[0] = in.normals.at(i).x();
+    x.normal[1] = in.normals.at(i).y();
+    x.normal[2] = in.normals.at(i).z();
     x.weights[0] = copy_from.weights[0];
     x.weights[1] = copy_from.weights[1];
     x.weights[2] = copy_from.weights[2];