Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add menu + stage music support #386

Merged
merged 101 commits into from
Jul 28, 2023

Conversation

DarylPinto
Copy link
Contributor

@DarylPinto DarylPinto commented Jul 4, 2023

What is this?

In a nutshell, this feature brings music back to Slippi Online. By checking the following box, game music will be restored:
image

How does it work?

When the checkbox is toggled on, a pair of threads that are completely detached from emulation are spawned. These threads read from game memory to determine which music tracks should be played. When a track should play, a chunk of the iso containing the song is loaded into memory, decoded into audio and played back over the default audio device. No external audio files are used and nothing is ever written to disk.

Features

  • Menu music
  • All stage music
  • Alternate stage music1
  • "Versus" splash screen jingle
  • Ranked stage striking music
  • Auto-ducking music volume when pausing the game
  • Break the Targets + Home Run Contest music
  • Controlling volume with the in-game options and Dolphin's audio volume slider
  • Lottery menu music
  • Single player mode2
  1. Alternate stage music has a 12.5% chance of playing. Holding triggers to force alternate tracks to play is not supported.

  2. In some 1P modes, stage music will differ from vs mode. This behavior is not supported by jukebox. Additionally, the following songs will not play:

    • Classic "Stage Clear" Jingle
    • Classic "Continue?" and "Game Over" Jingles
    • Credits music
    • Post-credits congratulations fmv music

Limitations

  • The WASAPI exclusive audio backend is not currently supported
  • Since music is not emulated by Dolphin, there are discrepancies from vanilla - music in the sound test, for example, does not work.

Technical Details

The code that adds music support can be found here:

This pull request includes the addition of a Rust submodule, a first for the project. See this comment for details about Rust's integration into Slippi.

ryanmcgrath and others added 30 commits March 28, 2023 03:13
…back-specific code, rewrite README for rust extensions slightly
…to Rust, expose more methods on Rust side for control, rebuild logger layer subscriber
- A generic Rust dependencies log container that we can use if we need
  to ever inspect dependency tracing logs.
- Moves the AudioCommon::GetVolume call out of the namespace for now
  since I want someone (or me, when I get more time) to verify that
  being inside a C++ namespace is safe.
- Remove once_cell as a dependency now that this is pinned to `1.70.0`.
…n about what is supported in these files...?
…rking directory for pre-build Rust step so that toolchain file is actually detected
@ryanmcgrath
Copy link
Collaborator

ryanmcgrath commented Jul 4, 2023

🐸x 🦀

Just adding on some notes and context to DRL's amazing work to get music back into the game.

In order to facilitate using a Rust lib, I had to make some changes to the Dolphin codebase and build process. A general changelog is below but can expand on anything as needed:

  • The Rust codebase is held within Externals/SlippiRustExtensions. It gets built into a dylib that is shipped with the app(s) and linked at runtime, etc.
  • Communication between Rust and C++ is done via the C API; the ffi module in Externals/SlippiRustExtensions exposes the "public" API for this. Auto-generated C headers can be found in Externals/SlippiRustExtensions/ffi/includes.
  • For platforms that use CMake (macOS, Linux), I updated the CMake build pipeline to use corrosion for building the project. This change is relatively straightforward.
  • For Windows, I've updated the Visual Studio solution files to support building the Rust project and including it. This is more custom in nature but I think it works fine, and it's been tested by DRL as well.
  • There's now a Rust SlippiEXIDevice that's held by the C++ CEXISlippi class; this is intentional so that the lifecycle on the Rust side meshes well. This is currently blank except for holding the Jukebox, but I figured doing this is useful in case there's a desire to continue doing things in Rust - it provides a sane structure for iterating here.
  • AN IMPORTANT CHANGE: in order to get access to certain memory offsets from the Rust side, I've altered the order of Hardware initialization. This moves Memory and ExpansionInterface so that Memory is always around when ExpansionInterface is started or destroyed. This appears to be fine but should be confirmed again in the Mainline move.
  • There are changes to the C++ EXI Device to facilitate creating/destroying the Rust EXI Device (mapping to C++ instantiation/destruction) and for toggling the Jukebox.
  • The Rust library has support for logging via Dolphin's logging pieces.
    • The Dolphin LogContainer class has been modified to hold its own LogType and to accept a flag marking it as sourcing logs from the Rust library. If that flag is true, a LogContainer will register itself with the Rust side in order to handle ferrying logs across.
    • The Rust side is a custom tracing-subscriber that interops with the above change to dispatch logs to their correct container.
    • There's more complete documentation in the ffi module README on this, as well as how to add new logging containers.
  • DRL's Jukebox needs a way to read the current volume from Dolphin, so there's a new AudioCommon GetVolume function that is passed in to the Rust library for the Jukebox to use.
  • There's a toggle in the GUI for whether the Jukebox should be enabled or not; this toggle is not there in the Playback builds and the Jukebox itself is not enabled for Playback builds at this time.
  • The CI pipeline has been updated to support installing Rust, building and bundling everything together. It will auto-run a check for formatting errors and will run tests in each OS environment, so we can add tests if we find we need them.

(Edit) one more bit of context:

  • This PR modifies the CI build to always publish an artifact for macOS runners, even if not signing/notarizing - it will only attempt the signing/notarizing if CI keys exist, otherwise it publishes an unsigned package for testing.

@ryanmcgrath
Copy link
Collaborator

(@eigenform may be a good candidate for extra eyes on this with regards to the C++/C/Rust interactions)

@NikhilNarayana NikhilNarayana changed the title add menu + stage music support for online play feat: add menu + stage music support Jul 4, 2023
@DarylPinto DarylPinto force-pushed the experimental/rust branch from 10edb02 to 035f883 Compare July 7, 2023 21:52
@DarylPinto DarylPinto force-pushed the experimental/rust branch from 1580bd0 to 620d409 Compare July 8, 2023 18:08
@NikhilNarayana NikhilNarayana merged commit e29fe70 into project-slippi:slippi Jul 28, 2023
rapito pushed a commit to rapito/Ishiiruka that referenced this pull request Aug 31, 2023
* Add support for Rust in CMake via Corrosion

* Tweak .gitignore to handle Rust target dir

* It's 3AM and I have no desire to deal with this

* Initial scaffolding for loading Rust library in Dolphin

* Expose the Dolphin logging mechanisms to Rust via some callback trickery

* Attempt patching Core.vcxproj to support building Rust dylib

* Attempt patching Core.vcxproj to support building Rust dylib, take 2 OR

* Instruct the build to actually use the steps

* Attempt the older style vcxproj definition format

* Push build targets to end

* Just don't use va_args when calling through from Rust

* Enable Windows project linking, move to generic project structure, start moving out of HW

* Add note to README about Rust requirement

* Update slippi-rust-extensions to have proper EXI Device skeleton

* Attach shadow EXI device to cpp EXI device, add feature flag for playback-specific code, rewrite README for rust extensions slightly

* Remove unused log file

* Ongoing logging work

* Tweak Dolphin LogContainer to auto forward enabled/level status over to Rust, expose more methods on Rust side for control, rebuild logger layer subscriber

* Remove debug flag for release mode but add note about it

* Reorganize module definitions, pass in sampler handler fn for SoundStream

* Fix log target

* Rename to SlippiRustExtensions, separate out into cargo workspace, rename General Rust log

* Tweak logging layer so that we don't double-allocate strings on all log messages, properly surface log locations

* cargo fmt

* initial port of slippi-jukebox code

* minor cleanup and leverage channels rather than atomic bool for stopping
threads

* Add config option for enabling/disabling Jukebox

* Invert shutdown order for Memory and ExpansionInterface so we avoid a null ptr race in the Jukebox, as we need Memory to still be valid at Jukebox shutdown

* update dolphin additional include dirs to facilitate new slippi config pane changes

* Ensure Core is running before trying to find an EXI device, as ExpansionInterface isn't initialized unless Core is running - fixes a crash on bad access

* Expose streaming sampler to jukebox, disable DVDInterface streaming sampler pushes to avoid null data being pushed in

* Expose setters for streaming sample rate and streaming volume to Rust

* jukebox: renaming variables for clarity

* try to use new jukebox sample functions

* jukebox: add support for all star rest area

* jukebox: add support for adventure mode field stages

* jukebox: continuously send chunks of pcm data to dolphin's audio mixer

* Audio somewhat coming through now, albeit with pops and pitch issues...

* Force-log samples to wav

* jukebox: switch from dolphin mixer back to rodio for music playback

* Mark doc example as notest

* Attempt an initial CI pass

* Tinkering with CI

* Specify CI working directory for cargo fmt

* Specify CI working directory for cargo fmt

* Specify CI working directory for cargo fmt

* Specify CI working directory for cargo fmt

* Specify CI working directory for cargo fmt

* Specify CI working directory for cargo fmt

* [CI] More permissive for compiling code

* jukebox: replace rand with fastrand

* jukebox: dont use static memory for menu & tournament tracks

* Attempt to resolve Windows playback lib loading oddity

* Add a method for grabbing the current volume level

* Have Dolphin pass over the iso path and a getter for the volume level

* Cargo fmt pass

* Include rustfmt.toml

* Remove the Jukebox config option from playback builds entirely, do not start the jukebox if we're in WASAPI Exclusive mode

* Ix-nay the bad check I copied by accident, lol

* SlippiRustExtensions: add build instructions for windows

* jukebox: make proper use of dolphin's volume + remove unused dependencies

* SlippiRustExtensions(readme): simplify suggested out-of-band build command

* jukebox: remove 'anyhow' dependency

* jukebox: simplify read_dolphin_state fn

* jukebox: reduce hps fingerprint size for track matching

* jukebox: remove 'bus' dependency and improve comments

* jukebox: add readme

* jukebox: add description field to Cargo.toml

* README cleanup, extra writeup for logcontainer creation

* Cleanup unused imports

* Rename DolphinState to DolphinGameState

* Pin to Rust 1.70.0

* Re-enable SFX, stop force-dumping WAV DTK audiograph logs

* Revert attempted change of disabling DTK reads

* Revise documentation surrounding logging infrastructure changes

* Ensure the Volume getter is outside of a C++ namespace, just to try and be careful with ABI oddities

* Add an extra LogContainer.

- A generic Rust dependencies log container that we can use if we need
  to ever inspect dependency tracing logs.
- Moves the AudioCommon::GetVolume call out of the namespace for now
  since I want someone (or me, when I get more time) to verify that
  being inside a C++ namespace is safe.
- Remove once_cell as a dependency now that this is pinned to `1.70.0`.

* Fix the typo, because C++

* Additional README contexts

* ifndef PLAYBACK for GUI toggles

* Always publish an artifact for macOS even if we're not signing and notarizing

* Try changing into the directory - why is there so little documentation about what is supported in these files...?

* Pin toolchain in CI to 1.70.0, update Visual Studio to change into working directory for pre-build Rust step so that toolchain file is actually detected

* Opt for a root symlink to the toolchain config per Nikki's idea, comment out forced 1.70.0 in CI build flow

* Revert symlink toolchain to see if there's a CI bug for Linux

* jukebox: reduce music volume by 20%

* jukebox: link to hps_decode crate in the docs

* ci: sed out rust version from rust-toolchain.toml

* ci: add id field for rust_ver

* jukebox: stop blocking the main thread when scanning iso for tracks

* jukebox: respect melee's volume setting when restarting jukebox during emulation

* jukebox: improve code readability

* jukebox: leverage disc filesystem table to locate .hps files

* jukebox: minor fst parsing code improvements

* remove '.unwrap()' from Jukebox destructor

---------

Co-authored-by: Ryan McGrath <ryan@rymc.io>
Co-authored-by: Nikhil Narayana <nikhil.narayana@live.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants