Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor the core's handling of firmware and BIOS images to rely less on the file system #1826

Merged
merged 69 commits into from
Sep 18, 2023

Conversation

JesseTG
Copy link
Contributor

@JesseTG JesseTG commented Aug 31, 2023

This pull request refactors how firmware images and BIOS images are loaded by melonDS. It makes the following changes:

  • The core no longer loads BIOS or firmware images files directly. The logic for that is moved into the frontend.
  • The core no longer writes firmware images directly to disk. Instead it uses a new Platform::WriteFirmware function, which in this case delegates to a new SaveManager.
  • The logic for separately saving Wi-fi settings is moved out of the core and into Platform::WriteFirmware.
  • The embedded FreeBIOS binary is moved into its own file, rather than a header.
  • Firmware blobs are vastly easier to use:
    • Firmware blobs can now be interacted with as objects rather than as raw byte arrays (although that's still an option). All properties described in the GBATek docs are accessible by name.
    • Generated firmware is given an identifier of MELN (no \0), to simplify special treatment.
    • enums are defined for all firmware fields with a restricted set of values.
    • structs are defined for the firmware header (including Wi-fi calibration), user settings, and Wi-fi configuration (normal and extended). Firmware objects don't store them separately; instead, these structures are exposed through (well-defined uses of) reinterpret_cast.
    • Firmware can be created with a constructor that takes a console type, an existing buffer, or a FileHandle.
    • Methods are added for common operations or queries, such as updating checksums or getting the current user data section (as defined by the GBATek docs).
  • Several ConfigEntry values were removed, since the core no longer needs to ask for them. The corresponding config values in Config.cpp are not removed; firmware and BIOS files can still be selected in the frontend as usual.

My use cases (all of which relate to melonDS DS) are as follows:

  • I wanted to allow the DNS settings to be overriden, and to set the default DNS settings to point to Kaeru WFC.
  • I wanted to allow individual firmware settings (name, birthday, etc.) to be overridden, instead of the all-or-nothing approach that the upstream frontend currently offers.
  • I wanted finer control over when changes to firmware are saved back to disk.
  • Since generated firmware is treated differently (i.e. FileRead isn't enough), I wanted to mimic that treatment for compatibility.

This PR does not affect how SD cards or the DSi NAND are saved or loaded; I intend to clean that up in a later PR.

- Remove some functions that were reimplemented in the Firmware constructors
- Initialize both user data objects to default
- Set both user data objects to the same touch screen calibration
@JesseTG
Copy link
Contributor Author

JesseTG commented Sep 15, 2023

Ah, I found a bug in this PR. As written, the frontend may overwrite a firmware file with the contents of a wfcsettings.bin file. Let me fix that before this is merged.

- Now they return a pair instead of two by-ref values
- Manage its buffers as unique_ptrs to mitigate leaks
- Reallocate the internal buffer if SetPath is asked to reload the file (and the new length is different)
- Firmware settings are now properly saved to disk (beforehand I misunderstood when the firmware blob was written)
- Firmware is no longer overwritten by contents of wfcsettings.bin
@JesseTG
Copy link
Contributor Author

JesseTG commented Sep 17, 2023

Okay, good to go. Bug's fixed.

src/NDS.cpp Show resolved Hide resolved
@RSDuck RSDuck merged commit 5bfe51e into melonDS-emu:master Sep 18, 2023
8 checks passed
kvnp pushed a commit to kvnp/melonDS that referenced this pull request Dec 25, 2023
… on the file system (melonDS-emu#1826)

* Introduce firmware-related structs

* Fix some indents

* Move the generated firmware identifier to a constant

* Document the WifiAccessPoint constructors

* Add some constants

* Remove a stray comment

* Implement Firmware::UserData

* Add Firmware::Mask

* Document Firmware::Buffer

* Add a Firmware constructor that uses a FileHandle

* Set the default username in UserData

* Update the UserData checksum

* Forgot to include Platform.h

* Remove some redundant assignments in the default Firmware constructor

* const-ify CRC16

* Replace the plain Firmware buffer with a Firmware object

- Remove some functions that were reimplemented in the Firmware constructors

* Fix some crashes due to undefined behavior

* Fix the user data initialization

- Initialize both user data objects to default
- Set both user data objects to the same touch screen calibration

* Follow the DS logic in determining which user data section is current

* Remove an unneeded include

* Remove another unneeded include

* Initialize FirmwareMask in Firmware::Firmware

* Use the DEFAULT_SSID constant

* Add SPI_Firmware::InstallFirmware and SPI_Firmware::RemoveFirmware

* Move a logging call to after the file is written

* Add a SaveManager for the firmware

* Touch up the SPI_Firmware::Firmware declaration

* Move all firmware loading and customization to the frontend

* Call Platform::WriteFirmware when it's time to write the firmware back to disk

* Fix some missing stuff

* Remove the Get* functions from SPI_Firmware in favor of GetFirmware()

* Implement SPI_Firmware::DeInit in terms of RemoveFirmware

* Add Firmware::UpdateChecksums

* Fix an incorrect length

* Update all checksums in the firmware after setting touch screen calibration data

* Use the Firmware object's Position methods

* Remove register fields from the Firmware object

* Install the firmware before seeing if direct boot is necessary

* Install the firmware before calling NDS::Reset in LoadROM

* Slight cleanup in ROMManager

* Fix the default access point name

* Shorten the various getters in Firmware

* Add qualifiers for certain uses of firmware types

- GCC can get picky if -fpermissive isn't defined

* Add an InstallFirmware overload that takes a unique_ptr

* Log when firmware is added or removed

* Don't clear the firmware in SPI_Firmware::Init

- The original code didn't, it just set the pointer to nullptr

* Fix a typo

* Write back the full firmware if it's not generated

* Move the FreeBIOS to an external file

* Load wfcsettings.bin into the correct part of the generated firmware blob

* Load BIOS files in the frontend, not in the core

* Fix logging the firmware ID

* Add some utility functions

* Mark Firmware's constructors as explicit

* Remove obsolete ConfigEntry values

* Include <locale> explicitly in ROMManager

* Fix up some includes

* Add Firmware::IsBootable()

* Add a newline to a log entry

- Whoops

* Log the number of bytes written out in SaveManager

* Mark FirmwareHeader's constructor as explicit

* Clean up GenerateDefaultFirmware and LoadFirmwareFromFile

- Now they return a pair instead of two by-ref values

* Refactor SaveManager a little bit

- Manage its buffers as unique_ptrs to mitigate leaks
- Reallocate the internal buffer if SetPath is asked to reload the file (and the new length is different)

* Remove some stray parens

* Fix some firmware-related bugs I introduced

- Firmware settings are now properly saved to disk (beforehand I misunderstood when the firmware blob was written)
- Firmware is no longer overwritten by contents of wfcsettings.bin

* Slight cleanup
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants