Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the patcher copy folders that have a certain named pattern #926

Open
snouz opened this issue Oct 18, 2024 · 24 comments
Open

Make the patcher copy folders that have a certain named pattern #926

snouz opened this issue Oct 18, 2024 · 24 comments
Labels

Comments

@snouz
Copy link
Collaborator

snouz commented Oct 18, 2024

My idea is that if you add a specific name in an integrated folder, for example MoguriMain_integrated, the patcher would automatically copy it to FFIX, and remove the "_integrated" in the name

image

This would allow projects that want to bundle Memoria to just have the patcher in their zip, and clicking it would copy their own projects there, with minimal human intervention (no need to find the right folder, extract...)

@barkermn01
Copy link
Contributor

it would also need to update the memoria.ini file, but should be possible, I would look at possibly using an archive type setup with multiple mod folders inside it so people can create mod pack installers?

Would you like me to look into it?

@faospark
Copy link
Contributor

faospark commented Oct 19, 2024

This would allow projects that want to bundle Memoria to just have the patcher in their zip, and clicking it would copy their own projects there, with minimal human intervention (no need to find the right folder, extract...)

why bundle patcher though ? can it not be a bat file ?

@barkermn01
Copy link
Contributor

why bundle patcher though ? can it not be a bat file ?

a .bat file is not trusted by anyone but people who know how to read them, and even less so by Anti-Viruses when they are downloading &/ extracting files. it does not look very complete does opening up command prompts we have a loverly UI when using it why not make a nice UI for setup, and if people want to build mod packs give them the ability to within having janky hacked together things.

So to support a .bat we would still have to code into the patcher for the bat file to invoke it because there is no way the .bat file is writing a datastore file with options set inside it. then since we would have to be coding a proper solution anyway might as well get it in line with the new styles and make it feal uniform.

@snouz
Copy link
Collaborator Author

snouz commented Oct 20, 2024

This would allow projects that want to bundle Memoria to just have the patcher in their zip, and clicking it would copy their own projects there, with minimal human intervention (no need to find the right folder, extract...)

why bundle patcher though ? can it not be a bat file ?

I don't see the added value of a bat file, when memoria patcher is already moving and copying files around by itself?

@faospark
Copy link
Contributor

faospark commented Oct 21, 2024

why bundle patcher though ? can it not be a bat file ?

a .bat file is not trusted by anyone but people who know how to read them, and even less so by Anti-Viruses when they are downloading &/ extracting files. it does not look very complete does opening up command prompts we have a loverly UI when using it why not make a nice UI for setup, and if people want to build mod packs give them the ability to within having janky hacked together things.

So to support a .bat we would still have to code into the patcher for the bat file to invoke it because there is no way the .bat file is writing a datastore file with options set inside it. then since we would have to be coding a proper solution anyway might as well get it in line with the new styles and make it feal uniform.

perhaps i did not phrase my response that well. i understand you point. i thought the idea was much more like case use for small projects. :)

@faospark
Copy link
Contributor

This would allow projects that want to bundle Memoria to just have the patcher in their zip, and clicking it would copy their own projects there, with minimal human intervention (no need to find the right folder, extract...)

why bundle patcher though ? can it not be a bat file ?

I don't see the added value of a bat file, when memoria patcher is already moving and copying files around by itself?

Noted. I didn't mean to suggest that a batch file is an alternative to your idea. Rather, my thought process was this: if a modder has a project or mod and simply wants to inject or transfer its contents directly into the game root, it's not that difficult to automate with a batch file. This is part of my own workflow for my mods, but it seems that's not the case here, so let me elaborate.

A few thoughts come to mind:

First, is there a real use case for bundling Memoria's patcher (the main installer) with someone’s project? Not long ago, Moguri had an installer with a version of Memoria, TheMighty bundled Memoria with his project, and Echo-S had its own installer linking to the patcher. However, having multiple patchers with the same name across different projects might confuse users. This could partly explain why Memoria, despite being both the engine and mod manager, is still relatively obscure. Over-all its weird that The Patcher is included in mods.. Say for example, can you imagine something like the 7th Heaven installer being bundled with another project....

Another point is that it might undermine the mod catalog unless the catalog itself is on the way out. Bundling patchers with individual mods could lead to modders and users neglecting the catalog altogether. A modder could easily package their mod with a patcher, and users might prefer these bundled mods out of convenience, bypassing the catalog entirely.

Lastly, there’s the potential for bad actors. Someone could repackage existing mods, make slight changes, and release it under a different name—perhaps even call it something like “FF9: Remake” in another language. This ecosystem, including the mod catalog and Nexus, is vulnerable to this. A person could bundle several mods into one folder, set the memoria.ini to read-only, and configure the game to skip the launcher altogether. They wouldn't even need the official installer since it’s already included. Furthermore, say for example, another voice mod that is not echo-s and shall remain nameless... does this.

Don’t get me wrong—I’m just thinking about the possibilities this opens up. I could be wrong, but we should consider whether this would truly be beneficial or if it risks opening Pandora’s box. While it may work well for a few projects, it also provides an easy path for those with bad intentions.

@barkermn01
Copy link
Contributor

barkermn01 commented Oct 22, 2024

@faospark you raise some very good points,

is there a real use case for bundling Memoria's patcher (the main installer) with someone’s project,

My aim is to ensure that our installer is used exclusively, regardless of the project. Although this is a long-term goal, the first version will simply be a setup

Not long ago, Moguri had an installer with a version of Memoria, TheMighty bundled Memoria with his project, and Echo-S had its own installer linking to the patcher

This is another reason to standardize our installer to avoid confusion. Regardless of the mod being shipped, the Memoria installer should always be our official installer signed the same way we sign the patcher now.

Another point is that it might undermine the mod catalog unless the catalog itself is on the way out.

Hopefully not because the mod manager should still attempt to update mods even if installed via the installer, helping to reduce this risk,

There are some aspects that are beyond our control within the Modding community. Whether or not a mod manager is used, people can still download manually and add malicious code to mods a trick users into installing the modified version, including utilizing batch files. If someone distributes a compromised version of Memoria, we can’t completely prevent it. However, my proposal aims to mitigate this risk. By having a uniform installer that attempts to download a signed version when online, similar to how the patcher is currently signed, we can help prevent malicious modifications. Our installer will pull from our GitHub Releases.

My master plan

Please note these are just my initial idea's on what i would like I'm working with @SamsamTS and @snouz to build this the way we all think is the correct way we might stop and never hit stage 3 i don't know yet.

Stage 1)
Build a setup tool that checks if it's online if so download and use the latest patcher from this GitHub Repo,

Stage 2)
Add support for mods to be shipped with the setup file (Memoria_Setup_Mods folder) with a instructions file, (JSON example for ease) i may turn it into a binary file format with some verification to stop people just tampering with the files.

{
"mods":[
   {
       "name":"Alternate Fantasy 9",
       "folder": "af9"
   }
]
}

The installer will check for Memoria_Setup_Mods/af9 folder will copy it to the Memoria install location named Alternate Fantasy 9, add the folder to the memoria.ini file that will also be shipped in the installer for if offline or downloaded from GitHub if it's online.

Stage 3)
Check the catalogue for the mod name, and the ID, verify them maybe even have a hash tree of the files inside the mod folder.

  • if it's not there warn people your installing a mod that is not part of the catalogue only do this if you trust the creator,
  • if not online warn them that mods can't be verified make sure they trust the creator.
  • if they are online and the mod is found in the catalogue then download it from the catalogue. This also means people creating installers like Moguri, Echo-S ect don't have to build a new installer Everytime they update there mod or Memoria update the installer will do it. Tsunamods Installer actually downloaded the Memoria.Patcher from GitHub it was @SamsamTS who built it but it broken when we moved to using a Signed Installer
  • Create a tool within Memoria that generates the necessary files for a repack. This tool would prompt users to select from their list of installed mods, then then build the setup package accordingly.

I hope this clears up any confusion and makes it clear I'm trying my best to make this as secure as possible.

@faospark
Copy link
Contributor

Thank you for responding @barkermn01, much appreciated. Don’t get me wrong, I see the intent is good, but I also think it's necessary to ask the hard questions. So here we go.

At this point, the installer is pretty much standardized. It has been for a while—it’s just that past devs tended to treat it as a dependency rather than what it actually is: The Engine, Launcher, and Mod Manager.

The confusion didn’t arise because there were multiple installers; it occurred because projects treated Memoria as a side dependency and prioritized their own project names. We are now working with a much more capable tool compared to before. To give an example, look at the 7th Heaven mod manager (which also installs FFNx). Out of the 124 mods published on Nexus for Final Fantasy VII, not one contains an installer for the 7th Heaven mod manager. Instead, they ask you to download the manager and install it separately.

Which leads to my first rhetorical question: Why are we even allowing the patcher to look like a minor dependency and not the main requirement for the mods? I think you’ve provided an answer, but it complicates things just to get one result.

The modding paradigm is straightforward: if a mod manager exists, you download and install it first. Then, you download mods from any source and install them according to the mod manager’s rules. For all intents and purposes, the proposed idea is exactly the same as what some mods have done with their installers—just possibly without the .exe. It’s essentially an installer for a project, regardless of how anyone wants to complicate it. It feels like a step back, not forward, in my humble opinion.

Upon your further elaboration, @barkermn01, I see this is for a very specific use case, which I appreciate you revealing.

... This also means people creating installers like Moguri, Echo-S ect don't have to build a new installer Everytime they update there mod or Memoria update the installer will do it...

First, I personally think @snouz would not want to revert to a separate installer, given the amount of time and effort they had to invest in packaging Moguri along with its other two components, plus setting up download services for it to be compliant and downloadable via the mod manager. Unless I’m wrong, I don’t see that happening.

Now, returning to the matter at hand: Since no other mods in the catalog, Nexus, or general Final Fantasy IX modding ecosystem come with an installer, we’re left with just one—Echo-S. So, here’s s question: Is it impossible for this project to be bundled as a Memoria Mod Package? It’s just a zip file with the required files. If it has other dependencies, you can still use your installer and just instruct people to ensure they have the latest Memoria installed—like the case of any other mod that uses a manager. Isn’t that the simpler approach?

I hope I don’t come across as antagonistic. If we are going to objectively assess the situation, we are trying to create a solution to a predicament that isn’t necessarily a Memoria problem.

Are we willing to add layers of complexity now and in the future, rather than simply ask people to install the latest version of Memoria and the voice mod on top of it?

This is also adding another layer where the entire Memoria project will adjust specifically for one project. It will create complications, as elaborated, with featured mods being assigned IDs for integrity checks, requiring an almost always-online system just to verify if a mod is featured or not.

If you guys aren’t busy, I’d also like to hear the thoughts of @Tirlititi and @Albeoris on this.

@SamsamTS
Copy link
Contributor

SamsamTS commented Oct 23, 2024

At this point, the installer is pretty much standardized. It has been for a while—it’s just that past devs tended to treat it as a dependency rather than what it actually is: The Engine, Launcher, and Mod Manager.

Amen to that. Let Memoria be the mod manager it's meant to be. I said it before but I'll say it again here, I'm against letting people distribute Memoria with their own projects. I'm in favor of having one and unique official source for Memoria.

Now something I've been thinking that could be good to make manual installation easier (in case downloading from the manager is not a possibility, which is the case for some people) is to have the launcher accept drag and drops. People would just have to drop the archive of the mod directly onto the launcher window and the mod would be installed and activated automatically.

@Tirlititi
Copy link
Collaborator

Agreed with faospark, and Samsam's idea of drag-and-drop, which can also be useful in some situations.

@firehawkx
Copy link

firehawkx commented Oct 23, 2024

Having co-created AtmaWeapon (mod installer for FF6), I agree with the previous few messages that it is usually way easier for casual users to just have a single mod manager/installer... Of course advanced user that really do not want to use it can always do everything manually - but changing the patcher to allow for distribution of a single mod with just a tiny part while excluding the manager itself seems to be opening the door to a lot more abuse and confusion -- and I think it will ultimately confuse the more casual players that would have otherwise enjoyed having the mod installer many optional choices.

@barkermn01
Copy link
Contributor

barkermn01 commented Oct 23, 2024

Well to start with here is the basic installer, it required the new Patcher, so currently i have the patcher and the installer on my Repo for people to test with. i have a draft PR, but i had already done most of the work for this hence it's here. feedback would be nice.

https://github.com/barkermn01/Memoria/releases/tag/test

If we're all happy with it i will update change stuff to use this repo ready for release.

@barkermn01 barkermn01 mentioned this issue Oct 23, 2024
3 tasks
@Albeoris
Copy link
Owner

To make this discussion less holy war-like, I’d suggest starting with a very brief description of usage scenarios, commonly known as User Stories.

As I understand it, there are currently two:

  1. The user installs Memoria using its installer, then enables the desired mods and downloads them from Internet.
  2. The user downloads a specific mod installer, installs it and all "dependencies" like Memoria.

Then, I’d consider the issues we face:

  1. When using the Memoria installer, mod installation is spread across two steps. This requires an Internet connection, several clicks in different places, and knowledge of where to click. This increases the entry barrier, making it difficult for some players to use. Without Internet access, the user has to deal with multiple files—installer, mods—and manually install them. But this approach motivates contributing to Memoria and create mods integrated into it.

  2. When using a mod installer, we get diverse and often incompatible installation processes, outdated components, and potential security risks. However, the entire installation is streamlined with a straightforward Next -> Next -> Next approach and is completed in a single run, without needing to enable anything manually. This approach motivates developers to create their mods, popularize them, and collect donations.

Memoria.Patcher works well as an installer for Memoria's core, but not for additional modifications for the reasons outlined above.

I fully agree that new mods should promote Memoria as the engine and help expand the player and mod maker community using it. However, I also believe that installing large-scale mods should not be more complicated than Next -> Next -> Next.

The plan described by @barkermn01 seems to address both problems if we integrate mod selection and the installation process.

The user downloads an "installer" or "launcher," runs it, selects the installation directory, and, on the next page, checks the desired mods.
We could provide both an online version, which displays an online catalog, and an offline version, where mods are loaded from a local directory, with the installer showing a warning if the directory is unavailable (as barkermn01 suggested). Configuration files in this directory could specify which mods are enabled by default. This would allow authors of large mods to preselect their preferred mods for users and create their own bundles, but all would still use the same Memoria installer.
And of course, it should support reusability - I could install a clean Memoria, then add a few mods and install them, later download an Echo-S package, and ultimately achieve the same state as if I had selected everything in the launcher after installing clean Memoria.

Local catalog can be like:

\Memoria\
  \ Memoria.exe
  \ Content
    \ Mods
      \ Alternate Fantasy
        \ enabled
      \ Echo-S
        \ default.cfg
        \ enabled

I believe this approach would address all the issues.
If desired, we can also make this directory a storage for all modified files, and instead of creating directories in the game folder, create hard-links to folders with mods inside it when launching the game via Memoria.exe

What do you think?

@SamsamTS
Copy link
Contributor

SamsamTS commented Nov 15, 2024

With the new drag and drop feature I recently added, manual installation is a lot easier now. The two situations I can see are

  1. The user can install mods directly from the catalog, all is well.
  2. The user cannot use the catalog for some reason or another. He can just download the mod(s) separately, drag n' drop on it/them on the launcher window and it/they'll be installed and enabled. Really easy.

@faospark
Copy link
Contributor

faospark commented Nov 15, 2024

@Albeoris i appreciate the response .
Ill clarify this one

The user installs Memoria using its installer, then enables the desired mods and downloads them from Internet.
The user downloads a specific mod installer, installs it and all "dependencies" like Memoria.

currently Memoria is now a mod manager so the true scenarios are

  • install Memoria Patcher , installs featured mods from the mod manager and activate on the mod manager
  • install Memoria Patcher , downloads mods from Nexusmod or other locations, extract on game root and activate via mod manager.

Addendum :
SamsamTS's further makes it easy for the second item mentioned above .

Now what is proposed on this issue raised it to make another pathway but more like and exemption if would be more direct.
in my opinion We should let Memoria Mod manager do it its thing... install and manage mods.

@Albeoris
Copy link
Owner

I mean something like:

image

  1. You always run Memoria.exe
  2. You can make a mod pack and distribute it, but users will still run Memoria.exe
  3. You can install it locally without internet access if you downloaded all the necessary mods in advance.
  4. If the online mod catalog is available, you can get a new version, even if the build contains an old version.
  5. If the mod author made his own build, but did not register his mod in the catalog, users will receive a warning that this is a third-party mod, and this will motivate mod authors to register their mods in the catalog.
  6. If the modifications selected by the author of the package are not installed, the installer window will appear. If they are installed, the Launcher will open immediately.

@SamsamTS
Copy link
Contributor

For security reasons having mods distributing an exe is a very bad idea. I'd rather they don't.

@Albeoris
Copy link
Owner

Albeoris commented Nov 16, 2024

For security reasons having mods distributing an exe is a very bad idea. I'd rather they don't.

@SamsamTS , why, if it will be our signed Memoria.exe installer?
The only thing they can do is add their mod archives to the Content folder.

@SamsamTS
Copy link
Contributor

SamsamTS commented Nov 16, 2024

There is no guaranty it's our signed installer they are distributing. The code source is available, a malicious person can compile their own version and distribute it. Having the installer distributed from one unique official source is better for safety or at the very least for peace of mind. I don't want to encourage modders to distribute our installer.

Now if all they are distributing is the Content folder, I'm fine with it. They most likely will zip it in some manner for distribution anyway and I find the drag and drop on the launcher more convenient than unzipping next to the installer and running it.

Also if people start getting a virus from some malicious installer we don't want people accusing us for putting it in.

@Albeoris
Copy link
Owner

There is no guaranty it's our signed installer they are distributing.

The same thing that is happening now with Memoria.Patcher.exe, you can also make a malicious self-extracting archive, or your own installer, or simply distribute a virus named Memoria.exe

The code source is available, a malicious person can compile their own version and distribute it.

No, he won't be able to, since we will have a signed installer.
And the fact that users ignore warnings that they have an unsigned executable file, you can't fix it.

That is, I understand your concerns, but I don't understand how they relate to the problems described here.
If the author wants to distribute his mod through his installer, you can't limit him in this.
The only thing you can do is give them a Memoria.exe build that doesn't require recompilation and solves his problems.

Now if all they are distributing is the Content folder, I'm fine with it.

Yes, that's the idea.

I find the drag and drop on the launcher more convenient than unzipping next to the installer and running it.

To do this you need to do 2 actions - install Mod Manager, install the mod.
Why, if you can select the mods to install together with the launch of Mod Manager?

Also if people start getting a virus from some malicious installer we don't want people accusing us for putting it in.

This is solved by downloading the mod from our catalog and checking the checksums if it is delivered locally. If the checksum does not match or the mod is not in the catalog, we will display a large warning about this, warning the user that this is done at their own risk.

Like:

  • A new version of the mod is available, are you sure you want to install the old one?
  • The content of the local version of the mod differs from the content registered in the mod catalog. This may indicate that the archive is damaged or infected. Are you sure you want to install it?
  • The local version of the mod is not registered in the mod catalog. We cannot validate it, the archive may be damaged, outdated or infected. Are you sure you want to install this mod?

@SamsamTS
Copy link
Contributor

SamsamTS commented Nov 16, 2024

No, he won't be able to, since we will have a signed installer.

I do it all the time with the patcher. Nothing has ever stopped me from compiling it. Memoria as a precedent of not being signed, people will run the unsigned patcher.

To do this you need to do 2 actions - install Mod Manager, install the mod.

That's the modding procedure for many other games. I don't really see much utility to be able to select the mods before rather than after installing the mod manager. It's either select the mods to install, then install the manager, or install the manager then select the mods.

For quite a while people were using an old version of memoria because a popular mod was distributing it. You have to realize that once people start distributing our exe, we lose control of it, we can't ever force them (the people distributing) to update it.

Now if you guys really want to go that direction I won't try to stop you but I still think it's a bad idea, no matter how many precautions you take.

@Albeoris
Copy link
Owner

Albeoris commented Nov 16, 2024

I do it all the time with the patcher. Nothing has ever stopped me from compiling it. Memoria as a precedent of not being signed, people will run the unsigned patcher.

And we can't do anything about it. :)

That's the modding procedure for many other games.

And this case is already covered by the built-in Mod Manager.

The problem is when you install the "product" and not the "mod". For example, the authors made a Polish localization with voice acting for characters, and distribute their package under the name "Polish voice acting for Final Fantasy IX".

Now it will be a custom PolishVoices.exe, but we can make it with the same Memoria.exe, in the Content folder of which \Mods\PolishVoices.zip is located. At the same time, this will not interfere with the security and compatibility of mods.

TL;DR;
I totally agree that Memoria.exe should be the only installer and Mod Manager, its existence in any modpack should ensure the PR of Memoria Engine.

But I don't see any reason why we can't give the opportunity to those who want to make such modpacks, if it doesn't create any downsides.

@SamsamTS
Copy link
Contributor

Something I might do, regardless if we do an installer or not, is extend the drag and drop functionality to support mod packs. Zip all desired mods and a preset (optional) together and there, your mod pack is done.

In fact I was thinking, if a preset includes mods, the launcher should check if they are installed and ask to install those that are not if they exist in the catalog. That way people can just share their preset, acting like a mod pack. (Drag and drop for presets is already implemented).

image

@faospark
Copy link
Contributor

I totally agree that Memoria.exe should be the only installer and Mod Manager, its existence in any modpack should ensure the PR of Memoria Engine.

Everyone here agrees this was the goal. As mod contributors, we're promoting Memoria to gain more recognition as the engine/launcher/mod manager. Unfortunately, 60-70% of people who play FFIX with mods refer to the launcher and manager as Moguri, not Memoria. Kudos to Snouz for bundling Moguri as a mod, but we still have to inform users and content creators that it's Memoria making this possible, not Moguri. It has not even been a year since the move was made But here we are already like allowing for another Mod to have pathway where in principle the mod is just treating Memoria like an inconvenient thing you need to install simply because the mod creators dont want to tell people users the need to install Memoria first...so they disguised it as a mod pack ?

But I don't see any reason why we can't give the opportunity to those who want to make such modpacks, if it doesn't create any downsides.

Well there is plenty of things to consider. In line with my previous points, Instead of requiring the mod in question to follow the same standards as other mods, we are effectively giving it special treatment by creating a unique pathway for it. This opens the door for mods that directly compete with or replace those in the mod manager to gain an unfair advantage—they can easily outperform those in the catalog because they are simplified for the user, who only needs to download the mod without even considering Memoria. While it’s not certain this will happen, the possibility is there.

1, This approach could have significant downsides. For example, a mod developed by an independent creator in the catalog could be easily overshadowed by one supported by a large modding group. If that mod is bundled in a mod pack with direct installation, users are unlikely to explore other alternatives in the catalog, even if they exist.
2, In that sense this could render the catalog obsolete, as any modding group could simply create a comprehensive replacement set of mods and or packs, sidestepping the catalog entirely. I mean what is the point of being in the catalog if i can just bundle my mod to have a patcher in it ? or have it be part of a modpack of a larger group ?

Furthermore, if this change moves forward, it shifts the perspective on what’s permissible. If Echo-S can leverage this flexibility, then all mods should be able to as well. As mod creators, we also need to compete for visibility and downloads. Currently, there’s already a layer of complexity in adding mods to the catalog; in fact, many of us create two versions of our mods—one with a direct link and another for alternative sites like NexusMods. Speaking with other contributors, I find that the general sentiment is that they would prefer a direct link to NexusMods, as downloads there offer some return. While being featured in the mod manager is valuable and serves as a form of endorsement, many contributors would appreciate more options that simplify this process.
if we are making it easy for one then it should be applied to everyone else... However, the thing that will suffer is Memoria. its recognition and function as a mod manager cause it will be down the drain.

Instead of simplifying how we look on this we need to look deeper implications of this down the line.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

7 participants