Previous migrations might be reverted after setting a fallback version

[jonasraoni]

Describe the bug

• After setting the fallback version at a given point, the downgrade for previous and successfully migrations are being executed, but they shouldn't, see: https://github.com/pkp/ojs/pull/3895/files#r1207504069.

• As an improvement to the upgrade process, the "save point" should probably be set automatically after every successfully migration (e.g. Status: pre-3.4.0 failed at migration XYZ), it should make the life of technically skilled users easier.

• Re-think our downgrade strategy: Downgrades are mostly useful to allow users to revert to a previous version without losing newer data and to speed up things (e.g. run a couple of updates instead of restoring a huge dataset from a backup).
  Given that we have a win/lose condition (e.g. reverting a single database migration is pointless/wrong, unless we can do it on the code as well), we've got to either implement the down() method properly (store deleted data + ensure newer data will be handled properly) in all migrations or ignore it altogether (for now I personally support the latter).

Perhaps this should be a discussion 🤔

[asmecher]

I think this is resolved for 3.4.0 by having the preflight check migration be the first to run, and making it the only migration that sets a fallback (as it currently is). It's not a perfect model but for 3.4.0 I'm satisfied. If you'd like to open a discussion for some broader thinking about upgrades, it's probably a broader subject for consideration -- for example, there are a few exchanges about the usefulness/uselessness of down() migrations that we could include over there.

[jonasraoni]

About the first point of my original message. Perhaps it's a bug, if we consider that any migration can make use of the "fallback" version, but I know it should be the first, so not a big deal.

And here's a short thought proposal.

Upgrade:
The fallback version is set only if the upgrade fails at the preflight migration. If it passes through the preflight, then fails on the next migration (let's say it's not worth/possible to cover all possible failures), all the progress is lost.

Besides bringing the "fallback version" stuff to the core upgrade code (which I've mentioned on the second point of my original message), I think the process can also be highly improved by allowing the administrator to upgrade interactively (--upto MigrationX or --interactive). It will allow the administrator to do a pitstop, issue a backup, then debug/consult the forum/resume from where the process stopped.

The preflight migration is good to prevent the administrator from resuming a failed upgrade too many times. Perhaps we can extend the upgrade with options. For example, instead of interrupting the upgrade with a message like "Remove or process the log files before proceeding", we could offer an upgrade option to automate the decision.

Downgrade:
I think a well written down() (able to recover data + accommodate new data into older structures) is most suitable for big organizations (e.g. banks), that cannot afford to stop their systems and must be able to recover fast from small weekly/monthly upgrades. On our case once the upgrade works, there's no turning back.

With the exception of plugins, we don't provide ways to downgrade from specific features/versions, those would require a code downgrade as well. Therefore our down() is useful only for failed upgrades, but it should be properly implemented.

With all that being said, I think the suggestion to deal with upgrades interactively is enough to get rid of the down, which is probably "too much work with few gains".