Improved groups for Requiem

[ProbablyManuel]

There are now seven groups for Requiem and its patches

Group	Name	Purpose
&requiemGroup	Requiem	Must be loaded near the end of the load order
&requiemBugsmasherGroup	Requiem - Bugsmasher	Must be loaded directly after Requiem
&requiemDLCGroup	Requiem - DLC Patches	Must be loaded directly after the Bugsmasher
&requiemWAFRGroup	Requiem - WAFR & CCR Patches	Should be loaded before all other patches
&requiemCCORGroup	Requiem - CCOR Patches	Should be loaded before all other patches but after WAFR & CCR patches
&requiemEarlyGroup	Requiem - Early Patches	Should lose conflicts against most other patches
&requiemPatchesGroup	Requiem - Misc. Patches	The bulk of Requiem patches

At the moment &requiemEarlyGroup includes AOS, ISC, CRF and Height Adjusted Races.

Further changes:

• Removed after metadata that is superseded by the new groups.
• Removed severals comments that are essentially superseded by the new groups. E.g. there is little reason to have # Must be loaded directly after Requiem.esp before the Bugsmasher because the group &requiemBugsmasherGroup tells you exactly that.
• Moved some metadata to provide a more coherent overview of the rules for Requiem. This helped me while updating the metadata.

These changes are ready to be merged but they won't be enough to ensure that LOOT v0.13.1 creates a decent Requiem based load order. Two problems remain:

1. Any mod that must be loaded after Requiem will be loaded directly after Requiem, i.e. before the &requiemBugsmasherGroup. This isn't where such mods should be loaded. They should be loaded in or after &requiemPatchesGroup. They could be added to &requiemPatchesGroup but this has the disadvantage that they will always be loaded at the end of the load order even if Requiem isn't present.
2. Every single Requiem patch/addon/tweak must be manually added to &requiemPatchesGroup or it'll be loaded directly after Requiem, i.e. before the &requiemBugsmasherGroup. There over 100 such plugins floating around on the Nexus and I'm likely to miss some if I were to undertake this quest. And such an approach doesn't feel very LOOT-like.

I think LOOT is missing functionality here. groups are a great replacement for global priority but I don't see a replacement for priority. All that had to be done is to give &requiemBugsmasherGroup, &requiemDLCGroup, &requiemWAFRGroup, &requiemCCORGroup and &requiemEarlyGroup some kind of negative priority.
Or in different words: I want to stick &requiemGroup, &requiemBugsmasherGroup, &requiemDLCGroup, &requiemWAFRGroup, &requiemCCORGroup and &requiemEarlyGroup together so that no plugin from a different group may be loaded between them.
With such a feature &requiemPatchesGroup could be dropped entirely because every Requiem patch has Requiem.esp as master anyway.
Thoughts?

[Ortham]

Removed after metadata that is superseded by the new groups.

After metadata isn't superseded, it's a stronger requirement than group membership, which can be ignored to avoid some cycles. I'd rather it stayed unless there's some reason to avoid the strength of after metadata.

Two problems remain

1. Are you referring to plugins with load after metadata or requiem as a master that aren't in one of your new requiem groups? Why not add load after metadata for those plugins to load after whatever's in the bugsmasher group, if it's always supposed to load after requiem? Given that the bugsmasher group currently only consists of two plugins, that seems reasonable. I suppose something like conditional groups would make this easier to achieve (plugins could go in a group that loads after the bugsmasher group if Requiem is present), but that's a huge can of worms.
2. I don't know what a more LOOT-like alternative would be... If you want to define the relative load orders of groups of plugins, you've got to define what plugins are in those groups.

I think LOOT is missing functionality here. groups are a great replacement for global priority but I don't see a replacement for priority.

You're right, groups aren't really a replacement for local priorities, but they suffered from the same problems as global priorities, and I think it's easier to come up with a better design in their absence (like how it's easier to spot the missing pieces if the jigsaw is mostly assembled).

Or in different words: I want to stick &requiemGroup, &requiemBugsmasherGroup, &requiemDLCGroup, &requiemWAFRGroup, &requiemCCORGroup and &requiemEarlyGroup together so that no plugin from a different group may be loaded between them.

LOOT can't guarantee that plugins will load directly after other plugins, and I don't understand why Requiem needs that. If it's because you know there are specific plugins that will cause problems if they load in between, why not add metadata to those plugins to ensure that doesn't happen? If you don't know what plugins will cause problems, you don't even know that there's a problem. It sounds a bit like "load this plugin last so it works": well, that'll work, but you can't load every plugin last. I think it's better to understand the problem so that you can be more precise about avoiding it, and end up with a more flexible system.

[ProbablyManuel]

LOOT can't guarantee that plugins will load directly after other plugins, and I don't understand why Requiem needs that.

This is more about compatibility between compatibility patches than Requiem. Requiem is just a notorious candidate for compatibility patches. Many Requiem compatibility patches and addons have conflicts with each other. Since this creates the need for an absurd amount of compatiblity patches between compatibility patches the patch makers eventually came to a silent consensus to support their patches in a certain order. The Bugsmasher aka USLEEP patch, that is shipped with Requiem itself, is always the first plugin after Requiem and every other plugin is expected to carry forward the USLEEP fixes. The DLC patches come next and every other plugin is expected to support them. Next are the WAFR & CCF patches and the CCOR patch, then the early patches and finally everything else. Hard dependencies are usually avoided so users can skip one group with minimal consequences. What makes this order great is that it works nicely even when you don't know what's inside everything else. If a mod overwrites USLEEP or the DLCs, it should have a good reason to do so (and if it doesn't, that's a problem with the mod and not load order related). If a mod overwrites WAFR, CCF or CCOR, these edits are likely more important than a consistency fix. If a mod overwrites a sound from AOS or a height from Height Adjusted Races, you may not even notice. However loading stuff from everything else before these five groups (&requiemBugsmasherGroup up to &requiemEarly) can and will cause problems. There are still some rules that need to be enforced in the everything else group, but simple load after metadata handles this fine.

Are you referring to plugins with load after metadata or requiem as a master that aren't in one of your new requiem groups? Why not add load after metadata for those plugins to load after whatever's in the bugsmasher group, if it's always supposed to load after requiem? Given that the bugsmasher group currently only consists of two plugins, that seems reasonable.

In point 1, I'm referring to plugins with load after metadata, i.e. mods that should be loaded after Requiem but aren't patches. I agree that adding more load after metadata is reasonable, when only "real" mods are considered.

If it's because you know there are specific plugins that will cause problems if they load in between, why not add metadata to those plugins to ensure that doesn't happen? If you don't know what plugins will cause problems, you don't even know that there's a problem.

I'm not talking about hypothetical problems when loading patches between these five groups. If you use all patches from Requiem 2.0 Patch Central, a Dragonborn patch, KFR and COR, LOOT v0.13 currently places 19 (!) Requiem patches too early. Thus some of their edits are overwritten by plugins from these five groups. This includes serious problems like cancelling out entire sections of the patches or invisible heads when wearing helmets. You argue more metadata could be added to solve these problems, but I think that's merely treating symptoms.
The thing is, it isn't &requiemPatchesGroup that should be loaded after the five groups. It's the five groups that must be loaded before everything else. A negative priority in v0.10 did this.

I don't know what a more LOOT-like alternative would be... If you want to define the relative load orders of groups of plugins, you've got to define what plugins are in those groups.

I'd like to get ride of &requiemPatchesGroup anyway. This would remove the need to define all plugins in this group. Adding every Requiem patch/addon/tweak to this group is essentially the same as maintaining a curated load order like BOSS. This is what I meant with it doesn't feel very LOOT-like.
Beacuse of Cycle Avoidance it's correct to have all Requiem patches in the default group, assuming it's guaranteed that they're loaded after the five groups.

What problems/disadvantages do you see with adding a directly_after field to groups? This field would enforce that no plugin from a different group can be loaded between these two groups. This has similar functionality than (negative) priority but with the advantages of groups: There is no need to guess which arbitrary integer covers your line of priority and the system is open for future changes. Adding a sixth group is as simple as inserting a node into a linked list. Multiple directly_after entries for one group will be considered a mistake.

[ProbablyManuel]

After metadata isn't superseded, it's a stronger requirement than group membership, which can be ignored to avoid some cycles. I'd rather it stayed unless there's some reason to avoid the strength of after metadata.

I think to answer this question we first need to decide how to handle Requiem pacthes in v0.13. If there is a benefit in keeping the after metadata, I'll readd it.

[Freso]

the patch makers eventually came to a silent consensus to support their patches in a certain order.

This sounds like it could be solved in a similar fashion to how LOOT "forces" official DLCs (and creation club content) to be loaded in a certain order (regardless of which ones are available/active): by stacking afters; see e.g., https://github.com/loot/fallout4/blob/27d758d1cbf3fed94a35062b786e6d69437d8f52/masterlist.yaml#L333-L1007

[Ortham]

What problems/disadvantages do you see with adding a directly_after field to groups? This field would enforce that no plugin from a different group can be loaded between these two groups.

It's an additional level of complexity that needs thought about how it fits in with the rest of the aspects of sorting and it may not be the best solution - some variation on priorities might be an alternative, for one, groups within groups might be another that's been proposed elsewhere. I don't have any specifics because it's not something I've thought about enough, and I think at this point the problem and directly loading after are too poorly defined to properly critique.

I'm interested in improving this situation, but I think I need a better understanding of the problem before I can proceed. I think I understand Requiem's pain point now, and as Freso says the situation is similar to that of hardcoded/CC plugins, so that's another example, but more examples of trying to load plugins closely together or similar things LOOT doesn't cater for well would be good.

This sounds like it could be solved in a similar fashion to how LOOT "forces" official DLCs (and creation club content) to be loaded in a certain order (regardless of which ones are available/active): by stacking afters

Perhaps, but that's such a bad solution that I added a step to the sorting process specifically to make doing that unnecessary in 0.13.1. It's an extreme case since libloadorder will refuse to apply anything that's not in that certain order, which isn't the case Requiem plugins, but it's still hugely verbose and difficult to maintain.

[ProbablyManuel]

This sounds like it could be solved in a similar fashion to how LOOT "forces" official DLCs (and creation club content) to be loaded in a certain order (regardless of which ones are available/active): by stacking afters

That's a lot more complicated and verbose than just adding every Requiem patch to &requiemPatchesGroup, but has the same effect.

groups within groups might be another that's been proposed elsewhere.

I don't see how this helps Requiem tough. Requiem patches in the default group will ignore group membership anyway because of cycle avoidance and still be loaded too early.

[Ortham]

I don't see how this helps Requiem tough. Requiem patches in the default group will ignore group membership anyway because of cycle avoidance and still be loaded too early.

You could have a requiem group that holds all the groups you've detailed in the OP that must load directly after another, and plugins in another group wouldn't get put in between those sub-groups because they're not part of the parent group.

[Ortham]

I totally forgot about this PR... Should I merge it now?

[ProbablyManuel]

It doesn't really matter at the moment because even with this pull request Requiem-based load orders won't be sorted correctly. But there is no reason not to merge either so now is as good a time as any.

[Ortham]

Well, I'll merge it so that the v0.13 branch doesn't diverge from this PR's too much, and when something like load-directly-after is done you can make another PR with whatever changes are needed then.