Fallback fonts with atlas clearing.

[saurtron]

Work done

• Introduce lua AddFallbackFont(vfspath) and ClearFallbackFonts().
  • Backend at CFontTexture::AddFallbackFont and CFontTexture::ClearFallbackFonts.
• Add the mechanism so atlases are properly cleared
• Adds the FontsChanged callin to signal applications text dlists or other text caches need to be refreshed.

Remarks

• To review, compare directly, or to "fallback fonts" branch: direct link to comparison to better see the clearing code
• Added it as separate PR to facilitate review.
  • Added it as DRAFT to avoid accidental merging before #1777. It can be merged at the same time by just merging this PR, or can be merged right after that one.
  • This is now the "fallback-fonts" branch I propose for inclusion, we can still merge only #1777 but otherwise that one will serve just as reference from now on.
  • • Compared to Mechanism to define priority fallback fonts #1777, this adds atlas clearing, so adding and clearing fallbacks will be properly reflected by the game at any time.
• Clearing atlases might not be critical, since most applications are expected to define glyphs at the beginning and not change too much, but it's needed for robustness.

New api

gl.AddFallbackFont(vfspath)

Add a fallback font. First added will have higher priority.

Link to code and inline documentation.

gl.ClearFallbackFonts()

Clear all fallback fonts.

Link to code and inline documentation.

unsynced callin 'FontsChanged'.

This is needed so the game/application can properly react to texture atlases having been changed. Modules usually cook font drawing into display lists, and those need to be cleared when receiving this callin, otherwise previously cooked text fallbacks won't show properly (base glyphs will, since we restore them into the same atlas coordinates).

Testing

• Get this bar branch test-fallback-fonts into your BAR.sdd
  • This branch has functionality for enabling/disabling one fallback through mouse left click, and also runs an automatic procedure to write into chat and enable/disable fallbacks.
  • Also listens for FontsChanged callin in order to clear dlists at gui_chat.
• Open skirmish
• Once it starts, the test widget will automatically run the following procedure:
  • Write something into chat, like: "blah🔥" (at frame 3)
    • it will use your os fallback
  • Enable fallbacks and write into chat (at frame 150)
    • should say "Fallback fonts, true"
    • The fire glyph should change to the provided font
  • Disable fallbacks and write into chat (at frame 450)
    • should say "Fallback fonts, false"
    • The fire glyph should change to the os font again

You can further play with it by using left mouse button to enable/disable fallback, and writing your own characters into chat.

After

writing with fallback for Noto enabled, then disabling it (note how the fire symbol reverts to the system default), then adding the fallback again. (this uses the new callin to refresh dlists at gui_chat, otherwise it wouldn't refresh properly).

[saurtron]

@lhog, @sprunk: Made this the default fallback-fonts PR, and converted the base fallback-fonts to DRAFT to make it more evident.

I think the feature is completed only in this branch, so not worth it all the hassle of going through two separate branches. Still, I will be maintaining that one in case you prefer to merge only that for starters.

[sprunk]

Is there a way to tell whether any given string has missing glyphs programmaticaly (after OS and backups are taken into account)?

-- OS should have some ascii font even if we clear them all
Spring.ClearFallbackFonts()
assert(false, Spring.AreAnyGlyphsMissing("ASCII"))

-- but other non-latin might be an issue
assert(true, Spring.AreAnyGlyphsMissing("προσταγμα"))
Spring.AddFallbackFont("greek.fnt")
assert(false, Spring.AreAnyGlyphsMissing("προσταγμα"))
assert(true, Spring.AreAnyGlyphsMissing("김정은"))

The use case is that I can't really distribute fonts for every random shithole country language in the world, but I can usually assume the people who set that language will have the font:

local originalEnglishText = "nuclear launch detected"
local translatedIntoKlingon = Spring.Utilities.I18N(originalEnglishText, "klingon")
local stringToDisplay
if not Spring.AreAnyGlyphsMissing(translatedIntoKlingon) then
	-- ok, we didn't distribute the font but presumably klingons have their font installed on their OS
	stringToDisplay = translatedIntoKlingon
else
	-- otherwise, display something sane (= english) instead of ending up with "���" or "□ □ □"
	stringToDisplay = originalEnglishText
end

[saurtron]

Is there a way to tell whether any given string has missing glyphs programmaticaly (after OS and backups are taken into account)?

Well, yes, sure we could have some code to check that.

The use case is that I can't really distribute fonts for every random shithole country language in the world, but I can usually assume the people who set that language will have the font:

The problem with this is then different users will get different fonts selected for that language, and in many cases it won't work good like happening here. Otherwise I believe we can be safe to assume every user will have fonts for their own language.

In any case, I'll try to make something, but it likely needs some refactoring as the whole glyph search thing really needs it to be more reusable for cases like that.

I'd rather do that reorganization after getting this in, so as to not complicate the PR too much. Also, even if not perfect it would already allow to solve most of the currently pending issues and see what turns out later.

If I do see it's feasible without much reorg then I can try to include that asap.

[sprunk]

Nah don't worry about it, there's no hurry. As long as it's feasible and doesn't conflict architecturally with what already exists it's fine.

[sprunk]

Does this mean font files not in the VFS (i.e. loose files somewhere on user disk aka VFS.RAW) can't be used? If they can be used, how do I specify which one to use if two are under the same path (one raw and one in the vfs)?

[saurtron]

Well, tbh I don't know XD. It goes through the same path other fonts use to load as well, basically doing: CFileHandler f(fontPath). I'm just reusing that part of the code through CFontTexture::LoadFontFace.

[sprunk]

If we're not (yet) letting people specify their own VFS mode then add a @remark that it uses the RAW_FIRST vfs mode.

[saurtron]

done.

[saurtron]

well, didn't use a specific @ tag since I don't see any that can be used here, just added it to the comment for the @param.

[Prehistoricman]

Tested and working on AMD GPU (RX 580)

[saurtron]

Ok managed to get this tested with windows amd, thx to Prehistoricman at discord.

[saurtron]

Tested and working on AMD GPU (RX 580)

thank you :D