Small Caps Not Working (in v6.12 – v7.031)

[Darkwing371]

Could it be that there's an error in the OpenType Feature for small caps?

Just downloaded the latest version (7.031) of the Libertinus pack and tested in InDesign: Serif and Sans do not produce the small caps, Display and some others do.

It only seems so work with the "i". This letter accurately is substituted with the small cap glyph.

I consider this bug critical and it should be checked and a new version rolled out asap.

Thanks and thanks for your work!

[alerque]

This would be a serious regression, but I am unable to confirm it is an issue. I just tested both Serif and Sans faces as well as bold and italic styles in English and Turkish (which has dotted and undotted i's and hence a problematic one for small caps if everything isn't working right). Everything checks out in 7.031.

SILE Code

```sile \begin{document} \neverindent \begin{raggedright} \font[size=30pt] \font[family=Libertinus Serif] Caleb \em{is} a \strong{hedgehog}. \font[language=tr]{Iı İi}\break \font[features=+smcp]{Caleb \em{is} a \strong{hedgehog}. \font[language=tr]{Iı İi}}

\font[family=Libertinus Sans]
Caleb \em{is} a \strong{hedgehog}. \font[language=tr]{Iı İi}\break
\font[features=+smcp]{Caleb \em{is} a \strong{hedgehog}. \font[language=tr]{Iı İi}}
\end{raggedright}
\end{document}

</details>

[Darkwing371]

Ok, thanks for checking! Then I'll need to investigate further why this behavior occures.

With the „old“ Linux Libertine and all other fonts I use, there was never such a problem. I'll try some older releases of Libertinus and see if that behavior starts maybe with a specific release version.

[Darkwing371]

Alright, I tested through the different versions now. Last know working version is 6.11. Since version 6.12 upwards this “error” occurs.

It is obvious that the filesize of the fonts in general nearly halfed since v6.12. So this means, there must a some kind of different “internal compression” going on, or the way the font files are build is completly different somehow. This may include the way OpenType Feature Tables are stored – and could cause some weird compatiblity problems in some softwares. I read the changelogs and they said, that since v6.12 you abandoned FontForge as a tool in favour of a different one – I suppose Fontship, that is mentioned in v7.000.

I hope I was able to convice you so far, that the new workflow you use since v6.12 causes this problem and needs further investigation and tweaking. I hope you can experiment with some exporting or compression options of Fontship, or anything else to fix this unfortunate behavior.

Should you be unable to test different font files with different settings – since you are unable to reproduce the behavior in the first place – I will help you and test the files for you. Just reply here accordingly and should it be your wish I will get in touch with you via email, so we can try to figure out what's the best solution to writing compatible files.

I fear that this weird behavior I ran into could be just a shadow hurrying ahead of a much bigger problem that could occur in the future of the font. You definately should try to figure out why this happens.

So long and thanks for your work on the font!

[alerque]

All of the 6.x series were released by Khaled and none of them used Fontship. He did phase out using the Python bindings to FontForge as a way to manipulate the font during the build process. For the 7.x series I did switch the build system to Fontship, but under the hood that is using very generic and well known tools such as fontmake to actually build the fonts, it just wraps the tooling in an easier workflow.

Since the trouble started in 6.12, Fontship isn't the issue. I am still using some of the same custom Python functions developed for 6.12 (Fontship just handles calling them), so we'll look into that. You're probably right in that something got optimized away or mistakenly not included.

However, I still can't replicate this. As I said and showed, the SmallCaps feature is working just fine in the released 7.031 font files. I suspect this might be a language-related issue. Calculating which glyphs to substitute with is dependent on the shaper knowing what language the text is in. I see you are writing "Test of Libertinus" as if your content was in English, but your UI is in another language. That makes me suspect that your content is also tagged as some other language (is that Dutch? German?). Can you do a test with the content specifically tagged as English? And also let me know what language you are having trouble with so I can check the language tagging in the font features.

[ivo-s]

I tested the newest release in LibreOffice and the features, including small caps, also work fine. How about other OTF features?

[Bolpat]

I just tested v7.31 in XeLaTex in various languages, German (@Darkwing371 has a German interface for sure, I'm German), English, Turkish and Romanian. Everything seems fine. All in all, it seems like this is a bug in InDesign and not the font.

[alerque]

Thanks for the extra testing @ivo-s and @Bolpat, that's helpful.

I suspected an InDesign bug right away (and still do). Adobe's shaping systems are notoriously bad. I'm also not particularly motivated to invest a bunch of my volunteer time trying to work around bugs in software written by a multi-hundred billion dollar company that is known for terribly buggy implementations of text shaping.

That being said I am curious to understand why v6.11 works and all later versions do not. The size decrease noted by @Darkwing371 bisects to 61f0348. The mechanics of how the subr is implemented has changed completely since then but it's still happening. I suspect Adobe's bug might be in this area, although Adobe's own type blog mentions using the exact tooling we're using for this, so it probably isn't a matter of support vs. no support but of buggy support.

Fontship does not have a configuration flag to turn off cffsubr (it's quite an industry standard at this point), but I modified the build rules to skip it. @Darkwing371 can you try this font file: no-subr.zip.

Fontship rules tweak to skip cffsubr

diff --git a/rules/rules.mk b/rules/rules.mk
index ecf2cb1..486a58e 100644
--- a/rules/rules.mk
+++ b/rules/rules.mk
@@ -313,7 +313,7 @@ $(STATICTTFS): %.ttf: $(BUILDDIR)/%-instance$(and $(HINT),-hinted).ttf $(BUILDDI
    $(normalizeVersion)
 
 $(BUILDDIR)/%-subr.otf: $(BUILDDIR)/%-instance$(and $(HINT),-hinted).otf
-   $(PYTHON) -m cffsubr -o $@ $<
+   cp $< $@
 
 $(STATICOTFS): %.otf: $(BUILDDIR)/%-subr.otf $(BUILDDIR)/last-commit
    $(GFTOOLS) $(GFTOOLSFLAGS) fix-font -o $@ $<

[Darkwing371]

Thanks @ all for taking care!

I also tested different things with the latest font release. I tried the OTF in the browser by using the small caps property in CSS and I tried the latest version of LibreOffice using the small caps: yes, I can confirm, it works as expected and real OpenType small caps are shown. Also other OpenType features seem to work well: stylistic sets, number variations, ligatures …

Nevertheless the "InDesign bug" must be triggered by something inside of the font – because it is the only font (I emphasize: only font) where this weird behavior occurs. I also tested the "no-subr font", 7.031 [3d1db57]-dev. This is the result: no difference, weird behavior still there:

Still it's just the "i" that is substituted with its small cap version. Why the "i"? I also tried marking the text as English language, but it made no difference.

I read that there seem to be two different "flavours" of builing such kind of OpenType features into a font: either via some kind of "substitution table" or being directly "encoded" into the font. I don't know if that makes any sense and could be a hint … I admit that at this level of technical depths I have no further knowledge (yet).

I just know: Libertinus worked perfeclty in version 6.11, but later became quirky.

@alerque Maybe you can elaborate on "the mechanics of how the subr is implemented has changed completely"? Can you post a link to an article of what this could mean?

[alerque]

I also tested different things with the latest font release. I tried the OTF in the browser by using the small caps property in CSS and I tried the latest version of LibreOffice using the small caps: yes, I can confirm, it works as expected and real OpenType small caps are shown.

Yes, I'm pretty confident at this point that the OpenType font itself is valid and correct and that InDesign's handling is wrong.

By the way I haven't seen from you what language your content is tagged in in InDesign. Did you try it with the content tagged as English? Can you please try the string İiIı with the content language tagged as Turkish just to prove that the language tagging is working correctly. Use another font if you need to to confirm the case changes are happening properly in Turkish and not if the language is set to English. That way we can rule out this being a content language issue.

Nevertheless the "InDesign bug" must be triggered by something inside of the font – because it is the only font (I emphasize: only font) where this wierd behavior occurs. I also tested the "no-subr font", 7.031 [3d1db57]-dev. This is the result: no difference, wierd behavior still there:

Yes, I agree there must me something specific that is tripping up InDesign or at least some way it would be worked around that we're not doing any more. We'll keep looking for what that is.

I'm actually not convinced my test font build actually didn't have cffsubr run on it, the file size should have been larger...

Still it's just the "i" that is substituted with its small cap version. Why the "i"? I also tried marking the text as English language, but it made no difference.

Why the I? Probably because InDesign is not passing the correct script/language settings to the shaper. Hence why I want you to test the string above in English and Turkish.

@alerque Can you elaborate on the "mechanics of how the subr is implemented has changed completely"? Can you post a link to an article of what this could mean?

Initially in v6.12 we were using tx and manually extracting, munging, and re-inserting data. Now we directly use Adobe's own Python based cffsubr library.

[alerque]

My last zip file had the wrong test font file in it! Try this one instead: no-subr-2.zip

[alerque]

Also color me a little bit skeptical of your tests here. What you claim in this test is Libertinus Serif Semibold 6.11 is not the semibold weight and it is the italic style.

How sure are you that you don't have multiple copies of font files installed somewhere the system is picking up on them? Is there any debug information in InDesign that shows exactly what font file and features were used to render a given text (something like the inspector available in most browsers)?

[Darkwing371]

Oops, sorry for making you confused, but it was just me not changing the written text to the actual font style I used. All the tests and font file swapping I do with the Serif Regular; of it I installed the 7.031-dev version – all other variants on my system are version 6.11 at the moment. At first I wanted to use the semibold to show version 6.11, but later I decided for italic – but overlooked to change the text accordingly. So no problem here, just a mistake.

But since I found it easily I can even provide proof of the used files:

I installed now version 7.031;[3d1db57]-dev from the second archive you provided (the bigger file) and it is also really installed:

But for more clarity maybe you should give future dev releases a different build number or dev number.

So far about the used font files.

[Darkwing371]

Now about the test string you suggested – here it is:

I made sure to tag the text frames with the specific content language. You see, that Turkish does something different. You also see, that even in ligatures there's just the "i" that becomes replaced by the corresponding small cap version.

Sidenote:
And you can also see that almost no font has a small cap version of the Capital Sharp S. Challenge: Be the first one on earth who has built it in his font. =D I mean ... it's logical, isn't it? ;)

So far I hope my test results can give you a hint where there lies the hare in the pepper.

I also read your linked article about the subroutinization: maybe you can give a printout of the tool, that shows the size of the subroutines implemented in v6.11 and 7.031? Maybe we hit a limit of 32k-3?

[alerque]

Now about the test string you suggested – here it is:

Ore those strings all marked as smallcaps? Did you try them in other fonts too?

I made sure to tag the text frames with the specific content language. You see, that Turkish does something different. But you also see, that the types of ligatures don't become replaced by the corresponding small cap versions. Bummer!

The small cap feature does work with ligatures:

But that brings up the interesting point that you have other open type features turned on such as stylistic alternates. The Th and st ligatures are not on by default but are part of the discretionary ligature and historical ligature sets . Try turning those off and see if your small caps start working. These can normally be combined (and Libertinus works with them) but maybe that is your InDesign bug source.

Sidenote:
And you can also see that almost no font has a small cap version of the Capital Sharp S. Challange: Be the first one on earth who has built it in his font. =D I mean ... it's logical, isn't it? ;)

No it isn't logical, your expectations here are off. You can't have "small caps" versions of capital letters because by definition the OpenType spec for the +smcp feature is to convert lower case letters to small versions of upper case forms:

This feature turns glyphs for lowercase characters into small capitals.

Hence why no other fonts do this. If you want that you need to convert to lower case first. You can do with with OpenType using the +c2sc feature.

And yes combining +smcp,+c2sc works in Libertinus Serif:

[Darkwing371]

Ok, during my tests I deliberately turned on some other OpenType features just to show that they are working correctly. This can not be the source of the quirk, because I had them disabled at the begining, turned them on in later tests and now have them turned off again completely – except for the small caps. And yes, all the strings are marked to become small capsy.

For the sake of clear view I'll soon alter my test document I post screenshots here of. I think it's better to use an alphabete.

(And about the Capital Sharp S I'll convice you in a different thread later. ;) Let's not mix the topics here.)

[khaledhosny]

I’m almost certain the fix for #303 is what trips InDesign. Before this fix smalllcaps lookups were single substitution lookups like most fonts, after that they became multiple substitution lookups. Adobe’s OpenType implementations are known for all sorts of misguided assumptions, and that snap feature always uses single substitution lookups might be one of them.

You might get better luck using World Ready Composer, or HarfBuzz (both should be supported by latest InDesign and you can look up how to use them).

[khaledhosny]

(It would have been hilarious if using Adobe’s subroutininzer, cffsubr, would cause such an issue).

[ivo-s]

@Darkwing371 Just a quick note on the sharp s, as there seems to be some confusion. The font already has everything. As Caleb explained, there are two separate features introducing small caps, and both seem to work properly:

In your example, you only have the first feature turned on, which does not convert the capital sharp s into a small cap.

[Darkwing371]

Well, at first I thought dealing with the Capital Sharp S would be off topic, but now that I researched this more: maybe this gives us a better understanding of what is going on.

I tested this in InDesign and the result is:

Not only the small caps aren't working correctly, but also the substitution of the all caps feature isn't working correctly. It does substitue ß → SS instead of ß → ẞ. What a bummer!

This whole behavior seems to be consistent througout Adobe Creative Suite 5 – I tested in Photoshop CS5 and InDesign CS5. Thou professional, since it is a fairly old software (2010) I'm at the verge of starting to believe that it is the inner workings of the typesetting software that produces this behavior, because "it doesn't know better". (But maybe it's still two separate problems, who knows.)

I'm beginning to believe, that the font format standards changed over time so dramatically, that older software, that's nevertheless fully capable of OpenType features, don't understand the new formats and "spots for saving things" the fonts have inside nowadays.

I tested it again in the browser with CSS properties, et voilá: it works and is all there.

Oh boy ... do I really need an Adobe Creative Cloud Abo because fonts grow more and more into "incompatiblity"? =o Nasty move, big tech ...

Philosophical question: Should fonts be as compatible as possible, or should they always embrace the latest technology and don't be backwards compatible, so that only the spearhead elitists can fully use them?

[Darkwing371]

Back to the small caps an the "i": I made a more clear test document now. Here you can have a better overview of what is substituted and what not.

If we leave out the Sharp S for now, can we solve the puzzle with the "i"? It seams to be substituted partially correct with the Turkish letters, right? So does it have to do something with language zones or the like?

Also could you please check whether the size of the subroutines that are encoded, exceed a certain limit as described in your linked article? This could be interesting!

[ivo-s]

I am getting a bit lost. The puzzle with the "i" has been answered by Khaled. There are 3 smcp lookup tables. Two contain just the "i" for purposes of including the dot in certain languages. One contains all the other letters. In terms of sequence, as displayed by FontForge, the table with the dotless "i" comes first, which strongly suggests that the software stops at the first lookup table, where there is only one entry, and therefore substitutes only "i". As to why, we do not know, but since you are using the discontinued Adobe CS5, the fonts work all right in other software, there are no objective bugs in the fonts that can be identified, I think there is nothing to look for. If the fonts compiled without subroutines didn't help, then the thing that triggers the CS5 bug can be anything, and quite likely cannot be turned off or fixed.

I also noticed that you are confusing small-cap substitution of the sharp s with an alternate stylistic set 3. The ss03 feature replaces the sharp s symbol with "ss" in all variants - lowercase, capital, and small caps. You obviously have that turned on. Without it, both capital and lowercase eszett work correctly, and properly convert into small-caps, as I showed in the image above. And there is no "all-caps substitution feature" in the OTF format that I know of. If you have something like that available, it is purely InDesign functionality.

It is bad that CS5 cannot handle some new fonts, but you generally cannot expect any software to be compatible into the future. I know from experience that the stuff I created in Adobe Flash CS5 in 2010 did not function properly in Adobe's own Flash Player some years later.

[Darkwing371]

... oh right, I forgot to mention in my lengthy post: I do not have Stylisitc Set 3 turned on. The SS-Substitution happens on its own.

And yes, maybe it's an InDesign option, but it uses the OpenType features or a combination thereof, like demonstrated in our thread above:

Anyway, I also believe this doesn't matter and leads the thread astray.

What I will do in the following days, is to download the test version of the latest InDesign CC and see what the behavior is there.

[alerque]

I'm beginning to believe, that the font format standards changed over time so dramatically, that older software, that's nevertheless fully capable of OpenType features, don't understand the new formats and "spots for saving things" the fonts have inside nowadays.

No, the OpenType spec has not changed in this regard.

I tested it again in the browser with CSS properties, et voilá: it works and is all there.

Yes, because the font is fine.

Oh boy ... do I really need an Adobe Creative Cloud Abo because fonts grow more and more into "incompatiblity"? =o Nasty move, big tech ...

The version you are using was broken when you bought it and Adobe wants you to fork over more money for a version that is slightly less broken (but still known to be broken in many ways). Not my problem. The font (provided free of charge) correctly adheres to the decades old font format standard and many free software packages exists that can correctly shape it. Browsers, word processors, specialty print layout engines etc.

It's really not my problem Adobe is ripping you off.

Philosophical question: Should fonts be as compatible as possible, or should they always embrace the latest technology and don't be backwards compatible, so that only the spearhead elitists can fully use them?

Fonts should be compatible with the spec. The spec already dictates many steps just for backwards compatibility, and spec compliant fonts work with spec compliant software from decades ago. Fonts should not be expected to make up for buggy shapers that don't actually follow specs. And the more so when the font project developers are donating their time and the buggy software shaper is from a $240 billion dollar company.

[alerque]

By the way you are welcome to try building yourself a version without the macron fix.

Start with a clone of this repository, then revert the related commit, then rebuild.

$ git revert f915575
$ fontship make -- otf

The Fontship project page has a one liner way to run fontship without installing anything if you have Docker, if not you can have Github build you one by forking the repository and pushing the revert commit to your fork. It will automatically build a full set of files that you can download and use.

The result may work for your purposes, but since it would be a regression on other issues I won't be considering that for mainline inclusion. If there is anything than needs to be done to have a valid working OpenType font that conforms to the spec then of course I would consider that, but that does not seem to be the case here. Literally all of the issues you've brought up work fine in the font using normal OpenType mechanisms, the issue is your not using an open-type compatible piece of software (however much their marketing might have led you to believe the contrary).