Cosmic text

[TotalKrill]

Replace ab_glyph with the more capable cosmic-text

Fixes #7616.

Cosmic-text is a more mature text-rendering library that handles scripts and ligatures better than ab_glyph, it can also handle system fonts which can be implemented in bevy in the future

Rebase of #8808

Changelog

Replaces text renderer ab_glyph with cosmic-text

The definition of the font size has changed with the migration to cosmic text. The behavior is now consistent with other platforms (e.g. the web), where the font size in pixels measures the height of the font (the distance between the top of the highest ascender and the bottom of the lowest descender). Font sizes in your app need to be rescaled to approximately 1.2x smaller; for example, if you were using a font size of 60.0, you should now use a font size of 50.0.

Migration guide

• Text2dBounds has been replaced with TextBounds, and it now accepts Options to the bounds, instead of using f32::INFINITY to inidicate lack of bounds
• Textsizes should be changed, dividing the current size with 1.2 will result in the same size as before.
• TextSettings struct is removed
• Feature subpixel_alignment has been removed since cosmic-text already does this automatically
• TextBundles and things rendering texts requires the CosmicBuffer Component on them as well

Suggested followups:

• TextPipeline: reconstruct byte indices for keeping track of eventual cursors in text input
• TextPipeline: (future work) split text entities into section entities
• TextPipeline: (future work) text editing
• Support line height as an option. Unitless 1.2 is the default used in browsers (1.2x font size).
• Support System Fonts and font families
• Example showing of animated text styles. Eg. throbbing hyperlinks

[TotalKrill]

@tigregalis I must have messed up somethign in the rebasing and everything, and I absolutely could not figure out how to fix it.

So basically, made an entirely new commit, containing all the changes in this branch, mashed into one. To make it possible for me to just keep the branch rebased onto main.

If you want some credit on the commit, do you have any suggestions on how to fix this? you can find the branch here https://github.com/TotalKrill/bevy/tree/cosmic-text-fix-githistory

The only way someone, including me can continue work on this is if the mess i made of the git-history is fixed, and this is the only way I could figure out how to do it

[nicoburns]

@TotalKrill https://stackoverflow.com/questions/7442112/how-to-attribute-a-single-commit-to-multiple-developers

[TotalKrill]

Hello!

I am having less time than anticipated ( sorry ) which means that adopting this might have been premature from my side.

However, changing to the cosmic-text crate from the current one would be beneficial, since cosmic text seems to become the rust text standard.

Seeing this, I am seeing that the started port tried to implement a few new features, and thus changed quite a bit of the original code. Since I did not write the first implementation, is there a way we could reduce the scope of this pull request, to get cosmic-text into bevy, for the reason to enable further improvements that cosmic-text CAN provide.

If that is an option, could someone help me with defining a clear goal of what a minumum acceptable pull request for this would be? @alice-i-cecile @tigregalis @nicoburns ?

[alice-i-cecile]

If that is an option, could someone help me with defining a clear goal of what a minumum acceptable pull request for this would be? @alice-i-cecile @tigregalis @nicoburns ?

Personally, I think we should go with a very minimal or incremental approach. As far as I can tell, we have consensus that "we should move to cosmic-text". Rewrites shouldn't block that, and even minor regressions in functionality (like multiple fonts of different sizes in the same text section) shouldn't either. Improvements don't have to be monotonic.

There are two ways to tackle that IMO:

1. Rip out the existing backend and swap to cosmic-text in a single PR.
2. Add an alternate backend controlled by a feature flag, which we iterate on across a series of PRs.

If we can do it in less than say 1k lines of code, I feel the former is the way. Regardless, my preference is to merge the migration early in the cycle and iterate to catch bugs and rearchitect as needed.

[nicoburns]

Copying from discord (there is also some more chat about text around that point in the channel):

I believe it needs:

• Request: Allow buffer dimensions to be undefined pop-os/cosmic-text#42 (comment) / Let a Buffer figure out its height during set_size pop-os/cosmic-text#70 for correct alignment
• WIP Multiple font size / line height spans pop-os/cosmic-text#150 for mixed text sizes (which I believe existing bevy text supports).

I can see a case that regressing on multiple font sizes could be acceptable (although I'm always quite loathe to break things that users might quite reasonably be depending on), but I feel like breaking text alignment wouldn't be?

And it could also do with a new Taffy release, which will allow the PR to remove a Mutex, but that's probably not blocking.

[tigregalis]

My general view is that the rebase was the hard part so this is 90% done for the MVP.

Seeing this, I am seeing that the started port tried to implement a few new features, and thus changed quite a bit of the original code. Since I did not write the first implementation, is there a way we could reduce the scope of this pull request, to get cosmic-text into bevy, for the reason to enable further improvements that cosmic-text CAN provide.

A lot of the original code was tied to the glyph_brush implementation, so swapping that out for cosmic-text necessitated rewriting most of pipeline.rs in particular, and everything else it touched. Feature-wise I think the only changes I made were being able to load system fonts and font references (so that those systems fonts could be supported), but I think those features are basically complete.

• Request: Allow buffer dimensions to be undefined pop-os/cosmic-text#42 (comment) / Let a Buffer figure out its height during set_size pop-os/cosmic-text#70 for correct alignment

The current code in this PR fakes correct alignment by using cosmic-text's left alignment (which is not broken), then just "visually" adjusts the glyphs in the line based on the calculated width.

Code

                // TODO: use cosmic text's implementation (per-BufferLine alignment) as it will be editor aware
                // see https://github.com/pop-os/cosmic-text/issues/130 (currently bugged)
                let x = x + match text_alignment {
                    TextAlignment::Left => 0.0,
                    TextAlignment::Center => (box_size.x - line_w) / 2.0,
                    TextAlignment::Right => box_size.x - line_w,
                };

Could use some testing but I think it works. This wouldn't work if we wanted to support the Editor abstraction (which I think we eventually do, but we'd have to use cosmic-text's alignment properly) but if we're reducing the scope we can probably merge it as is.

• WIP Multiple font size / line height spans pop-os/cosmic-text#150 for mixed text sizes (which I believe existing bevy text supports).

To be honest, the current state of the mixed font sizes PR for cosmic-text would work for Bevy in its current state: it correctly sizes and positions the glyphs. Cosmic-text does a little more than that though, and primarily that's the Editor abstraction. I'm not sure what Bevy's policy is with (temporarily) vendoring dependencies? What's left in that PR is to finalise parts of it in ways that aren't relevant to this PR but are relevant to cosmic-text: the Editor abstraction is currently a little broken, and migrating all of the tests and examples; got stuck on a couple of those.

If acceptable a roadmap for this PR would be:

• Accept having a single font size, and migrate to cosmic-text 0.10.0
• Alternatively, vendor the multiple font sizes PR, which is based on latest cosmic-text, then restore multiple font size functionality in the Bevy impl
• Adapt TextPipeline::create_buffer to use Buffer::set_rich_text from 0.10.0 and throw away our implementation.
• fontdb::Database::load_font_source now returns the ID so use it
• Create some issues to circle back to the to-dos

[TotalKrill]

I agree with the incremental implementation approach, and thus preferably I would like this commit to ONLY migrate to cosmic text. Event thought the system fonts did look almost finished.

The API for using the system fonts did not really feel very "Bevy", so I put the removal of that feature in a separate commit, so that it could be easily re-added in another PR after the backend switch feels okay.

Then I used the Buffer::set_rich_text from cosmic text, and removed some warnings.

When I move over to load_font_source, I believe we can start looking at a path to ensure the code quality for merging. With as minimal a change as can be possible.

[TotalKrill]

tested the performance in text_debug, and there was a 10x slowdown when running with Shaping::Advanced, compared to Shaping::Basic. Not a big fan of the regression of performance.

	Bevy	Cosmic Text Shaping::Basic	Cosmic Text Shaping::Advanced
--release	~2400fps	~2200fps	~1000fps
--debug	~200fps	~200fps	~20fps

With this test, I would suggest using Shaping::Basic for now

[TotalKrill]

I think the last part of this, might be what to do with all the sprinkled TODOs left in the codebase. I am personally a bit ambivalent, I do like them, since it would probably guide someone looking to implement additional features.

Also, maybe a new PR for the system font feature as well, since I suspect a merge would just squash these commits.

[alice-i-cecile]

I think the last part of this, might be what to do with all the sprinkled TODOs left in the codebase. I am personally a bit ambivalent, I do like them, since it would probably guide someone looking to implement additional features.

I'm fine to leave them. If we remove them, we should make an issue or ten.

Also, maybe a new PR for the system font feature as well, since I suspect a merge would just squash these commits.

Agreed. I would like that in a different PR.

[ickshonpe]

tested the performance in text_debug, and there was a 10x slowdown when running with Shaping::Advanced, compared to Shaping::Basic. Not a big fan of the regression of performance.

Bevy Cosmic Text Shaping::Basic Cosmic Text Shaping::Advanced
--release ~2400fps ~2200fps ~1000fps
--debug ~200fps ~200fps ~20fps
With this test, I would suggest using Shaping::Basic for now

Any idea why it's so expensive, what does the advanced text shaping give us?

[nicoburns]

Any idea why it's so expensive, what does the advanced text shaping give us?

I believe it's using RustyBuzz instead of Swash for shaping + doing proper unicode line breaking. I think it gives us support for things like non-latin text and ligatures. I'm not 100% sure on the details, but it's something like that. It should only affect FPS like that for constantly changing text, as shaping only needs to happen once per string, and isn't required for relayouts (although I'm not sure if the current implementation enables this).

Iced added Shaping::Basic specifically for debug mode performance (I don't think it's intended for shipping to end users). But IMO we'd be better off just always compiling deps in release mode. Taffy is also super-slow in debug mode, and I'm sure a lot of the rendering stuff is too.

[nicoburns]

I imagine Shaping::Basic would be a good tradeoff for some games / parts of games that know that they will only be rendering really basic text though. Perhaps it could be exposed as an option on a per-text-node basis?

[TotalKrill]

Yeah I can definitely see the benefits of having advanced shaping for specific use cases. However, I would also argue that for changing backend, this would be enough, and I rather err on the side of performance :) One further PR could be to discuss adding a feature gate for the shaping methods. But again, I would rather put that in another PR, where discussions of why it is so slow, and what it gives us could be considered in further depth. Since it's quite Low code / potentially high impact

…

On Mon, Nov 13, 2023, 16:38 Nico Burns ***@***.***> wrote: I imagine Shaping::Basic would be a good tradeoff for some games / parts of games that know that they will only be rendering really basic text though. Perhaps it could be exposed as an option on a per-text-node basis? — Reply to this email directly, view it on GitHub <#10193 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAYY7BXGGRGXYE62SCBMFCDYEI5F3AVCNFSM6AAAAAA6H4OQLGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMBYGM4TSNBQGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[TotalKrill]

Seems there are now merge conflicts, I consider this ready to merge, with follow up PRs preferably done after this has been merged.

Even thought it's gonna be squashed ehne merged, I assume that my branch on my fork will not be affected, so that code I removed can be collected from there afterwards

[alice-i-cecile]

Even thought it's gonna be squashed ehne merged, I assume that my branch on my fork will not be affected, so that code I removed can be collected from there afterwards

Correct :)

[rparrett]

API for specifying text bounds (especially unbounded axes) in Text2dBundle changed as a result, but IMO much more logical.

👍 from me.

Bikeshedding requested

I guess

pub const fn new(width: f32, height: f32)
pub const fn horizontal(width: f32)
pub const fn vertical(height: f32)

would roughly match a similar pattern in UiRect.

[rparrett]

I know this comment is from the old Text2dBounds struct, but is this behavior the same in cosmic text?

[rparrett]

Non-blocking

[alice-i-cecile]

I think this is ready to merge now: I'm settled on this direction and the code quality is good.

[rparrett]

I think this is in a good place for iteration in further PRs

[alice-i-cecile]

@TotalKrill once CI is green (there's a new lint), ping me and I'll get this merged ASAP.

[alice-i-cecile]

Thank you to everyone involved with the authoring or reviewing of this PR! This work is relatively important and needs release notes! Head over to bevyengine/bevy-website#1653 if you'd like to help out.