Improve loading times for maps with CJK text

[huangyingjie]

Motivation

I need a new feature that used to reduce glyphs pbf size.
When using Chinese with mapbox-gl-js in browser, it always download too many font pbfs that are too large(each 100Kb+, nearly 8M total in Beijing). This made me waiting over 10s for map initialization.

• eg. Too many pbf requests for CJK Regular/Bold klokantech/klokantech-gl-fonts#3

When the language is English, the problem disappear, because it downloads very few pbfs.
I knew ?optimize=true, but this cannot help reducing font size.

@xrwang

[huangyingjie]

Does this resolve the same problem? #395

[mourner]

@huangyingjie yes, and it's tracked in mapbox/node-fontnik#36. We should definitely pick this up again.

[jingsam]

I think there two possible ways to ameliorate the problem with CJK glyph:

1. Reduce the range of glyph request, current is 256, testing 128 or 64 may reduce the total size of glyph of a map view.
2. Rearrange the order of CJK glyph and put the frequently used characters in a continues range. This may need a mapping between code points and glyph code in mapbox-gl-js.

[mikemorris]

Reduce the range of glyph request, current is 256, testing 128 or 64 may reduce the total size of glyph of a map view.

This may reduce the total data downloaded slightly but would likely increase HTTP request overhead and doesn't really address the underlying issue.

Rearrange the order of CJK glyph and put the frequently used characters in a continues range.

I tried doing this a while back without much success, building a set of frequently used glyphs to always be loaded initially based on names in OpenStreetMap data, then loading other ranges on demand. The problem was that there's such a long tail of uncommonly used glyphs that loading a typical map view will still grab many disparate glyph ranges even after preloading a common set of ~4096 glyphs. I even tried building a single range of all necessary glyphs based on OSM data, but this was incredibly fragile and even slight changes to the underlying data caused many additional glyph range requests.

I think the most promising approach here would be to completely scrap the concept of glyph "ranges" for ideographic scripts, as they don't provide the benefits seen in syllabic scripts (where only a few ranges are necessary to render all labels in a given script). Instead, we could dynamically request only the glyphs needed to initially render a map view, and request subsequent glyphs in batches as necessary when the view changes. We'd have to weigh the backend performance impact here of potentially losing an HTTP caching layer, unless this is somewhere we could take advantage of HTTP/2 pipelining when requesting individual glyphs instead of combining them dynamically on a server.

^ @huangyingjie @jingsam I'm interested in hearing your perspectives on whether the approach I suggested sounds promising, as I'm primarily approaching this issue through an engineering/analytical lense rather than an a linguistic understanding.

Let's track further discussion on this topic over in mapbox/node-fontnik#36.

[mourner]

Hey everyone! I built a new tiny library that can generate an SDF using system fonts and Canvas 2D on the fly: https://mapbox.github.io/tiny-sdf/

I'll be experimenting with integrating it into GL JS to be used for CJK text so that CJK glyph pbf downloads can be avoided entirely. There will be drawbacks like getting small stuttering on tile loads (because of Canvas text rendering on the main thread), and worse font rendering quality (because SDFs would be generated from a small bitmap rather than an exact letter geometry). But this is probably still better than waiting for a 10+ MB glyph download for a single map screen — especially with slow internet speeds in China, like #3748 where user waited 4 minutes for the map to load on a 50kb/s connection.

cc @jfirebaugh @yhahn @kkaefer @mikemorris @itcongaili

[1ec5]

Wow, this will be huge (err, tiny) for many CJK use cases. There are still some thorny questions to work out – what else is new in CJK? – but hopefully we can come up satisfactory answers to take advantage of the work you’ve done.

Will the style designer retain any control over the font face? Will we generate the SDF based on the default font, or will we insist on a pan-Unicode font named in the font stack, like Arial Unicode MS? (Seems like TinySDF would be compatible with either approach.) Han unification means that the same Unicode codepoint may be used for five distinctly shaped glyphs, one per language. Unicode relies on language-specific fonts to render the right shape.

There can also be significant differences among the various script styles and therefore among the various system fonts for a given language. If the designer sees a gothic (sans-serif) font in Studio, they may be inclined to set it at a size that becomes illegible on a system that defaults to a Mingti (serif) or Songti font. The designer may also want to vary typefaces the same way Western labels may be set in roman or italics.

Will there be any affordance for fonts that don’t fit a square grid perfectly? Some system CJK fonts, like SimLi, are patterned on a rectangular grid.

[bearnxx]

@huangyingjie Did you solve the problem？

[huangyingjie]

@bearnxx Yes, I build a dynamic server to produce pbfs.
Then loading all pbfs just need indeed, not loading by range. There is still some tiny problems, but the whole problem has solved.
I have a plan to send pull request, but I'm not sure it would be accepted.

[jayantchens]

@huangyingjie 可以告诉我具体怎么解决这个问题吗？

(Google Translate: "Can you tell me how to solve this problem?")

[huangyingjie]

@JunYanchan Splitting the .ttf file into .pbf by fontnik, then transform them into js object by package pbf, then save them into a sqlite. Then query data from sqlite by /search?font=198;233;5433;32;555, the result is a pbf file.
But there is still a bug exists when merge many result into just one pbf. I will read the source code of pbf.

[jayantchens]

@huangyingjie感谢你的回复，可以详细加我QQ（494584784）聊聊吗？因为这样说 好像还是有点模糊哦

(Google Translate: "Let's talk on QQ talk")

[jayantchens]

@huangyingjie Where did you download the.ttf file?

[mourner]

Hey everyone, we have some great progress on this issue in PR #4895 — please check it out! Feedback welcome.

[1ec5]

Will the style designer retain any control over the font face? Will we generate the SDF based on the default font, or will we insist on a pan-Unicode font named in the font stack, like Arial Unicode MS? (Seems like TinySDF would be compatible with either approach.)

#4895 gives the style designer no control over the font face, although apparently a developer using GL JS does retain the ability to specify fonts via the localIdeographFontFamily option on Map.

Han unification means that the same Unicode codepoint may be used for five distinctly shaped glyphs, one per language. Unicode relies on language-specific fonts to render the right shape.

@ChrisLoer and I brainstormed on some approaches to solving this issue, although it probably remains outside the scope of #4895:

• Automatically detect a “language” for a style based on its symbol layers (e.g. “most common name_{lang} tag”).
• Automatically build a font-family for TinySDF based on the font stack in the style, with rules like “if you see one of the fonts on this ‘Chinese’ list in the font stack, and you’re drawing in the Chinese glyph range, override with that font locally”.
• Hard-code font fallback lists specific to each national variant of CJK ideographs, similar to Wiktionary’s stylesheet (search for “Chinese”) and choose the most appropriate list based on the browser’s navigator.language. Chinese speakers would see Chinese fonts regardless of the style or country they’re looking at; likewise, Japanese speakers would see Japanese fonts everywhere. Perhaps this would be a fallback in the event that the developer doesn’t specify localIdeographFontFamily.

[ChrisLoer]

@JunYanchan @huangyingjie @bearnxx We've merged PR #4895 into master, so it's now possible to set up a set of local font "overrides" to use for generating CJK glyphs instead of fetching them from the server. You can see an example that shows two maps before and after the override at https://github.com/mapbox/mapbox-gl-js/blob/master/debug/tinysdf.html. For a live demo without any set up on your part, see https://chrisloer.github.io/tiny-sdf/index.html.

We plan to keep making improvements to our CJK support, but we're counting on feedback from Mapbox users like you who are actually developing Chinese maps. If you could try out the changes, it would help us a lot.

• Are the performance concerns sufficiently addressed by these changes?
• Are you able to make maps with the appearance you hope for?
• Do the changes work across the set of devices you're trying to support?
• Is the configuration method acceptable?

谢谢！

[jingsam]

Wow, this really was a big improvement which incline to the rule of css fonts. Designers have choices of the level controlling on the map visual styles. If designers want to keep identical map styles, just use online fonts; otherwise they think some fonts are not matters, use local fonts.

I'm wandering when this feature comes to mapbox-gl-native?

[jayantchens]

Thank you very much, I have tried to use this method, and really load a lot faster, and now the map load 1MB + on it, in fact, our company designers on the font is not particularly large requirements, can reference the local font is enough.

[ChrisLoer]

@jingsam and @JunYanchan Thank you!

I am currently investigating doing something similar on Android. Every platform (Web/Android/iOS/Qt/etc.) handles fonts differently, so we will have to develop solutions one at a time.

[huangyingjie]

@ChrisLoer Really awesome, unbelievable improvement.

• Absolutely Yes.
• Any fonts my computer had installed can be rendered.
• Yes, at least so far.
• Consider making this as default behavior, with font-family like: '"Songti SC", "Hiragino Sans GB",sans-serif'.

[huangyingjie]

@ChrisLoer I find that only CJK will rendered by local fonts, any other glyphs in range: 0-255、65024-65279 are still fetched from remote server.
Can we render all glyphs at local? So that I don't need server at all. Is there any problem?

[ChrisLoer]

@huangyingjie The current approach relies on all of the locally rendered glyphs having a glyph "advance" of "1 em". This assumption works for CJK ideographs, but the assumption doesn't work for glyphs in many other writing systems (including the glyphs in the 0-255 range). Because the browser doesn't tell us how wide each glyph is, and because we don't completely control which font the browser uses to render the glyph, we can't reliably guess the glyph advance. If we don't get the glyph advances right, labels will end up looking broken.

So for now, the answer is "no, we can't render all glyphs locally", although we'll keep looking at alternative solutions.

[ChrisLoer]

I'm wondering when this feature comes to mapbox-gl-native?

@jingsam We're working on a gl-native port right now, you can watch the progress at mapbox/mapbox-gl-native#10522, and if you want to try using those builds locally feel free to comment on that PR asking for guidance.

[mourner]

I guess this issue can be safely closed as done for GL JS?