SDF Text Labels!

[jasonbeverage]

Hi everyone!

This PR adds Signed Distance Field (SDF) text rendering to Cesium. Signed Distance Fields are a rendering technique made popular by Valve for rendering crisp, scalable text (and other images) without using a ton of texture memory. Regular bitmap fonts tend to start to look bad when they are rotated or when they are scaled up or down, but Signed Distance Fields solve this problem by storing the "distance" from the edge of a glyph in the texture instead and using a special fragment shader that can render crisp antialiased edges.

This isn't totally done but I wanted to get it out there to get some feedback.

Here is an example of what the new fonts look like.

Here is a summary of major changes to Cesium.

• Glyphs are now being generated by the TinySDF library https://github.com/mapbox/tiny-sdf. TinySDF is a library from MapBox that generates character glyphs using a canvas and then computes the SDF texture directly from the pixels in the canvas. There are other libraries that generate SDF textures but they generally work offline are use the TTF font files directly and can't run right in the browser.
• The Billboard and BillboardCollection classes have been updated to do optional SDF rendering. This is not part of the public API and is only turned on by the LabelCollection.
• Previously every small change to a font or a character color would result in a new, colored, glyph being generated in the texture atlas. So generating a 24px green B and a 24px yellow B would result in 2 characters in the atlas. Or requesting a 48px green Q and a 49px green Q would also result in 2 glyphs. Now, all fonts, regardless of requested size, are generated once at a fixed size (48px currently) and then scaled up or down under the hood using the billboard scale. This works with the Label.scale parameter as it did before, but it also works by setting a CSS font. So setting a label to use "48px Calibri" and another to use "24px Calibri" will use just a single glyph set between them for the font, but the character billboards will be scaled.
• Because of the way that outlining works when using SDF it is no longer necessary to generate glyphs with outlines included, freeing up more space in the texture atlas as well. You can change the outline at runtime and it just sends a parameter to the shader and does all the outlining there, so you can do things like animate the outline size of a label and it'll be fast.
• There haven't been any changes to the Cesium public API, the label API should work as it did before.
• There is a new Labels SDF example that lets you scale labels up and down and play with the outline width with a slider as well as shows an animated label.
• I added a new BillboardCollection.debugTextureAtlas property that will display a full screen debug quad that shows the contents of the texture atlas. The Labels SDF example shows that off with a checkbox.

What is left to do?

• The glyphs are generated at size 48 currently. This seemed to give a good balance of texture size and quality for everything I've tried. We might be able to get away with a smaller font size, but I might need some help with coming up a good test case for this.
• I still need to write some new unit tests.
• The glyphs are currently grayscale RGBA textures. We really don't need RGBA and can use single band luminance textures instead. I've got code that does this and it works fine, but there are issues with the TextureAtlas class using a luminance texture when it does it's resizing b/c it tries to attach it as a color buffer to the frame buffer object, which is illegal. If we want to use luminance textures in the future we'll need to change the way the texture atlas resizing works.
• There is a little jitter when labels are scaled down, it might be something with the texture filtering, where is that set?
• I need to do a little work to get the measureText based kerning to work again.
• I need to tweak the outline width setting a little so that it doesn't let you push the outlining higher than it can actually go.

Looking forward to your feedback!

[cesium-concierge]

Thanks for the pull request @jasonbeverage!

• ✔️ Signed CLA found.
• ❔ Changes to third party files were made.
  • Looks like a file in one of our ThirdParty folders (ThirdParty/, Source/ThirdParty/) has been added or modified. Please verify that it has a section in LICENSE.md and that its license information is up to date with this new version.
• ❔ Unit tests were not updated.
  • Make sure you've updated tests to reflect your changes, added tests for any new code, and ran the code coverage tool.

Reviewers, don't forget to make sure that:

• Cesium Viewer works.
• Works in 2D/CV.
• Works (or fails gracefully) in IE11.

[lilleyse]

@jasonbeverage I haven't looked at the code yet but these were some of the things I noticed while going through all the Sandcastle examples:

• Text looks sharper but also a bit thinner than master which can lead to some aliasing. Is this what you mean by the jitter / texture filtering?

Some of the ones I noticed it on was Callback Property, CZML Reference Properties (used in the gifs), and KML.

master

sdf

• The text box fill color expands too far. See 3D Tiles Interactivity and CZML Billboard and Label (screenshot).

• There are some crashes which look like the same error:
  • Labels - Offset label by distance
  • Labels - Scale label by distance
  • Picking - hover the mouse off the globe (gif)

[lilleyse]

Would we want to use SDF in PinBuilder/writeTextToCanvas? I think this would probably be a lot of work and not worth doing, but I'm curious if you had thought about it.

[lilleyse]

• Use a new image for the Label SDF sandcastle icon

[jasonbeverage]

Hi @lilleyse, I pushed some changes that should fix the crash and the odd label positioning you were seeing. When I added the CSS font based scaling I ended up breaking the positioning.

I also pushed a dynamic SDF scale factor that seems to look pretty good as well. Give it a look and see what you think.

[lilleyse]

The outline color alpha doesn't seem to be blending properly: working sandcastle on cesiumjs.org

[jasonbeverage]

Good catch. I'm encoding the width in the alpha component of that attribute. I'll pack the color so its compressed and add the alpha back in.

…

On Tue, Apr 30, 2019, 2:29 PM Sean Lilley ***@***.***> wrote: The outline color alpha doesn't seem to be blending properly: working sandcastle on cesiumjs.org <https://cesiumjs.org/Cesium/Build/Apps/Sandcastle/#c=bVRrb9owFP0rFl8IEjNlfa2UViuhndBoO7Vl01SqykkuYOHYke1A2bT/vmub0NKSD3nc63Pu69xMSplariSZgo2VUDpK3f2G5dAkTBQz1iB/x5LgtWCaeCc5IzEYXubUIx43CGrVqChAx8xA1Hg6DTgNttRyC0InWuUXjj2Ea5KCaQNXQjEbhagNRP8by7F0YRccluDiSlhWRD+9Laqn/jNW0jIuQdcdcCxbLfIwAw+8VhkIYjVL54ZYNBrLLBA1IarUJOeSE1YUgqfMNYK+RgzAs6p+VVqBEX7xzM5Ih+zR48Nm8JiUCUBTm+41t876YtFRv4OsvsNl0PcYnKT+TQNI99ITJbjnbxBCLYOLrYIHi6g/YWeqIrHuBWjr63rNOYc8AWTn0ioylyqdY1SiEgN6wRIBBqtct7HyUt+gaMPh2oj8PS6zd+RI6Qz922sCAnKQ1rhmOttogA9mCfZDkAmWzu26n1YpkTA3wkylpQNRVNxlwPdWgyyqr4/4Ab5Pzg1o5XLhcmpek2xWvJuhxxrccBkRLMFcUV0CgsCXHMdmlyp4TEgrvGNWQWHUpCCBFprn3PIF9ollWfRGdEN3Pt6QRl6mGyLkCYQetpZNoQz3CXQ2K8C0xTcm9/0e9GGKkzfRp+ND2j46aB8dHTfJ/gk9OfzsrsZaNxZerJPSjxkXDGsvZpxVmppwIbygOttrdnfZp67usGp7tF2RJTjrqValzHbBesOL+PsOuX44NrrcTf92VTqkWpMJ7qir4OBL8UJiJniieVWBsSsBna023zsTvRoMh88XN/3n29HDcHBzidpvnNaata5HnAf0V54XCreg1CKitGUhLwTKwLSSMp2DpakxblDdVgXqZnxBeHY2rr37fYxrJBXMGPRMSiHu+R8Y1867LTy/BcM/ldPiLS6fYCt3ZNY+HwYjpbTbws+PqLVa3zD+Bw> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#7730 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACPXYSIKIRJGHXVYVNRYJDPTCFW5ANCNFSM4HETUX5A> .

[lilleyse]

@jasonbeverage I went through the code and thought it looked pretty good. I pushed a commit with style tweaks 56ca95a and left a few comments.

[lilleyse]

@jasonbeverage the updates look good, thanks!

[pjcozzi]

Thanks again, @jasonbeverage. I only briefly reviewed and left hopefully a few useful comments and perhaps a few truly minor ones.

Some thoughts:

1. Would you like to write a short guest blog post about this? Maybe a tad about the implementation, but more about why this feature matters, the visual quality improvement, etc.
2. Did you do any performance tests compared to the traditional raster approach? Like add 10K static labels so that the fillrate is ultra high and see the difference. It's OK for it to be slower under load, but it would be good if we had a handle on it.
3. Also for performance, do you have any concerns on dynamic use cases, e.g., constantly updating a label like a number would be the most important one, but I imagine there are some slow paths such as changing the font, maybe size, etc. Totally OK, would just like to know what they are.

Thanks! 🚀

[jasonbeverage]

@pjcozzi sorry I missed these reviews coming through for some reason, I'll try to address them as soon as I can.

To respond to your questions:

1. Sure I'd be happy to write a guest blog post. I'll do something similar to what I did for the particle system blog post and send it along.
2. I didn't do any specific performance tests vs the previous approach. The fragment shader is more complicated b/c of the multi-sampling to remove the aliasing, so I imagine it will be slower than the original approach but hopefully not drastically. I can can do a little FPS test and let you know what I find.
3. As for the most common use cases I'm going to say that SDF should actually be FASTER than the original approach. Changing font size, color, and outline width all resulted in the generation of new glyphs previously. For example in the new Labels SDF example it scales the Morgantown label up and down each frame. Cesium master would chug along generating different sized glyphs for each of the potential scales until the scaling had oscillated up and down one cycle and all the glyphs for the potential sizes had been generated. With SDF the glyph is generated once and that's it. Changing fonts of a label requires new glyphs to be generated as it does in the master, but there is the slight overhead of computing the SDF field from the glyphs so it will be just a little slower. I haven't seen any case where it's a new bottleneck though.

[pjcozzi]

1. Cool, I will connect you with @OmarShehata via email now.
2. Great, I look forward to the stats
3. Fantastic news!

[jasonbeverage]

Hey @pjcozzi I addressed most of your reviews with either a code change or a comment. I'm going to take a look at a FPS test next.

[jasonbeverage]

I did a little sandcastle test to display a big number of labels

var viewer = new Cesium.Viewer('cesiumContainer');
var scene = viewer.scene;
scene.debugShowFramesPerSecond = true;

Cesium.Math.setRandomNumberSeed(3);
 
var labels = viewer.scene.primitives.add(new Cesium.LabelCollection());
var i = 0;
for (i = 0; i < 10000; i++) {
    labels.add({        
        position: Cesium.Cartesian3.fromDegrees(Cesium.Math.randomBetween(-180.0, 180.0), Cesium.Math.randomBetween(-90.0, 90.0)),
        text: 'Label ' + i,
        fillColor: Cesium.Color.WHITE,
        outlineColor: Cesium.Color.BLACK,
        outlineWidth: 1.0,
        font: '24px Calibri',
        style: Cesium.LabelStyle.FILL_AND_OUTLINE
    });
}

At 10K labels there is no difference between the master and the SDF branch on my machine, they both run at 60fps with the globe totally covered in labels.

At 100K labels the master runs at 10fps and my SDF branch runs at 7fps, so there is a difference in performance at some level but 100K labels is such an extreme case I don't think its going to be noticeable to anyone doing a sensible use case in Cesium.

[pjcozzi]

@jasonbeverage great news on performance; does not sound like a concern at all for the cases you tested.

For test coverage, the best thing to do would be to just run coverage and double check that the new code paths are covered well. Have you done this? See https://github.com/AnalyticalGraphicsInc/cesium/tree/master/Documentation/Contributors/TestingGuide#run-all-tests-against-combined-file-run-all-tests-against-combined-file-with-debug-code-removed

Otherwise, I'll leave it to @lilleyse for the final merge and release as time allows for him.

[lilleyse]

I did one last smokescreen through the Sandcastle examples and they look fine. Thanks @jasonbeverage!

I'll try to get to #7875 this month so SDF can look even better,

[jasonbeverage]

Awesome, thanks @lilleyse!