interpolate expression: strange numeric issues

[derMart]

mapbox-gl-js version:
v0.48.0
browser:
Chrome 68.0.3440.106 64bit Windows 7
Firefox 61.0.2 (64-Bit) Windows 7

Steps to Trigger Behavior

I want a text field on a map to have a height given in geographical units (mercator "kilometers").
So the text field should cover the same area on the map independent of the zoom level.
This works using the interpolate expression except some very strange numeric scaling errors occuring.

Consider the following minimal example:

map.addSource('text', { type: 'geojson', data: { type:'FeatureCollection', features: [ {type: 'Feature', geometry: { type: 'Point', coordinates: [0, 0] },
        properties: {
          fS: 300 / 78.206,
          fS10: 300 * Math.pow(2, 10) / 78.206,
          fS11: 300 * Math.pow(2, 11) / 78.206,
          fS20: 300 * Math.pow(2, 20) / 78.206
        }}],}});

// non inlined versions -> should be identical but are not
map.addLayer({id: 'text10',type: 'symbol',source: 'text', layout: {"text-allow-overlap": true,"text-field": "test","text-size":
  ['interpolate', ["exponential", 2], ['zoom'],0,['get', 'fS'], 10, ['get', 'fS10'],]}
});
map.addLayer({id: 'text11',type: 'symbol',source: 'text', layout: {"text-allow-overlap": true,"text-field": "test","text-size":
  ['interpolate', ["exponential", 2], ['zoom'],0,['get', 'fS'], 11, ['get', 'fS11'],]}
});
map.addLayer({id: 'text20',type: 'symbol',source: 'text', layout: {"text-allow-overlap": true,"text-field": "test","text-size":
  ['interpolate', ["exponential", 2], ['zoom'],0,['get', 'fS'], 20, ['get', 'fS20'],]}
});

// inlined versions -> as expected they are identical
map.addLayer({id: 'texti10',type: 'symbol',source: 'text', layout: {"text-allow-overlap": true,"text-field": "test","text-size":
  ['interpolate', ["exponential", 2], ['zoom'],0,300 / 78.206, 10, 300 * Math.pow(2, 10) / 78.206,]}
});
map.addLayer({id: 'texti11',type: 'symbol',source: 'text', layout: {"text-allow-overlap": true,"text-field": "test","text-size":
  ['interpolate', ["exponential", 2], ['zoom'],0,300 / 78.206, 11, 300 * Math.pow(2, 11) / 78.206,]}
});
map.addLayer({id: 'texti20',type: 'symbol',source: 'text', layout: {"text-allow-overlap": true,"text-field": "test","text-size":
  ['interpolate', ["exponential", 2], ['zoom'],0,300 / 78.206, 20, 300 * Math.pow(2, 20) / 78.206,]}
});

So the height of the text field should be approx. 300km at the equator.
The first three layers use the get expression to retrieve the calculated values from the source.
The second three layers inline the fS value in the interpolate expression.

Link to Demonstration

(sorry, I don't know how to add mapbox-gl to jsbin without using and sharing my access key)

Expected Behavior

All six text fields should have the same size and lineup independent of the zoom level.

Actual Behavior

Only the three inlined text fields line up.
Two of the three other text fields using the get expression are scaled down drastically.

It is very interesting that the inlined version work as expected while the others are not. Maybe mapbox is evaluating the non inlined version using a shader at float precision and the inlined version using javascripts double precision? However, at zoom level 0 fS evaluates to ~4 and at zoom level 11 to ~7800. Even using float precision to calculate the interpolation, I would not expect such large errors when exponentially scaling between the two values. Maybe the calculation itself has some numeric issues?

Thank you very much in advance!

[ChrisLoer]

Hi @derMart thanks for the report, sorry it took a while for us to get back to you. This definitely sounds like a shader precision issue -- we were using 16 bit ints and dividing by 10 to get a precision of .1. @ansis's recent PR #7125 just changed that to a precision of 1/256, which limits the upper range even more.

Basically, even though it's logically sound, the way we do composite zoom expression evaluation just can't handle very large or very small values.

It would still be helpful for us to see your JSBin to get a quick feel for what you're trying to do -- you can always just post it with a <INSERT ACCESS TOKEN HERE> section and we can fill in with our own.

[derMart]

There really is not much more to add to the code, but here is the jsbin you asked for.

Thank you very much for the answer, this explains the problem very well. When you divide 65k by 10 you will clamp anything greater than 6553.6, which explains the problem occuring when using zoom level 11 or bigger. If you change that to divide by 256 (which I havent found in the pull request, is it the right one?) this will be really bad news, as I will not be able to use mapbox for my purposes and 'zoom (in)dependent' labels won't be possible in general.
This means any value smaller than 1/256 and any value larger than 256! will be clamped when used in any expression. Please document that, as this is very unexpected and quite a limitation! 256 is not a large number and I can imagine a demand for a higher domain range in a lot of use cases.
I don't quite get why you limit yourself to 16 bit precision in the first place. Scanning through some code I have seen comments about the limited number of shader attributes. According to webglstats.com it is safe to assume to have 16 vec4 attributes, which are 64 floats. Do you really need more? And if so, it might be better to store them inside a texture instead of limiting the precision / range so drastically to double this number ... just my 2 cents

best

[ChrisLoer]

Oops, too many things on my clipboard, the right PR is #7125. The PR does add a logging mechanism to at least help developers figure out what's going on when they hit this limit. The way we think about text-size "256" is already a pretty large value -- I don't think we have a good handle on the use cases that would require larger. The other time I saw this problem, someone was trying to do something similar to what you're doing here -- use an exponential scaling function in order to size in "geographical" units instead of pixel units. Even without this precision limitation, that approach will run into other problems (for instance, with text, the SDF rendering is going to start looking weirder and weirder as it gets bigger).

We should think about the use case a little more -- what are you trying to use "geographically scaled" text for? Would it work to use something like a partially transparent image source to "stamp" the giant text on top of the map?

You're right that we do have to economize on the number of attributes we upload, but in this case the choice to go with 16-bit ints is to reduce the size of the buffers we upload (and thus increase rendering speed). We would use 16-bit floats if we had them on all platforms, but IIRC we can't count on them.

[derMart]

Hey, thanks alot for the explanation. I understand. Those decisions are tough if you have a large / heterogenous user and device base. Well probably my use case is a bit exotic:
For academic research I am generating a "tag map" (see here). So I have alot of text fields on the map. Using bitmaps as text is way uglier than using sdf (and wouldn't the small range problem exist here too with mapbox?) though the silver bullet would be something like this ;-).

I render those text fields on very different scales (street to continent scales), so a solution which works flawlessly for all scales would be nice.
If I know the smallest and largest height I use in one static map, I can estimate the best min and max zoom level in the interpolate expression accordingly (though maximum of 256px height is not soo much), but this limits me and is cumbersome to implement if I change those scales dynamically. Unfortunately I discovered some other problems using mapbox for my use case which are 1) I can not update multiple layers synchronously (one gets rendered with the new content before the other) and 2) changing / updating a layer takes way to long if you have large amounts of data points / text fields. So at the moment, I am implementing a custom solution which fits those needs. With that said, if I am the only one with this problem feel free to ignore and close this issue.

However, if I get that right (remember the inlined version in the examples were calculated correctly).
Whenever you use a 'get' expression, even as an intermediate in a calculation, this will get clamped. So this does not mean that only the final result of the expression is problematic when larger than 256px, but also any intermediate step. Which further might screw the whole expression (as it does in the case above).
(EDIT: With the new divider, don't you limit all styling properties to a maximum of 256px, not just text height?)
Maybe in such cases it might be a solution to do the actual expression calculation in the CPU to avoid that problem and only upload the result? Then you also will have to upload less data probably, but I understand, this is a tough one.

[ChrisLoer]

That's really interesting, thanks! I think I am going to close this issue because we don't have a planned next step, but I will keep thinking about the "tag map" case. If you want to try experimenting with a fork that just expands the range of text-size, take a look at symbol_attributes.js. You could change the data type of a_data (a_size = a_data.zw), and work from there.

the silver bullet would be something like this ;-).

Yup, vector text rendering is definitely on our want-to-do list.

I can not update multiple layers synchronously (one gets rendered with the new content before the other)

🤔 As long as the layers are in the same source, I would expect them to get updated synchronously. But there are a lot of potential complications...

changing / updating a layer takes way to long if you have large amounts of data points / text fields

Yeah, changing text requires re-doing layout and re-generating tiles, so it's much more expensive than updating "paint" properties. If you can effectively separate data into separate sources, so that you're generating tiles independently for different parts of the data set, it might help a bit (although at the cost of introducing synchronization issues).

[mike-unearth]

Hi folks, I'm new here, but I've been evaluating Mapbox GL as a potential replacement for Leaflet in the next version of our GIS platform. Just wanted to drop a line saying that geographic sizing of text (and of all geometries on the map) is a must-have for our customers. I can't overemphasize how impressed I am with what you all have built, but I'm pretty surprised that specifying the size of anything on a map in meters would be considered an edge case.

We started out by using pixel-based sizing wherever possible in our app because it's generally easier, but feedback from customers eventually led us to implement geographic sizing for everything, all the way down to line dasharrays and text halos. So far I've had success getting mapbox to support this using variations of this meters-to-pixels expression throughout our styles:

"line-width": [
    "interpolate", ["exponential", 2], ["zoom"],
    0, 0,
    24,  ["/", ["get", "line-width"], ["abs", ["/", ["*", 40075016.686, ["cos", ["get", "latitude"]]], 4294967296]]]
]

Unfortunately this doesn't work for text due to the 256px limit in the JS SDK (although it does seem to work for our mobile apps). The only workaround I've found is to intercept all the text label features, draw them in a hidden canvas (which required reimplementing several of the text-* styles for the 2d canvas), and then rendering them as icon symbols instead of text symbols. Not sure yet what the perf implications will be; many of our customers' maps have hundreds of text labels.

I'll have to ramp up on webGL before I can attempt to make any helpful contributions, but I wanted to jump in and amplify the concerns of @derMart and others who share this use case. Happy to open a fresh issue if that's warranted.