Support custom layer types powered by WebGL

[edenh]

@ansis and I have discussed the possibility of allowing people to write custom glsl shaders. While the learning curve is fairly steep, the possibilities are pretty mindblowing when considering integrated data vis and animated, interactive basemaps.

Checkout these shaders for inspiration:
https://www.shadertoy.com/view/MsXGRf
https://www.shadertoy.com/view/Msl3Rr
https://www.shadertoy.com/view/XdsGDB
http://glsl.heroku.com/e#13858.13
http://glsl.heroku.com/e#13880.2

I'm not really sure exactly how we would accomplish this, but hoping for discussion here.

cc @mourner @kkaefer @yhahn

[jfirebaugh]

No current plans to do this.

[enjalot]

I know this is closed but I'd love to see it for things like controlling saturation or other levels on a raster layer.
custom shaders on a layer could potentially enable arbitrary clipping via mask images which would be fun too.

[lucaswoj]

This is worth leaving open, now that we "support" this in gl-native

[mourner]

things like controlling saturation or other levels on a raster layer

@enjalot can't you control saturation and other stuff with raster paint properties?

[lucaswoj]

As I said in #1742, this would be less scary if we had a GL state tracking abstraction #145

[enjalot]

@mourner oops I didn't see the other levels besides opacity before. I still believe that this could lead to very interesting customizations! depth of field, blurs, animations and a lot of other shiny stuff 💯

[lucaswoj]

EDIT: A newer specification is discussed in #281 (comment) below 👇

---

Here's our plan for implementing this feature:

• There will be a new style layer type called "custom"
• Developers can register a rendering callback for each custom layer
• When a custom layer is drawn, the callback is called and passed the WebGL context within in a declarative state-tracking wrapper a la OpenGL state management system #145
• The callback may draw directly to the WebGL framebuffer (higher performance, must use declarative state-tracking wrapper) or copy content from another framebuffer (lower performance, no restrictions on rendering)
• Custom layers must use the GL state tracker correctly or else the map may not render correctly or the application may stop working

[lucaswoj]

Based on the experience we've had with custom layer types on GL Native and custom sources in GL JS, I'm proposing a new specification:

• We will implement GL state tracking per OpenGL state management system #145 (this explains RendererAtoms)
• We will implement a general-purpose backgrand processing utility per Refactor "Worker", "Dispatcher", "Actor" into a general purpose "PooledWorker" utility #3034
• We will add a Map#addCustomLayer(id: string, layer: StyleLayer) method
• We will define a new StyleLayer interface which is easy-to-understand and powerful

interface StyleLayer {
    getRendererAtoms(
        camera: Camera,
        callback: (error: Error, atoms: Array<RendererAtom>) => void
    );
}

• We will create a reference implementation of the StyleLayer interface that copies content from another GL context (allowing the use of 3rd party canvas wrappers)
• We will refactor existing layer types to use the same interface internally, allowing composability
• We will refactor the custom source interface to use a similar mechanism (Map#addCustomSource(source: StyleSource))

[lucaswoj]

I'm on board with that plan if the overhead isn't too big.

[mourner]

Given that we do 500-1000 draw calls per frame with vast state changes between them, 2 more shouldn't make much difference. We could also introduce a "red button" option to forego state resets completely (with "use at your own risk" label).

For resetting state, there's an existing module we might find useful: https://github.com/stackgl/gl-reset

[kronick]

For reference, Mapzen supports something similar to what @mourner is suggesting ("augment existing shaders" that draw each layer) allows creation of maps like this: https://mapzen.com/blog/behind-the-scenes-tron2/

Here's how they're integrated: https://mapzen.com/documentation/tangram/Shaders-Overview/ and some of their modular shader "blocks": http://tangrams.github.io/blocks/

This could be a very performant way of natively supporting some cool animated visualizations that right now are being built external to the core (such as with Uber's Deck.gl).

[lbud]

This was inadvertently closed due to some 🌳 history magic (the commit message that closed this intended to close style-spec's issue of the same number) — reopening.

[gerbsen]

Hey there, are there any new plans/timelines for this feature?

[terion-name]

@mourner applying shaders to map layers (as in your demo) — is very, very powerful and awesome thing and very expected from a webgl-map. Did something move in this direction?

[kaligrafy]

The hack proposed by @mourner no longer work since 0.39

I used that a lot. If someone can fix the hack, it would be very appreciated, since it worked very well and was easy to implement! Until we get an official way to do this.

[mourner]

Hey, I'll try to update the hack to work with recent releases. However should really look into exposing a proper API for this.

[kaligrafy]

Thanks a lot! I hope there will be an API for that too!

[mourner]

Opened a new issue with a summary of our current thinking about this API: #6456 — closing this issue in favor of the new one. Let's discuss there.