Efficient streaming of Geometry data

[themikelester]

Relevant forum discussion: dynamic update of polylines with the API (not czml)
Relevant issue: #932 Dynamic Buffers

Background

Doarama uses Cesium to replay GPS tracks. We can load and play many tracks at once. Currently the tracks can be quite large (500+KB as a typed array). To avoid stalling while the tracks are downloaded, we attempt to stream them. Once we have a bit of all the tracks, we create a Primitive for each track via PolylineGeometry and start playing. As more data for each track becomes available, we create an additional Primitive containing the new points and add it to the same PrimitiveCollection. We can rebalance the number of Primitives per-track as we please. We also support live tracking, so we won't have all the points available during initialization.

The problem

When dealing with large tracks, the creation time for each Primitive exceeds 100ms (on my 2017 MacBook). Because we're streaming and don't have a dedicated loading screen, paying this cost on the main thread causes massive visual hitches. Because we're not dealing with combining GeometryInstances, Primitive.loadAsynchronous() does not have that much to do. The cost lies primarily with createVertexArray(). Using interlaced geometry is of interest; about 50% of the cost is the WebGL call to bufferData(), the rest is interleaving and construction of the typed arrays which could be done async on a web worker.

Potential solutions

• We could rebalance the number of Primitives per track, so that the vert count in each is small enough so that the initialization cost does not effect framerate. For our large scenes, this would likely exceed 1000 Primitives. This is not ideal because we're paying a needless runtime cost for all those primitives once the streaming is complete.
• Increase the amount of async work that can be done when Primitive.asynchronous is set. The typed arrays for the vertex buffers could be generated in a web worker (and efficiently transfer'd back). Only the WebGL calls to bufferData need to be on the main thread. This should cut the initialization cost for our Primitives by about half. That may be enough of a win to let us get away with option 1. Regardless, it's probably a win for Primitives in general.
• Just do it outside of Cesium. We would add a hook to submit our own draw commands at the right point in the render pipeline. Set up and started rendering vertex/index buffers containing our initial geometry. As more data is streamed in, use bufferSubData() to upload each chunk to a separate VB/IB. Once the trail is fully uploaded (or the user hits the end of the initial track), we swap the buffers.
• Add functionality to support streaming geometry into Primitives. This could be done at the VertexArray level. In which case Primitive would need to support swapping its VertexArray (Primitive._va) with another. VertexArray would need functionality to add geometry, update the number of vertices, and pass through calls to Buffer (which already supports bufferSubData()). There are many other ways to achieve this, but the core functionality would be the same as option 3.

Ideally I'd like to pursue option 4. Is this something that the broader Cesium community could use? Eventually this system could be extended to support true dynamic Geometry/Primitives (which I see a lot of requests for), but I think streaming is an easier problem to solve first.

[pjcozzi]

Thanks for the background and ideas @themikelester!

If (2) is low-hanging fruit, perhaps it is worth a try. Worse case, it ends up being broadly useful to the community, but not the end game for your use case.

For (3), we would discourage a specific and custom plugin point for rendering. Basically, the plugin point today is implement a Primitive and then return DrawCommand(s) in the update function. If you really wanted to go this route, I would encourage you to take a broader approach as discussed in #648.

For (4), there is interest in a dynamic buffer of vertices that has efficient add/remove/add-many/remove-many (more use cases: #932) that under the hood could be implemented with chunking, resizing, and swapping GL buffers as you suggest. At first thought, it is not clear to me that updating Primitive is the right approach; it might be cleaner to be a separate DynamicPrimitive, StreamPrimitive, etc. That would probably become obvious as you dive deeper.

So I would suggest perhaps a quick look at (2) and then (4) as needed.