performance degrades with multiple sources vs a single source

[vicapow]

Using multiple sources instead of just one, produces a notice impact on performance. The use case is for creating a choropleth so might not be relevant once data drive styles lands.

Single Source

map.once('load', function loaded() {
    map.batch(function batch() {
        map.addSource('features', {type: 'geojson', data: featureCollection});
        map.addLayer({
            id: 'features-fill',
            type: 'fill',
            source: 'features',
            paint: {'fill-color': 'red', 'fill-opacity': 0.6}
        });
    });
});

### Multiple Sources

map.once('load', function loaded() {
    map.batch(function batch() {
        features.forEach(function eachFilter(feature, index) {
            var id = 'feature-' + index;
            map.addSource(id, {type: 'geojson', data: feature});
            map.addLayer({
                id: id + '-fill',
                type: 'fill',
                source: id,
                paint: {'fill-color': 'red', 'fill-opacity': 0.6}
            });
        });
    });
});

[kristfal]

Did some experiments with this a while back. The performance issues we saw were primarily related to number of unique layers, not number of sources. Could be this in your case as well.

Lots of geoJSON sources mainly affected tile parsing speed iirc, but we didn't see much performance degradation on desktop until we reached 200+ mb of added geoJSON data.

For making a choropleth, I'd recommend you add the data as a single geoJSON source (if it is huge, serve it as .mbtiles) – and then create as few layers as possible (one for each color) and set the layer filter on the fly for each color.

Also, if you want/need a livable mobile experience – try to stick with less than 120 layers in total. We're seeing consistent crashing on iPhone 6 after tipping 150. Don't even want to think about whats happening on older iOS devices. Android devices fares a lot better in general.

[vicapow]

Oh, @kristfal interesting. That's worth investing more. How where you able to create multiple layers for a single geojson source? Would love to give that a try, just don't see how that's possible.

From what I've observed (checkout the flame graph in the second screenshot above), a lot of time is spent in postMessage. I'd be willing to bet there's a lot of info going to and from the web workers that doesn't need to be.

[jfirebaugh]

@vicapow Can you use a URL in the data option of the GeoJSON source so that it is directly loaded in the worker rather than serialized/copied from the main thread?

[jfirebaugh]

Related issues: #1504, #1505.

[kristfal]

@vicapow Call addlayer after you add source.

Here is a generic method I use for adding, updating and removing geoJSON data and styles. Using Polymer data bindings, hence one argument.

Object structure looks like this:

{
  "layers": {
    "layer1": {
      "id": "layer1",
      "type": "line",
      "interactive": true,
      "source": "geoJSON",
      "paint": {
        "line-color": "#7EC4ED",
        "line-width": 5,
        "line-opacity": 0.3
      }
    },
    "layer2": {
      "id": "layer2",
      "type": "line",
      "interactive": true,
      "source": "geoJSON",
      "paint": {
        "line-color": "#7EC4ED",
        "line-width": 5,
        "line-opacity": 0.3
      }
    }
  },
  "source": "geoJSON",
  "data": {
    "type": "FeatureCollection",
    "features": [
      {
        (...)
      }
    ]
  }
}

Method:

/**Add, update or remove geoJSON data and layers
*/

setGeoJSON: function (object) {
        var ls = object,
            sourceObj
        // Remove existing layers if present
        Object.keys(ls.layers).forEach(function (key) {
          try {
            this.mgl.removeLayer(Object.keys(ls.layers)[i]);  
          } catch (err) {
            // No layers to remove. Improvement: If mgl exposes getlayer, use if instead of try.
          }
        });
        // Remove existing source if present.
        if (this.mgl.getSource(ls.source)){
          this.mgl.removeSource(ls.source);
        }
        // Update source and set style if data is present.
        if (ls.data !== null) {
          // Set new source
          sourceObj = new mapboxgl.GeoJSONSource({
            data: ls.data
          });
          this.mgl.addSource(ls.source, sourceObj);
          // Set new layers
          this.mgl.batch(function (batch) {
            Object.keys(ls.layers).forEach(function (key) {
                batch.addLayer(ls.layers[key]);
            });
          });    
      }
    }

[jfirebaugh]

@kristfal Nice example. Another improvement you could make to it would be to use the batch API.

[jfirebaugh]

Oh, I see it is using it for addLayer, but not for removeLayer, removeSource, and addSource. Any reason for that?

[kristfal]

@jfirebaugh The method was made in mind for adding/removing one source at the time along with multiple styles, no other reason – otherwise I'd add batch for addSource.

When it comes to removeLayer, iirc the entire batch failed if one of the layers was not present. This could happen in my use-case, so it was safer to iterate layer by layer.

On a side note: try/catch feels quite cheap. Would be great to expose getLayer.

[vicapow]

@kristfal In your example snippet, how is that enough information to know what feature to use? Is the layer ID also used to find the associated feature in the feature collection?

[kristfal]

@vicapow Ah, yes, forgot to detail that part. Expanded continuation of last example:

Pass along properties in the geoJSON object.

  "data": {
    "type": "FeatureCollection",
    "features": [
      {
        "type": "Feature",
        "geometry": {
          "type": "LineString",
          "coordinates": [
            [
              10.63881115057565,
              59.9132451089657
            ],
            [
              10.63893210143842,
              59.91327620582647
            ]
          ]
        },
        "properties": {
          "id": "55d44052d1b00f4d38dbb446"
        }
      }
    ]
  }

Set style filters that matches the properties:

"layer2": {
      "id": "layer2",
      "type": "line",
      "interactive": true,
      "source": "geoJSON",
      "filter": [
        "==",
        "id",
        "55d44052d1b00f4d38dbb446"
      ],
      "paint": {
        "line-color": "#7EC4ED",
        "line-width": 5,
        "line-opacity": 0.3
      }
    }

You can update filters afterwards with setFilter. If you want to update multiple layers at the same time, be sure to use the batch API if you can. It is a lot faster than iterating.

[vicapow]

I'm going to close this for now, since using the filters was enough to address the performance issues for my use case.

[musicformellons]

@kristfal You mention serving a layer as .mbtiles which sounds interesting to me. The advantage would be that just data in the viewport is 'served' right?

How would that work exactly? I suppose following components are needed!?:

1. make .mbtiles from the geojson
2. serve the tiles on your own server via tilestache (or something else?)
3. add it as a source to mapbox gl js

Did you manage to get this working?

[kristfal]

Hey @musicformellons,

1. I'd recommend Tippecanoe, it is fast, simple to use and has a nice API.
2. Unless you really need to host the data in-house, I'd use Mapbox's hosting service. It is cheap and scales well. Just upload your mbtiles to it and you're good to go.
3. Mapbox's hosting service also includes a tool called Mapbox Studio which allows you to build custom map styles based on your custom uploaded data. The alternative right now is to write it by hand following the GL style spec. The style file can then be used in mapbox-gl-js to serve your newly made custom maps.