How to modify feature properties at runtime?

[ezheidtmann]

This is probably a support request or a request for documentation. Thanks for your help!

While waiting for data-driven styles and/or arithmetic operations in the style spec, I have a need to style my features based on values which can be calculated from feature properties. I think I could do this if I could add new properties to features as they arrive, and modify loaded features when I need to change the style. How can I iterate through all features and add or modify properties?

Here's an example. I have a feature with these properties:

    "properties": {
      "alltime_total_uses": 169, 
      "id": 472,
      "alltime_total_uses_bad": 7, // negative rating
      "alltime_total_uses_good": 68 // positive rating
    }

And let's say I'm coloring the features based on the property alltime_total_uses_bad. Then the user asks for the features to be colored based on the number of uses without any rating, i.e. total - bad - good. I could add a property called alltime_total_uses_unrated and then adjust the filters on my style to change the appearance of the map.

The volume and combination of options that I'd like to support makes it impractical for the backend to provide all these properties in the original tiles.

Thanks again for any advice! I will update this issue if I find a solution that works for me.

[lucaswoj]

Hi @ezheidtmann!

One way you might approach this (while waiting for data-driven styles) is to put these features in a GeoJSON source instead of vector tiles. Updating GeoJSON is easy and efficient. Take a look at this example and let me know what you think.

[ezheidtmann]

Thanks for the thoughts, @lucaswoj! I moved to vector tiles because of the quantity of features: I have a feature for (almost) every road segment in a city. And right now, my road segments are extremely short (i.e. just a single line segment), so I have around 2000 features in a zoom-14 tile. Would you still recommend using geojson for this application?

[lucaswoj]

I suspect that 2000 features in GeoJSON will still have adequate performance. Its worth a try! As far as I can see, your only other options are modifying vector tiles in the browser (possible but a pain) or using the experimental data-driven styling branch (possible but a pain).

[ezheidtmann]

OK, many thanks!

For others that may find this later: Vector tiles live in worker threads, so if I'm reading the source correctly, modifying them requires new message-passing features in worker.js.

[timrobertson100]

Thanks @ezheidtmann for explaining that you concluded it is not possible with current code. I have been struggling with this same issue. If you can recall ever finding a solution, without asking the server to rerender the vector tile, please can you point me at it?

ETA: I understand protobuf and pbf.js a little, so "modifying vector tiles in the browser" is probably not all that hard for me given I produce them in Java serverside - except I have no idea how to get a pointer to them. Any hints appreciated.

[ezheidtmann]

@timrobertson100 I think you can do it if you are willing to run a modified version of mapbox-gl-js. Because the rendering happens in worker threads, and there is no shared memory in JS workers, you will have to send modifications to the appropriate workers (and modify worker code to accept such messages) in order to achieve this goal. I solved my problem on the server, so this is all just ideas. Good luck.

[timrobertson100]

Thanks Evan. Server probably seems the most logical, but then I start reconsidering PNGs+UTFGrids or Torque again.

On 12 Mar 2016, at 20:45, Evan Heidtmann notifications@github.com wrote:

@timrobertson100 I think you can do it if you are willing to run a modified version of mapbox-gl-js. Because the rendering happens in worker threads, and there is no shared memory in JS workers, you will have to send modifications to the appropriate workers (and modify worker code to accept such messages) in order to achieve this goal. I solved my problem on the server, so this is all just ideas. Good luck.

—
Reply to this email directly or view it on GitHub.

[lucaswoj]

Again, I strongly recommend using GeoJSON data and modifying properties with setData for this use case.

[stackTom]

Is there no other alternative besides geoJSON or adding the data on the backend? I have a similar issue, but have to deal with 15 million features, so GeoJSON is out of the question.

[lucaswoj]

@stackTom Your only options currently are GeoJSON or vector tiles. @anandthakker is working on a system that allows more data formats in #929. That said, I do not anticipate that any format will significantly lessen the challenge of handling 15 million features client-side. Have you considered using the Mapbox Dataset API?

[stackTom]

Actually, I convert my GeoJSON into mbtiles using tippecanoe, have a tile server, and it works great. I was just wondering if there was a way to change my features properties on the fly in the browser, of it the properties need to be in the JSON already before making the mbtiles. I've not tried that API as I hadn't heard of it.

[lucaswoj]

Ah! I that case both #929 and the Mapbox Dataset API have the potential to be great solutions to your problem. Let me know if you want access to the dataset API.

[stackTom]

Thanks for the offer. If this is a solution to my problem, then I would love access. Before that, though, can you give me a rough idea how this would apply to my use case? My main problem is I currently have a property for each point in my GeoJSON. Once data is received from my mbtile server, the circles are colored on the map based on this property
(thanks to the new data driven styling in mapbox). I would like to add an option to my site to be able to recalculate this property value, and thus change the coloring of the markers. I need a dedicated mbtiles sever as my data is too big to simply load a GeoJSON source in the browser. Thanks for any help/ideas.

[lucaswoj]

I recommend asking Mapbox support this question. 😄

[stackTom]

Will do. Thanks again for all the help, I'll give them a link to this thread and see what they say.

[strech345]

Hello,
i need the same functionality.
because the values i will show on the map are very dynamic i coudn't put this information to the tiles. And i need and want tiles, because its a hough amount of polygons and i think tiles are the feastes technic.

For me the following workflow would be great:

• at every tile load i get all features, and the boundary of the tile
• i will request our server to get the right values of all features of this tile
• i will set the value to the property or do other stuff to color the features

Is this in a special way possible, or is there a other solution? Please no geojson, i need tiles ;-)

[timrobertson100]

@strech345 - I ended up producing vector tiles on the fly with the data I needed. Would that be possible?

You can't get at the tile in the browser easily. I tried, and it is very invasive changes to the javascript which would be difficult to maintain over time.

[strech345]

@timrobertson100 - Thanks,
i don't want to prefer this solution, but i'm interessting in it ;-). What software you use for? And how fast it will be generated?
Its possible to get at load the tile raster cell numbers. I think with the size 512px per tiles and the z-level it would be possible to calculate the boundary coordinates of the tiles. Does anyone knows how to do this?

[strech345]

to convert the tile coordinate to geographic coordinate i found a solution : SphericalMercator.bbox
https://github.com/mapbox/mapbox-studio-classic/blob/5ac2ead1e523b24c8b8ad8655babb66389166e87/ext/sphericalmercator.js

map.showTileBoundaries =true;
map.on('dataloading', function (e) {
    if(!e.tile)return;
    var coord = e.tile.coord;
    var mec = new SphericalMercator({size : 512});
    var test = mec.bbox(coord.x, coord.y, coord.z);
    console.log(coord.z + '/' + coord.x + '/' + coord.y + ' : ' + test.toString()); 
})

[timrobertson100]

@strech345 - I'm using Java and render data in 2 ways for a bunch of map views.

1. Where there are less than 250,000 records the backend (HBase) presents the data as a list of features with lat, lng, year, type , count. I then filter to the required year/type ranges, accumulate counts, project to whichever projection I'm dealing with, and render the vector tiles on the fly. This takes around 75msecs, and looks like this (arctic projection):
   
2. Where there are more than 250,000 records I preprocess the data into a tile pyramid (Spark processing) and store the data (HBase) in a vector tile where there is a layer for each type, and the metadata holds year:count pairs. I do this for the 4 map projections we support. These can be large for 650,000,000 points (10MB) but then I apply the year range, type filters on the server and reduce the metadata to just have total count. This brings them down to under 2MB - still too slow for client side rendering in production but I swap to Mapnik for server side rendering for that. This rewriting process takes around 300msecs. The tiles of course can be rendered client side and looks like this for 650m records (WGS84 plate carrée projection):
   

One thing I quite like it binning the data to things like hexagons. This is a processing layer on top of the point data layer, which adds a binning process to rewrite the point tiles into polygon tiles. This requires buffering to handle boundary cases (hexagons need to span tile boundaries to tesselate), so I have 512px tiles with 64px buffers in the backend for the large data tiles (2. above) and for the other views (item 1. above) I just produce them on the fly from lat lng global data:

For backwards compatibility etc I just render in PNGs using Mapnik. Even using 10MB vector tiles as a unit to pass around server side and filter works well, as long as they stay server side and are compressed to PNGs for the client delivery. E.g.:

All of this work is available here if you want to browse around: https://github.com/gbif/maps

In particular the filtering of tiles is: https://github.com/gbif/maps/tree/master/common/src/main/java/org/gbif/maps/common/filter

[lucaswoj]

I just documented the steps we'll take to support a custom source API here: #3326. Always looking for help! 🙏

[strech345]

@timrobertson100 Thanks for the big infos and the nice pics. Yeah i think for millions of points it could be the right way to aggregate it on server. What format you use for the 250.000 records? And which software you use for creating vector tiles?

[timrobertson100]

Hi @strech345 - It's all custom code I'm afraid. For the 250,000 records I encode as Protobuf, but my own schema. For the vector tiles, I use the Java library and I provided a few patches to improve the performance for large point data.

[strech345]

Is there now a solution to add, change properties doring the feature live time?
I'm now loading the properties and create for every range one layer and set the filter by id depending on the loaded value. Thats not nice but works.
So Im interested to find a better solution.

[ezheidtmann]

@strech345 The "expressions" feature that landed last month might help, depending on your requirements. https://blog.mapbox.com/announcing-expressions-in-gl-js-a72b55d0a6af

[strech345]

@ezheidtmann
Thanks. But with the new expressions for me it looks like its only works with included paraleters of the tiles and not dynamically added. I'm right?

[strech345]

@ezheidtmann
You are right, its possible to use outside data :-), I only have to check how it will works by edit my data dynamic by add and remove depending on tile load / unload.

var myData = {"a1":'#EED322'};

map.on('load', function () {
    map.addLayer({
        'id': 'maine',
        'type': 'fill',
        'source': {
            'type': 'geojson',
            'data': {
                'type': 'Feature',
                'geometry': {
                    'type': 'Polygon',
                    'coordinates': [[
			[-67.13734351262877, 45.137451890638886],
                        [-69.06, 43.98],
                        [-71.08482, 45.3052400000002],
                        [-70.6600225491012, 45.46022288673396],
                        [-67.13734351262877, 45.137451890638886]]]
                },
		'properties':{
			"id": "a1"
		}
            }
        },
        'layout': {},
        'paint': {
            'fill-color':
		["get", ["to-string", ["get", "id"]], ["literal", myData]]
        }
    });
});

[strech345]

Does someone know how to get value of pv?

var myData = {"a1":{"pv":'#EED322'}};

[strech345]

and a second Question, why the expression doesn't work by using setPaintProperty. It looks like only values are possible.

map.setPaintProperty('mainel', 'fill-color', 
  ["get", ["to-string", ["get", "id"]], ["literal", myData]]);

[ezheidtmann]

@strech345 Not sure exactly what you're trying to do, but here's something: Vector tiles (and thus GeoJSON loaded into Mapbox GL) don't support nested properties. When a GeoJSON object is loaded into Mapbox, any nested properties are serialized into JSON strings.

It looks like expressions also don't support nesting, even for literals. Maybe you need to flatten your myData object into a single-level { [id]: color } shape.

[strech345]

now map.setPaintProperty() works with version 0.42.1 :-)

[josiekre]

@stackTom Did Mapbox support help you find a solution for recalculating a property value that controls the coloring of your markers served from your own mbtiles sever (not GeoJSON)?

[stackTom]

@josiekre For my particular use case, I ended up just overlying a layer of GeoJSON on top of my mbtiles layer whenever I need to change the colors of the markers. I have not revisited the issue, though, because this solution serves my needs for the time being.