Merge polygons in landuse layer.

[zerebubuth]

This merges polygons in the landuse layer to improve tile size. Two different types of merging are used; at higher zooms we try to merge into as large a multipolygon as possible and at mid and low zooms we also do a dilate/erode step to try and blend together adjacent polygons. This can be extremely effective when merging large polygons separated by small spaces, such as paths within a forest.

For example, tile 9/263/166 near Groningen, the Netherlands originally had a 663kB landuse layer, but this reduced to 84kB, a drop of 87%. The total tile size went from 859kB to 280kB, a drop of 67%. Of the remaining size, 155kB is roads.

The vast majority of the features in this tile came from kind: grass polygons:

It looks as if each individual field has been mapped as a separate feature (or rather, imported). There are many small gaps between adjacent polygons, often for small water courses or paths. This means it's hard enough to merge the geometries - and on top of that, the different sizes of adjacent features means that wildly different min_zoom values get set.

Re-using the buffering merge implemented earlier for buildings in #1698 and allowing different min_zooms to be merged together (it's generally purely area-based, unlike buildings), gives:

Which still shows a lot of detail in the boundary, but has almost entirely removed small gaps and voids.

The irony is that the grass landuse kind doesn't even seem to be rendered by Bubble Wrap at zoom 10, so here's a screenshot of a slightly modified style:

Connects to #1721.

[nvkelso]

Just curious, why is this 1 more zoom at 15 (versus 14) than in v1.6?

[zerebubuth]

I found that we still had some tiles at zoom 13 & 14 which had quite a lot of grass landuse. In the particular case I was looking at, they were all suburban lawns and, while the polygons couldn't be merged very well, they benefited from being merged into the same MultiPolygon.

So I added the second merge_polygon_features, which does a simpler merge step at zooms 13 & 14. I extended the range of the drop_small_inners to cover those zooms too, just to make sure we would be dropping any courtyards or cut-outs in those lawn features, or fields elsewhere.

For the lawns, I didn't get as substantial savings (only about 30% smaller for the layer - 15% smaller for the whole tile), and I figured we could just look at it when we have a global build and figure out whether this merging is doing what we want.

[nvkelso]

What is the min_zoom of the resulting polygon, I'm reading this as it's stripped off?

Could it be the min of the input min_zoom values instead?

[nvkelso]

(Perhaps this is answered in the new _props_merge function below... but can you walk thru an example, please?)

[zerebubuth]

Yup, that's exactly what's happening. When _merge_features_by_property runs, it calls our _props_pre function for each feature, and whatever we return is the "key" used to group features to be merged. We pop the min_zoom off the properties so that the merge algorithm will merge adjacent polygons with different min_zoom - which happens all the time in the patchwork of highly variable-sized fields in the 3dshapes import.

However, _merge_features_by_property (now) also keeps the original properties, and passes the list of those into our _props_merge function below, and the merged feature gets assigned whatever we return. The _props_merge implementation I've used here takes the minimum of all the min_zooms, as this will guarantee that we see the merged feature at least at all the zooms of the biggest feature we queried from the database or RAWR tile.

The step-by-step run-through might be something like:

1. We call merge_polygon_features on a layer with two features, [(shape1, props1, fid1), (shape2, props2, fid2)].
2. merge_polygon_features calls _merge_features_by_property.
3. _merge_features_by_property calls back into merge_polygon_features' _props_pre for each feature, where we strip the min zoom. In this example, props1.pop('min_zoom') == props2.pop('min_zoom') == props.
4. _merge_features_by_property stores the features indexed by that result, so we get something like: {props: [(shape1, props1), (shape2, props2)]}
5. _merge_features_by_property tries to merge shape1 and shape2, getting shape_merged
6. _merge_features_by_property calls back into merge_polygon_features' _props_merge with [props1, props2], and we take the min of all the min_zoom parameters, returning props_merged.
7. _merge_features_by_property calls back into merge_polygon_features' _props_post, where it resets the area on props_merged (and is why we don't bother with that in step 6).
8. _merge_features_by_property returns a new list of features: [(shape_merged, props_merged, some_fid)], where some_fid is one of the fids of one of the merged features.

[nvkelso]

Huge file savings, huzzah!

I had one minor question about min_zoom (I think some minimum min_zoom value should still be present on the output). Other than that, LGTM.

[nvkelso]

Agree that all the landuse / landcover polygons could be added to Bubble Wrap now that we have good coverage. I think they've been in Walkabout for a while...

[nvkelso]

I made a GIF from your two frames: