Add OSM alternate viewpoints for country boundaries

[nvkelso]

In #1809 we added schema support for alternate points of view in the boundaries layer, and added data from Natural Earth.

This issue tackles adding data parsing from OpenStreetMap, where the data for this is sparse today. And we'll need to figure out how to build polygons out of them so we can still do the left/right boundary labels.

[zerebubuth]

I've been playing around with the data for this for a while, and it doesn't look like the data is even in a suitable state today to build sensible-looking lines, let alone polygons for left/right labels. For example, this is the "on the ground" view from OpenStreetMap:

And this is what the data indicates India's view should be:

And what the data says Pakistan's view should be:

And what the data says China's view should be:

What's being shown?

This is looking at the disputed_by tag on the way and the claimed_by tag on relations that it is part of. Weird, I know, but the disputed_by tag on the relations means that the claim is disputed - and dropping the way on that basis leads to loads of gaps. If a way is part of a country boundary relation (i.e: would appear in the "on the ground" view) then it's flagged "on the ground".

• If a way is in a relation claimed_by a view, it's displayed in red.
• If way is tagged as disputed_by a view, it's not shown.
• If neither of the above and it's flagged as "on the ground" then display in green.

What's going on with the extra random lines?

Most of the line-of-control between India and Pakistan is tagged as disputed_by=IN;PK, but some segments aren't. Since the line-of-control is the "on the ground" view of the border between the two countries, these untagged segments get displayed in green even though they are disconnected from other parts of the border. Looks like a simple case of bad tagging - should be easy to correct.

Some other problems appear to stem from typos, for example this alleged dispute between India and Switzerland (fix: s/CH/CN/).

How widespread is this data?

This particular schema seems to only be usefully well implemented around the three-way IN / PK / CN dispute. There's some related tagging on the IL / SY dispute, but the SY claims aren't collected into a claim relation, nor are SY's claims tagged as disputed_by IL.

Other disputes, such as Doklam, the Ilemi Triangle, Hala'ib Triangle are mapped with a simple disputed=yes tag and no claimed_by or disputed_by information as far as I can tell.

Next steps

On the basis of the flakiness of this data, I'm not sure how to reliably recover view-dependent polygons to ensure the correct sidedness of the labels. At the moment, all borders claimed_by country X are claimed to be borders of country X - but I know it gets more complicated; there are countries which have an opinion about other countries' borders, so it might take some more sophisticated logic to figure out.

[nvkelso]

See active discussion in OSM wiki:

• https://wiki.openstreetmap.org/wiki/Talk:Proposed_features/Mapping_disputed_boundaries

[nvkelso]

I've prototyped almost all of this data into OpenStreetMap. Remaining work:

• Ensure that ways in a claim are all tagged disputed=yes and disputed_by=*.
• Spit up complicated relations (like with China) into separate sections as each section is recognized (or not recognized) by other countries.
• Add recognized_by=* tags on small handful of relations.

For the tile generations:

• Add in ways that are disputed but not part of admin_level=2;4 relation but has the claimed_by tag. That would match Natural Earth, though they aren't displayed by default in OSM Carto. We could have name on this feature from the relation, but not name:left, name;right.
• We don't need to calculate POV for the left and right names on boundary lines – we can keep just doing name:left, name:right for the default POV. If there is a POV on the line, we could label it with the default name instead – but that's actually a bit hard so let's also skip that for now.
• For the boundaries that need to be "unrecognized", let's try to select those lines (based on admin_level=2;4, dispute{d}=yes, disputed_by=*), buffer them, and intersect with the name:left, name:right lines that OSM2PGSQL gives us?

[nvkelso]

We're missing data / modeling around Cyprus, followup here: #1898.

[nvkelso]

We should also add the wikidata_id concordance <> kind mapping from NE to OSM for country and region as well as some countries don't recognize each other as countries (see Israel, Taiwan) at all or as a different kind.

Followup in: #1900

[nvkelso]

@zerebubuth Do you see why Taiwan country label comes in at zoom 6? It's min_label in NE is 3.

[zerebubuth]

Do you see why Taiwan country label comes in at zoom 6? It's min_label in NE is 3.

Ah, that's interesting! The place point from OSM has a Wikidata ID of Q22502, which is Taiwan Island. However, Taiwan the "state with limited recognition in East Asia" is Q856, which is the value in the Natural Earth data. This means that the data doesn't join up, we don't assign an __ne_min_zoom and therefore we treat Taiwan as if it were an unrecognised country as per #1826.

The country boundary relation in OSM has the correct value, so I'm tempted to call this a data issue - the place point is a label for the relation, not just the island. If we want to work around this, it might be possible to link back up to the relation from the place point... but might be complicated.

[nvkelso]

Agree it's a data problem, I've edited OSM to have Q865 instead (from your link, not the link text):

• https://www.openstreetmap.org/changeset/70789652

[nvkelso]

Followup for disputed modeling in OSM (works in NE now) in #1901.

[nvkelso]

@zerebubuth how could we solve this one? We don't want Western Sahara to label on the Morocco side of the boundary on the south side of this image. But doesn't look like there's enough data in tiles to turn it off – to know that Morocco disputes the "left" side of the line to turn that label off.

[nvkelso]

NOTE: I think the stub of boundary above is https://www.openstreetmap.org/way/526798853 that had a disputed_by tag on it, but was missing disputed=yes. Fixed in https://www.openstreetmap.org/changeset/70790024.

[nvkelso]

This is working as intended for country, region, and recognized_by POV, closing.

There are a couple data issues to sort out, and some followups in Followup in: #1900 (better recasting of kind from NE wikidata_id join) and #1901 (disputed lines from OSM).

[zerebubuth]

We don't want Western Sahara to label on the Morocco side of the boundary on the south side of this image. But doesn't look like there's enough data in tiles to turn it off

You're right - we don't build the disputed / claimed shapes of the countries, so we can't automatically ensure that the label point is within the boundary always. However, we can:

1. Ensure that it's within the consensus boundary, and use the OSM country polygon to generate a point "on surface" within that polygon if the label point is outside of it. This would be kind of awkward to do, since we would need to alter the label placement location in the database.
2. Change the data upstream so that the label locations are in "viewpoint friendly" locations (i.e: not in or too close to any disputed boundaries). For countries which are wholly disputed it might be easier - since the disputant won't see the label at all.

[zerebubuth]

Ooops, sorry - I misunderstood what you meant. I was thinking of the country label, but I think you probably meant the boundary label.

The way to turn it off is somewhat complex and annoying, but analogous to the (claimed) border between India and Afghanistan. What we can do is:

1. Set disputed=yes, disputed_by=MA on the way,
2. Add a claim relation containing the way with claimed_by=MA.

Then the claim will appear for Morocco with the label in the name of the claim relation (no left/right labels). Other countries will see it with the left/right labels according to the consensus country polygons.

[nvkelso]

I'll do some more editing as advised in #1810 (comment) and then we can do a new build once that shows up in a planet file.