-
Notifications
You must be signed in to change notification settings - Fork 120
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Alternate viewpoints (v0.1) #1895
Conversation
Until now, we had only been including boundary features from the `planet_osm_polygon` table and nothing at all from the `planet_osm_line` table. However, the disputes and claims are linear features, so we needed to include those. However, because `osm2pgsql` creates both linear and polygonal features for boundaries (ways and relations), then we must exclude all the linear features that are in `planet_osm_line` _except_ for the disputes and claims. This is further complicated because we change the geometry type during the query, dropping the polygon for its (oriented) boundary. The way I've done this is to include a new flag, `mz_boundary_from_polygon`, which basically means "this was a polygon, but we transformed it", and this allows us to write YAML that works. However, it's kind of a horrible hack and I mildly regret doing it. A better alternative might have been to leave the polygons in the query (appropriately clipped to an expanded bounding box) and do the boundary transform at a later stage in the stack. Lots of changes to the integration tests to adjust for the new hacky flag...
In tilezen/vector-datasource#1895, I added a little hack to work around the way we change the geometry type of `boundaries` layer features. They used to come entirely from the `planet_osm_polygon` table, but now we also include disputes and claims from `planet_osm_line`. However, we have to filter out a bunch of features from `planet_osm_line` that we don't want, but have the same tags as the polygons we do want. Long story short: The query in `vector-datasource` sets this flag when transforming a polygon into a boundary line, so we have to do the same thing when reading from the RAWR tiles.
Nice! I see |
'kind': 'unrecognized_country', | ||
'kind:bb': 'country', | ||
'kind:cc': 'country', | ||
# 'kind:dd': 'country', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TODO: This will be uncommented out once recognized_by
is plumbed thru.
|
||
self.assert_has_feature( | ||
z, x, y, 'boundaries', { | ||
'kind': 'unrecognized_country', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder, if like the name fan out we did in #1454, if we should also indicate kind:aa
and kind:dd
= unrecognized_claim
, since it's meaningful those countries have an opinion (versus generic default)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the situation is a little bit different for boundaries. For the names, we were removing data that had come as input, which broke multi-level fallback. Single-level fallback worked just fine.
Is a situation in which we'd need to do multi-level fallback? In this test example, would we ever do a fallback sequence kind:aa
, kind:bb
, kind
?
The disputed_by = AA;DD
on the claim
means that AA
and DD
dispute the claim, not whether they do or do not agree with the locations of the borders on the ground - I've assumed any viewpoint not listed in claimed_by
or recognized_by
doesn't agree (and will default to kind: unrecognized_*
).
When I first prototyped this, projecting the disputed_by
from relations onto constituent ways produced borders which were disputed_by
pretty much everyone and disappeared from the map. So I'd be very wary of using the disputed_by
information from claim
relations.
Is there a real-world example of where it's useful to have a multi-level fallback projected from the claim
relation?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is a situation in which we'd need to do multi-level fallback?
Generally countries are "aligned" with some another country in that they have a few opinions about their immediate neighbours, but lean on either ISO or the aligned country's foreign policy.
In practice someone developing in country X shouldn't need to know if Tilezen does or doesn't have data for that country's worldview (POV), but they should still indicate that as the primary POV, and ideally list out ISO or the other country Y (which we likely have in the data) as fallback, before falling back to defacto kind values.
In this test example, would we ever do a fallback sequence kind:aa, kind:bb, kind?
I think it's more likely it'd be kind:xx
(primary), kind:aa
(secondary)
To be clear, I think the fallback only happens client side not server side.
I wonder, if like the name fan out we did in #1454, if we should also indicate kind:aa and kind:dd = unrecognized_claim, since it's meaningful those countries have an opinion (versus generic default)?
Is that possible in current data structure, or would it cause problems like you described? I'm open to not doing the fan out.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, cool. I added a test for this in 74b4b7f, although the logic is getting a little bit complicated now!
'https://www.openstreetmap.org/relation/148838', | ||
], clip=self.tile_bbox(16, 10417, 25370, padding=2)) | ||
self.generate_fixtures( | ||
# https://www.openstreetmap.org/relation/148838 |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
yaml/boundaries.yaml
Outdated
# | ||
# NOTE: these come _first_ so that they match in preference to regular | ||
# country borders. (this assumes disputed_by is never set on a whole | ||
# country boundary -- perhaps true?) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These 2 are technically entirely disputed_by
their neighbours, but instead tagged with claimed_by
? How should this be setup / changed (data or processing)?
Taiwan: https://www.openstreetmap.org/relation/449220
Western Sahara: https://www.openstreetmap.org/relation/5441968
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I was mistaken when I wrote that comment. I've added a test for this in c2efb38, and it works, so I removed the comment and replaced it with better tagging recommendations.
For Western Sahara, the tagging looks good to me. I think 🤞 that it should work the next time we do a build.
For Taiwan, it's missing:
- A
boundary=claim
relation withclaimed_by=CN
andadmin_level=2
. I had a look around, and couldn't find one. It should contain the same ways as relation 449220, the country border. - The ways making up the country border need
disputed=yes
anddisputed_by=CN
on them. I didn't audit all the ways, but a few tested at random (e.g: this one) were missing those tags.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agree I need to add OSM tagging for Taiwan.
Great progress, this approach seems workable. I left comments above, will review again once |
No, it's an internal flag that's used to indicate a feature is the boundary line of an admin polygon, rather than from the lines table. I mentioned it in the PR description:
Note that we drop all |
…OSM for disputes & claims.
…r claims to include admin_level.
I'd forgotten about the general drop all |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM so far. There's a stub for _BOUNDARY_KINDS
for more than country in this PR, we can continue more work in additional PRs to make it easier to review?
Yes, sounds like a good idea! |
Added support for alternate viewpoints of country borders.
Alternate viewpoints are used where two or more countries have different opinions about where their borders are. In many cases, the governments of these countries require that maps display their particular viewpoint, so it's very practically useful to have data in the tiles to support that.
The data output in the tiles will have the usual
kind
for de-facto boundaries, but additionalkind:xx
data for the viewpoint of country XX (ISO 3166-1 alpha-2 code). For generally-recognised boundaries that XX thinks are interior to its territory, we setkind = country
,kind:xx = unrecognized_country
. For generally not recognised boundaries that XX thinks are its borders, we setkind = unrecognized_country
andkind:xx = country
. Therefore, it's possible to recover the status of any given border by a similar method we use for translations; usekind:yy
if it exists for the viewpoint YY, falling back to the regularkind
if not.This is done by looking at:
claimed_by
tag or other viewpoints listed in therecognized_by
tag. (NOTE:recognized_by
support is still a TODO).dispute(d)=yes
and with a list of viewpoints indisputed_by
will "erase" those borders for the listed viewpoints. This is done by intersecting a buffered version of the ways with the polygonally-derived country borders.Until now, we had only been including boundary features from the
planet_osm_polygon
table and nothing at all from theplanet_osm_line
table. However, the disputes and claims are linear features, so we needed to include those.However, because
osm2pgsql
creates both linear and polygonal features for boundaries (ways and relations), then we must exclude all the linear features that are inplanet_osm_line
except for the disputes and claims. This is further complicated because we change the geometry type during the query, dropping the polygon for its (oriented) boundary.The way I've done this is to include a new flag,
mz_boundary_from_polygon
, which basically means "this was a polygon, but we transformed it", and this allows us to write YAML that works. However, it's kind of a horrible hack and I mildly regret doing it. A better alternative might have been to leave the polygons in the query (appropriately clipped to an expanded bounding box) and do the boundary transform at a later stage in the stack - however this might be left for a rainy day issue.Things to note:
claimed_by
andrecognized_by
viewpoints.name
property on them is whatever thename
tag was on the claim relation or disputed way.kind
of features in that area according to the claimant viewpoint.Still TODO:
recognized_by
.Connects to #1810.