Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lua changes for targeted disputed borders fixes #2065

Merged
merged 77 commits into from
Mar 18, 2022
Merged
Changes from 76 commits
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
bfec354
adding disputed_by for reference lines
tgrigsby-sc Feb 4, 2022
a3e673c
switching polarity of recognition
tgrigsby-sc Feb 7, 2022
1b82db4
Update boundaries.yaml
tgrigsby-sc Feb 7, 2022
665867b
Merge branch 'travisg/20220204-disputed_by-unpacking' of https://gith…
tgrigsby-sc Feb 7, 2022
01e2755
more comments
tgrigsby-sc Feb 7, 2022
d9d52f1
starting to fix for example 1
tgrigsby-sc Feb 8, 2022
fb12139
updates
tgrigsby-sc Feb 8, 2022
2b432e8
updating comments
tgrigsby-sc Feb 8, 2022
dd7f18c
adding todos
tgrigsby-sc Feb 8, 2022
d9d49ec
Merge remote-tracking branch 'origin/master' into travisg/20220204-di…
tgrigsby-sc Feb 12, 2022
7c634a8
lua update for taiwan
tgrigsby-sc Feb 14, 2022
ab85e9b
remove disputed tags from ways
tgrigsby-sc Feb 14, 2022
0d8c2af
adding kind_detail to disputed_reference_line
tgrigsby-sc Feb 14, 2022
3d730aa
delete tz_admin_level once we're done with it
tgrigsby-sc Feb 14, 2022
f23ea41
making this work when we have a claim with only disputed_by
tgrigsby-sc Feb 15, 2022
3797723
turn disputed into administrative lua
tgrigsby-sc Feb 15, 2022
45228fb
disputed with no list of who disputes or claims
tgrigsby-sc Feb 16, 2022
edd2304
updating to reflect actual world data
tgrigsby-sc Feb 16, 2022
5ca4752
updated LUA
tgrigsby-sc Feb 16, 2022
4842284
Converts Northern Cyprus admin_level 5 features to 4
Feb 18, 2022
8f4a26d
admin_level 3 are recognized as countries
tgrigsby-sc Feb 18, 2022
275022e
Test change to apply boundary=disputed to ways involved in boundary=d…
Feb 22, 2022
33c5658
Merge remote-tracking branch 'origin/travisg/20220204-disputed_by-unp…
Feb 22, 2022
68715e3
Previous commit didn't work as expected. Trying a different approach …
Feb 23, 2022
98d4b5f
reverted lua change
Feb 23, 2022
d7af921
all disputes are now disputed_reference lines, unless you are a dispu…
tgrigsby-sc Feb 23, 2022
aa68e5e
Merge branch 'travisg/20220204-disputed_by-unpacking' of https://gith…
tgrigsby-sc Feb 23, 2022
8aa30c0
adding recognized_by and claimed_by to the dispute_mask data
tgrigsby-sc Feb 24, 2022
153f4cd
removing colon from test
tgrigsby-sc Feb 24, 2022
ebd6d60
remove disputed:True from all output
tgrigsby-sc Feb 24, 2022
3a04505
drop recognized_by from output as well
tgrigsby-sc Feb 24, 2022
d7623be
adding ne debug dispute id to all disputed output
tgrigsby-sc Feb 24, 2022
3581e05
documentation
tgrigsby-sc Feb 24, 2022
c8fe78c
updating to new understanding
tgrigsby-sc Feb 24, 2022
9c6e07f
simplifying logic
tgrigsby-sc Feb 24, 2022
e2e418f
unrecognized_country -> disputed_claim
tgrigsby-sc Feb 25, 2022
2e427aa
Fix recognized_by typo
Feb 26, 2022
b83aa5b
Testing out trimmed down lua script
Mar 1, 2022
6b496b5
Fixed bug
Mar 1, 2022
b5eafe9
Further trimming testing
Mar 2, 2022
7232975
Further trimming testing
Mar 2, 2022
f8cbceb
Added bit to remove disputed relation after moving tags to ways and f…
Mar 2, 2022
e6c29f6
apply admin_level:XX viewpoints last
tgrigsby-sc Mar 1, 2022
cd98dba
remove admin_level:3 handling
tgrigsby-sc Mar 1, 2022
c6f363a
Merge branch 'travisg/20220204-disputed_by-unpacking' of https://gith…
tgrigsby-sc Mar 2, 2022
063e5b5
support disputed_by for places
tgrigsby-sc Mar 2, 2022
4ca3e01
Reverted relation removal part as it caused unexpected issues. Update…
Mar 2, 2022
18c9181
Merge remote-tracking branch 'origin/travisg/20220204-disputed_by-unp…
Mar 2, 2022
b2871ea
Lua change to push claims to ways as well as regions we need to confl…
Mar 10, 2022
1e3e080
More changes to merge disputes onto ways
Mar 11, 2022
2948ba9
Testing out behavior
Mar 11, 2022
747bcd1
More tests
Mar 14, 2022
7f0decf
Merge remote-tracking branch 'origin/master' into junderwood/lua_changes
Mar 15, 2022
6e450d6
Merged latest changes
Mar 15, 2022
0ae7da5
test
Mar 15, 2022
04c1413
osm2pgsql changes
Mar 16, 2022
3e37ac2
reverted change
Mar 16, 2022
577348a
label updates
Mar 16, 2022
9e83abd
Possible solution for fixing claims
Mar 16, 2022
8898f2e
reduced scope of claim change.
Mar 16, 2022
b27d1b0
Change to west bank for rendering
Mar 17, 2022
e025c8c
Added Somaliland
Mar 17, 2022
d1d1c59
typo
Mar 17, 2022
79aec39
Try something else
Mar 18, 2022
bf62b6e
helps to not have typos
Mar 18, 2022
7519605
added disputant
Mar 18, 2022
594dabb
fixed lua
Mar 18, 2022
7114868
Another attempt
Mar 18, 2022
802ae74
typo
Mar 18, 2022
57cfd00
revert
Mar 18, 2022
ebace4b
fix outdated comment
Mar 18, 2022
b241e4d
sorted where possible
Mar 18, 2022
2c261d0
Redefined Israel to Palestine label for some countries
Mar 18, 2022
2562c87
Redefine West Bank and Gaza Strip to region for ID, PK, SA
Mar 18, 2022
8a1115f
Merge remote-tracking branch 'origin/master' into junderwood/lua_changes
Mar 18, 2022
e11c2d5
israel change
Mar 18, 2022
6e39e2b
Kosovo fix
Mar 18, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
238 changes: 210 additions & 28 deletions osm2pgsql.lua
Original file line number Diff line number Diff line change
@@ -227,6 +227,8 @@ local tables = {}

--for storing node tags later
local disputed = {}
local claimed = {}


tables.point = osm2pgsql.define_table{
name = prefix .. '_point',
@@ -438,41 +440,186 @@ function osm2pgsql.process_node(object)
if object.tags.place and object.tags.wikidata == 'Q200667' then
output_hstore['disputed_by'] = 'PK'
end
-- Recast Gaza Strip label as country
if object.tags.place and object.tags.wikidata == 'Q39760' then
output_hstore['place'] = 'country'
nvkelso marked this conversation as resolved.
Show resolved Hide resolved
output_hstore['place:ID'] = 'region'
output_hstore['place:PK'] = 'region'
output_hstore['place:SA'] = 'region'
end
-- Recast West Bank label as country
if object.tags.place and object.tags.wikidata == 'Q36678' then
output_hstore['place'] = 'country'
output_hstore['place:ID'] = 'region'
output_hstore['place:PK'] = 'region'
output_hstore['place:SA'] = 'region'
end
-- Recast Western Sahara label as country expect for FR;IN;PS;SA;MA;TR;ID;Pl;NL
if object.tags.place and object.tags.wikidata == 'Q6250' then
output_hstore['place'] = 'country'
output_hstore['disputed_by'] = 'FR;ID;IN;MA;NL;Pl;PS;SA;TR'
end
-- Redefine Israel country label for SA;PK;ID
if object.tags.place and object.tags.wikidata == 'Q801' then
output_hstore["name:id"] = 'Palestina'
output_hstore["name:ur"] = 'فلسطین'
output_hstore["name:ar"] = 'فلسطين'
end
-- Recast Taiwan country label as region label for China POV
if object.tags.place and object.tags.wikidata == 'Q865' then
output_hstore['place:CN'] = 'region'
end
-- Recast Taiwan region to county labels for China POV
if object.tags.place and (object.tags.wikidata == 'Q133865' or object.tags.wikidata == 'Q166977' or
object.tags.wikidata == 'Q249995' or object.tags.wikidata == 'Q74054' or object.tags.wikidata == 'Q249994' or
object.tags.wikidata == 'Q249868' or object.tags.wikidata == 'Q181557' or object.tags.wikidata == 'Q249996' or
object.tags.wikidata == 'Q249870' or object.tags.wikidata == 'Q249872' or object.tags.wikidata == 'Q63706' or
object.tags.wikidata == 'Q82357' or object.tags.wikidata == 'Q244898' or object.tags.wikidata == 'Q198525' or
object.tags.wikidata == 'Q194989' or object.tags.wikidata == 'Q245023' or object.tags.wikidata == 'Q140631' or
object.tags.wikidata == 'Q1867' or object.tags.wikidata == 'Q249904' or object.tags.wikidata == 'Q115256' or
object.tags.wikidata == 'Q237258' or object.tags.wikidata == 'Q153221') then
if object.tags.place and (object.tags.wikidata == 'Q115256' or object.tags.wikidata == 'Q133865' or
object.tags.wikidata == 'Q140631' or object.tags.wikidata == 'Q153221' or object.tags.wikidata == 'Q166977' or
object.tags.wikidata == 'Q181557' or object.tags.wikidata == 'Q1867' or object.tags.wikidata == 'Q194989' or
object.tags.wikidata == 'Q198525' or object.tags.wikidata == 'Q237258' or object.tags.wikidata == 'Q244898' or
object.tags.wikidata == 'Q245023' or object.tags.wikidata == 'Q249868' or object.tags.wikidata == 'Q249870' or
object.tags.wikidata == 'Q249872' or object.tags.wikidata == 'Q249904' or object.tags.wikidata == 'Q249994' or
object.tags.wikidata == 'Q249995' or object.tags.wikidata == 'Q249996' or object.tags.wikidata == 'Q63706' or
object.tags.wikidata == 'Q74054' or object.tags.wikidata == 'Q82357') then
output_hstore['place:CN'] = 'county'
end
-- Recast Taiwan country label as region label for China POV
-- Turn off Kosovo country label for CN;RU;IN;GR
if object.tags.place and object.tags.wikidata == 'Q1246' then
output_hstore['disputed_by'] = 'CN;GR;IN;RU'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For all other countries we consider we'd end up with kind:XX = country for the other Natural Earth POVs.

end
-- Hide Kosovo region labels for several POVs including China and Russia
if object.tags.place and (object.tags.wikidata == 'Q1008042' or object.tags.wikidata == 'Q1021775' or
object.tags.wikidata == 'Q112657' or object.tags.wikidata == 'Q124725' or object.tags.wikidata == 'Q15710469' or
object.tags.wikidata == 'Q227569' or object.tags.wikidata == 'Q248378' or object.tags.wikidata == 'Q25270' or
object.tags.wikidata == 'Q392505' or object.tags.wikidata == 'Q42328687' or object.tags.wikidata == 'Q474651' or
object.tags.wikidata == 'Q4864476' or object.tags.wikidata == 'Q59074' or object.tags.wikidata == 'Q59086' or
object.tags.wikidata == 'Q59089' or object.tags.wikidata == 'Q608274' or object.tags.wikidata == 'Q62172' or
object.tags.wikidata == 'Q733155' or object.tags.wikidata == 'Q738901' or object.tags.wikidata == 'Q739808' or
object.tags.wikidata == 'Q786124' or object.tags.wikidata == 'Q911241' or object.tags.wikidata == 'Q939112' or
object.tags.wikidata == 'Q963121' or object.tags.wikidata == 'Q991291' or object.tags.wikidata == 'Q991291' or
object.tags.wikidata == 'Q991313' or object.tags.wikidata == 'Q991332' or object.tags.wikidata == 'Q994245' or
object.tags.wikidata == 'Q994730') then
output_hstore['disputed_by'] = 'CN;RU;IN;GR'
nvkelso marked this conversation as resolved.
Show resolved Hide resolved
end
-- Recast Northern Cyprus as country label and turn off for several POVs including China and Russia
if object.tags.place and object.tags.wikidata == 'Q23681' then
output_hstore['place'] = 'country'
jeffdefacto marked this conversation as resolved.
Show resolved Hide resolved
output_hstore['disputed_by'] = 'CN;RU;IN;GR;CY'
end
-- Turn off Abkhazia label for most countries
if object.tags.place and object.tags.wikidata == 'Q23334' then
output_hstore['place:AR'] = 'region'
output_hstore['place:BD'] = 'region'
output_hstore['place:BR'] = 'region'
output_hstore['place:CN'] = 'region'
jeffdefacto marked this conversation as resolved.
Show resolved Hide resolved
output_hstore['place:RU'] = 'region'
output_hstore['place:DE'] = 'region'
output_hstore['place:EG'] = 'region'
output_hstore['place:GB'] = 'region'
output_hstore['place:GR'] = 'region'
output_hstore['place:ID'] = 'region'
output_hstore['place:IL'] = 'region'
output_hstore['place:IN'] = 'region'
output_hstore['place:IT'] = 'region'
output_hstore['place:JP'] = 'region'
output_hstore['place:KO'] = 'region'
output_hstore['place:MA'] = 'region'
output_hstore['place:NL'] = 'region'
output_hstore['place:NP'] = 'region'
output_hstore['place:PK'] = 'region'
output_hstore['place:PL'] = 'region'
output_hstore['place:PS'] = 'region'
output_hstore['place:PT'] = 'region'
output_hstore['place:SA'] = 'region'
output_hstore['place:SE'] = 'region'
output_hstore['place:TR'] = 'region'
output_hstore['place:TW'] = 'region'
output_hstore['place:UA'] = 'region'
output_hstore['place:VN'] = 'region'
end
-- Turn off South Ossetia label for most countries
if object.tags.place and object.tags.wikidata == 'Q23427' then
output_hstore['place:AR'] = 'region'
output_hstore['place:BD'] = 'region'
output_hstore['place:BR'] = 'region'
output_hstore['place:CN'] = 'region'
jeffdefacto marked this conversation as resolved.
Show resolved Hide resolved
output_hstore['place:DE'] = 'region'
output_hstore['place:EG'] = 'region'
output_hstore['place:GB'] = 'region'
output_hstore['place:GR'] = 'region'
end
-- Hide Kosovo country and region labels for several POVs including China and Russia
if object.tags.place and (object.tags.wikidata == 'Q474651' or
object.tags.wikidata == 'Q939112' or object.tags.wikidata == 'Q59074' or object.tags.wikidata == 'Q1008042' or
object.tags.wikidata == 'Q739808' or object.tags.wikidata == 'Q991332' or object.tags.wikidata == 'Q248378' or
object.tags.wikidata == 'Q963121' or object.tags.wikidata == 'Q786124' or object.tags.wikidata == 'Q124725' or
object.tags.wikidata == 'Q42328687' or object.tags.wikidata == 'Q991313' or object.tags.wikidata == 'Q994730' or
object.tags.wikidata == 'Q59089' or object.tags.wikidata == 'Q15710469' or object.tags.wikidata == 'Q608274' or
object.tags.wikidata == 'Q733155' or object.tags.wikidata == 'Q112657' or object.tags.wikidata == 'Q994245' or
object.tags.wikidata == 'Q25270' or object.tags.wikidata == 'Q4864476' or object.tags.wikidata == 'Q991291' or
object.tags.wikidata == 'Q991291' or object.tags.wikidata == 'Q392505' or object.tags.wikidata == 'Q59086' or
object.tags.wikidata == 'Q738901' or object.tags.wikidata == 'Q1021775' or object.tags.wikidata == 'Q911241' or
object.tags.wikidata == 'Q227569' or object.tags.wikidata == 'Q62172') then
output_hstore['disputed_by'] = 'CN;RU;IN;GR'
output_hstore['place:ID'] = 'region'
output_hstore['place:IL'] = 'region'
output_hstore['place:IN'] = 'region'
output_hstore['place:IT'] = 'region'
output_hstore['place:JP'] = 'region'
output_hstore['place:KO'] = 'region'
output_hstore['place:MA'] = 'region'
output_hstore['place:NL'] = 'region'
output_hstore['place:NP'] = 'region'
output_hstore['place:PK'] = 'region'
output_hstore['place:PL'] = 'region'
output_hstore['place:PS'] = 'region'
output_hstore['place:PT'] = 'region'
output_hstore['place:SA'] = 'region'
output_hstore['place:SE'] = 'region'
output_hstore['place:TR'] = 'region'
output_hstore['place:TW'] = 'region'
output_hstore['place:UA'] = 'region'
output_hstore['place:VN'] = 'region'
end
-- Turn off Nagorno-Karabakh label for most countries
if object.tags.place and object.tags.wikidata == 'Q2397204' then
output_hstore['place:AR'] = 'region'
output_hstore['place:BD'] = 'region'
output_hstore['place:BR'] = 'region'
output_hstore['place:CN'] = 'region'
output_hstore['place:DE'] = 'region'
output_hstore['place:EG'] = 'region'
output_hstore['place:ES'] = 'region'
output_hstore['place:FR'] = 'region'
output_hstore['place:GB'] = 'region'
output_hstore['place:GR'] = 'region'
output_hstore['place:ID'] = 'region'
output_hstore['place:IL'] = 'region'
output_hstore['place:IN'] = 'region'
output_hstore['place:IT'] = 'region'
output_hstore['place:JP'] = 'region'
output_hstore['place:KO'] = 'region'
output_hstore['place:MA'] = 'region'
output_hstore['place:NL'] = 'region'
output_hstore['place:NP'] = 'region'
output_hstore['place:PK'] = 'region'
output_hstore['place:PL'] = 'region'
output_hstore['place:PS'] = 'region'
output_hstore['place:PT'] = 'region'
output_hstore['place:RU'] = 'region'
output_hstore['place:SA'] = 'region'
output_hstore['place:SE'] = 'region'
output_hstore['place:TR'] = 'region'
output_hstore['place:TW'] = 'region'
output_hstore['place:US'] = 'region'
output_hstore['place:VN'] = 'region'
end
-- Turn off Somaliland label for most countries
if object.tags.place and object.tags.wikidata == 'Q34754' then
output_hstore['place:AR'] = 'region'
output_hstore['place:BD'] = 'region'
output_hstore['place:BR'] = 'region'
output_hstore['place:CN'] = 'region'
output_hstore['place:EG'] = 'region'
output_hstore['place:GR'] = 'region'
output_hstore['place:ID'] = 'region'
output_hstore['place:IL'] = 'region'
output_hstore['place:IN'] = 'region'
output_hstore['place:MA'] = 'region'
output_hstore['place:NP'] = 'region'
output_hstore['place:PK'] = 'region'
output_hstore['place:PL'] = 'region'
output_hstore['place:PS'] = 'region'
output_hstore['place:PT'] = 'region'
output_hstore['place:RU'] = 'region'
output_hstore['place:SA'] = 'region'
output_hstore['place:SO'] = 'region'
output_hstore['place:TR'] = 'region'
output_hstore['place:TW'] = 'region'
output_hstore['place:UA'] = 'region'
output_hstore['place:VN'] = 'region'
end

output.tags = output_hstore
@@ -541,13 +688,24 @@ function osm2pgsql.process_way(object)
output_hstore.disputed_by = nil
output_hstore.recognized_by = nil
output_hstore.dispute = nil
output_hstore.disputed = nil
end

-- Redefine extra admin ways as disputed
-- Redefine extra disputed admin ways as administrative to avoid them
if object.tags.boundary == 'disputed' then
output_hstore.boundary = 'administrative'
end

-- Adds suppress any ways involved with claims. The claim relation will render instead for everyone.
for k, v in pairs(claimed) do
if k == object.id then
output_hstore.dispute = 'yes'
if v.disputed_by then
output_hstore.disputed_by = 'AR;BD;BR;CN;DE;EG;ES;FR;GB;GR;ID;IL;IN;IT;JP;KO;MA;NL;NP;PK;PL;PS;PT;RU;SA;SE;TR;TW;UA;US;VN'
end
end
end

-- Adds dispute tags to ways in disputed relations
for k, v in pairs(disputed) do
if k == object.id then
@@ -620,8 +778,20 @@ function osm2pgsql.process_relation(object)
return
end

-- Adds tags from boundary=disputed relation to its ways then discards the relation to remove redundancy
if (type == 'linestring' or type == 'boundary') and object.tags.boundary == 'disputed' then
-- Adds tags from boundary=claim relation to its ways
if (type == 'linestring' or type == 'boundary') and object.tags.boundary == 'claim' and object.tags.admin_level == '4' then
for _, member in ipairs(object.members) do
if member.type == 'w' then
if not claimed[member.ref] then
claimed[member.ref] = {}
end
claimed[member.ref] = object.tags
end
end
end

-- Adds tags from boundary=disputed relation to its ways
if (type == 'linestring' or type == 'boundary') and (object.tags.boundary == 'disputed') then
for _, member in ipairs(object.members) do
if member.type == 'w' then
if not disputed[member.ref] then
@@ -643,7 +813,11 @@ function osm2pgsql.process_relation(object)
-- Adds tags to redefine Israel admin 4 boundaries for Palestine.
if type == 'boundary' and (object.tags.admin_level == '4') and object.tags['ISO3166-2'] then
if osm2pgsql.has_prefix(object.tags['ISO3166-2'], 'IL-') then
output_hstore['admin_level:PS'] = '6'
output_hstore['admin_level:BD'] = '8'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will require a style sheet change, which is ok.

But I think a better result would be kind:xx = unrecognized in the output tiles because we want a way to turn them off while still showing locality lines elsewhere. Is there a different way you can tag these here to achieve that result?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can stay.

output_hstore['admin_level:ID'] = '8'
output_hstore['admin_level:PK'] = '8'
output_hstore['admin_level:PS'] = '8'
output_hstore['admin_level:SA'] = '8'
end
end

@@ -658,6 +832,14 @@ function osm2pgsql.process_relation(object)
output_hstore['admin_level'] = '4'
end

-- Turn off West Bank and Judea and Samaria relations
if type == 'boundary' and object.tags.wikidata == 'Q36678' then
output_hstore = nil
end
if type == 'boundary' and object.tags.wikidata == 'Q513200' then
output_hstore = nil
end

if enable_legacy_route_processing and (hstore or hstore_all) and type == 'route' then
if not object.tags.route_name then
output_hstore.route_name = object.tags.name