Convert Natural Earth's road levels into shield enums, cleanup names

[nvkelso]

Picking up a thread in #890 (comment)...

We're exporting Natural Earth's road level values in tiles now, but they're only really useful for generating road shields. But they're not really setup that way. Let's use this as a stub for what highway shield enums should look like per country (country codes in the sov_a3 and ref values are in the name column (I've verified there isn't super long text in there).

• Interstate – surprisingly these are in all three USA, CAN, MEX, with the vast majority being in USA only.
• E - E roads in Europe, but Europeans don't always seem to take kindly to these types of shields (preferring their own national shields).
• Federal – with sov_a3 values of USA, CAN, MEX, and which looks to be New Zealand (and can be limited using thecontinent value of Oceania.
• Ferry – there are a few ferries with "shield text" ref values in the name column. They look to all be weird E road continuations.
• State - No indication of which state is given.
• Other - There are a few name values, but not many. Random small roads around North America.
• ``(empty), same asU/C below, there are a lot of these.
• U/C – there are only 5 of these

[zerebubuth]

A European's perspective on "E" roads: They're a post-hoc addition, often not visible on signage. For example, the E-30 follows the M4 motorway east from south Wales towards London and (according to Wikipedia) turns north onto the M25 London orbital motorway. Mapillary has a good photo of the junction signage, which shows M25 to Watford, but no E-30 to Moscow. So if E-30 was shown on the map instead of M25, then I'd be pretty confused.

It would be nice to have the information in tiles and perhaps show E roads as extra, secondary shields if there's space, but showing them instead of national shields would probably make it more difficult for people to navigate.

[nvkelso]

Merging #900 here as it's all the same work:

• What did you see? Refs are actually in the names, there are more potential refs in the alt names.
• What did you expect to see? Less junk!
• add ref: { col: name } to the &ne_properties block when it's not one of (Sunshine Coast Highway, Seward Hwy., and Portage Glacier Hwy.).
• If it is one of those 3 values (or just a long shield text?), then put it into name.
• level seems like it should be converted to OSM style prefix on the ref values.
• namealt seems like another ref so use the OSM setup to add it to the ref value with a ; separator?
• namealtt is like level in that it indicates which shield artwork to apply for the namealt value above.
• remove labelrank, it seems to be junk data

[zerebubuth]

Looking at the data in Canada, the TCH leaving Vancouver heading east is level=Interstate, name=1, but then it stops at Hope and although the TCH continues across all of Canada, I can't find another level=Interstate line in Canada.

Is this the mapping we want?

   level    | sov_a3 |   continent   | network
------------+--------+---------------+---------
 E          |        | Asia          | e-road
 E          |        | Europe        | e-road
 Federal    | CAN    | North America | ca_??_primary
 Interstate | CAN    | North America | ca_transcanada
 Federal    | MEX    | North America | MX:MX
 Interstate | MEX    | North America | MX
 Federal    | USA    | North America | US:US
 Interstate | USA    | North America | US:I
 Federal    |        | Oceania       | NZ:SH

Rather annoyingly, it seems that there isn't a universally well-followed convention for the network tag. This means we might want to try to normalise them, or at least whitelist the ones which fit the schema we expect. The Canadian ones, for example, often seem to use _ instead of : in their hierarchy, and there's a lot of case-insensitivity going on.

The Canadian networks also seem to put the different segments of the roads in OSM into each province, which makes me wonder if level=Federal doesn't really fit in Canada. We have no way of knowing which province the road is in, unless we add that data by intersecting the roads with the province boundaries. We could invent a network value, but that would need special treatment in the style.

[nvkelso]

Yes, I'm fine saying something like ca_??_primary (or CA:??:primary) to complete this issue. Looks like we'd need to say that for level IN (Interstate, Federal, State), though.

I'm proposing we ignore assigning ca_transcanada at all from Natural Earth (roll it intoca_??_primary), because...

Turns out there are many CAN Interstate roads in Ontario that aren't part of the ca_transcanada route IRL or in OSM's route relation (and the most of the NE route is tagged Federal instead). And to further complicate matters a major branch of the TransCanada is shielded as Yellowhead Highway instead of the mapple leaf. We could spend some time intersecting &/or looking at ref values, but it doesn't appear OSM is setting good network relations for other "Interprovincial" routes that should be custom shielded anyhow. Canada, you are weird! We'd love to help but sort yourself out!

Plus since this just applies to low zooms, it's unlikely different Canada "province" shields would be used to style these shields at these zooms, and we don't expect to have that shield artwork for any house style until at least v1.1 tiles release.

If it's an problem in the future we can file a new issue to intersect with province boundary, or wait for Natural Earth to do that for us upstream.

See also:

• https://en.wikipedia.org/wiki/Numbered_highways_in_Canada
• Highways Canada federal gov't pays out to maintain (seems totally unrelated)
• https://en.wikipedia.org/wiki/Numbered_highways_in_Canada
• https://en.wikipedia.org/wiki/National_Highway_System_(Canada)
• Crowsnest Highway - different shield
• Alaska Highway - different shield
• Mackenzie Highway - different shield

[nvkelso]

[nvkelso]

If we sorted out the bad Interstates in the capital region we'd mostly have the ca_transcanada, but let's not do that now:

[nvkelso]

This is working! But a few caveats...

I'm still seeing properties for (ref a retrospectively confusing comment in #896 (comment)) that I think we should remove for v1 (and never should be been included at zoom less than 7 anyhow):

Remove properties:

• level
• namealt
• namealtt

NOTE: Looks like we could sometimes get more shield info by looking at those properties in Europe. But that's for another day (especially since they are mostly E-Roads).

General screenshots:

[nvkelso]

Also the this is outstanding:

• add ref: { col: name } to the &ne_properties block when it's not one of (Sunshine Coast Highway, Seward Hwy., and Portage Glacier Hwy, in those cases leave it in name, but otherwise remove name repurposing those values for ref).

[zerebubuth]

Those names (Sunshine Coast Highway, Seward Hwy., and Portage Glacier Hwy) come from namealt. Do you want the logic to be:

if namealt in ('Sunshine Coast Highway', 'Seward Hwy.', 'Portage Glacier Hwy.') then
  output['name'] = namealt
else
  output['ref'] = name
end

Did you want output['ref'] = name on those special ones as well? Although it looks like Seward is the only one with a name, '9', but it is AK-9, so that makes some sense.

[nvkelso]

Nice! That makes sense to me.

Looking at the queries.yaml logic on queries.yaml#L494-L504, I wonder if we should NOT be dropping the shield information at lower zooms (else the only zoom this will show up in is zoom 7).

Here's the current logic:

  - fn: vectordatasource.transform.drop_properties
    params:
      source_layer: roads
      start_zoom: 0
      end_zoom: 14
      properties: [name, ref, network, shield_text, all_networks, all_shield_texts]
      where: >-
        (kind == 'rail' and zoom < 15) or
        (kind == 'minor_road' and zoom < 14) or
        (kind == 'major_road' and zoom <  7) or
        (kind == 'highway' and zoom < 7)

But it feels like we should keep just the singular network and shield_text on highways at lower zooms.

So something more like:

  - fn: vectordatasource.transform.drop_properties
    params:
      source_layer: roads
      start_zoom: 0
      end_zoom: 14
      properties: [name, ref, network, shield_text, all_networks, all_shield_texts]
      where: >-
        (kind == 'rail' and zoom < 15) or
        (kind == 'minor_road' and zoom < 14) or
        (kind == 'major_road' and zoom <  7)

  - fn: vectordatasource.transform.drop_properties
    params:
      source_layer: roads
      start_zoom: 0
      end_zoom: 6
      properties: [name, ref, all_networks, all_shield_texts]
      where: >-
        (kind == 'highway')

(Please verify the end_zoom: 6 part, I get confused by our < or <= each time!)

And looking at zoom 7 we are generating the right shield bits from Natural Earth now, so the singular part should "just work" then.

[zerebubuth]

Problem: The filters which drop name at low zoom act to prevent the "Sunshine Coast Highway" coming through. Given that we don't have other names on NE data, should we stop the drop_properties at zoom 8, the first OSM zoom, and just leave the NE data how it is? Or do we want to drop name or ref at zoom 4 or 5 to help with merging?

[nvkelso]

Drrr. If they're just going to get dropped, let's just not export names sourced from Natural Earth (no changes to dropping behavior, only export the ref values sourced from NE when they aren't those couple string values).

[nvkelso]

👍