Upgrade tileservers to mapnik3

[gravitystorm]

On behalf of the openstreetmap-carto project, we'd like to see the OSMF tileservers upgraded to using mapnik3. We are confident that our stylesheet works on both mapnik2 and mapnik3 and are happy to make any tweaks that are uncovered.

Most importantly, there's a step-change improvement in text rendering quality for non-latin scripts just by upgrading.

[tomhughes]

There's a PPA that Dane maintains with mapnik3 packages right?

[tomhughes]

Hmm apparently not for mapnik 3

[tomhughes]

Sent a message via launchpad asking if they plan to add a PPA for the 3.x releases.

[tomhughes]

I guess the other question is, what's the story with mod_tile? Is it mapnik 3 ready?

[pnorman]

I guess the other question is, what's the story with mod_tile? Is it mapnik 3 ready?

Supposedly it builds, but I doubt it's very tested.

I opened a ticket mapnik/debian#17. I have upload access, but not PPA creation access.

[pnorman]

I'm working to get Mapnik 3 into trusty-backports, as mapnik/debian#17 has sat unanswered for 3 months.

[matthijsmelissen]

That's really useful. Thanks @pnorman!

[tomhughes]

So does that mean the latest version of Ubuntu already has it?

If so, and you don't want to try and get it into backports, or aren't able to, then we can always just put a build in our PPA instead.

I mean if it's going to be in 16.04 then we're not far from that anyway.

[pnorman]

So does that mean the latest version of Ubuntu already has it?

Yep. 16.04 has 3.09, 15.10 has an earlier 3.0.x

If so, and you don't want to try and get it into backports, or aren't able to, then we can always just put a build in our PPA instead.

I want to get it into backports for wider use and writing guides. From OSGeo Live I understand it builds fine against, so it should be fairly easy once I work through the backport process.

I mean if it's going to be in 16.04 then we're not far from that anyway.

Ya, 16.04 will come with several upgrades which will make all of this much easier.

Once I have Mapnik 3 going, I still need to test mod_tile

[pnorman]

xenial (16.04) to trusty (14.04) backport built cleanly, but I still need to test it, and test with rdepends like mapnik-vector-tile

[pnorman]

I guess the other question is, what's the story with mod_tile? Is it mapnik 3 ready?

It compiles and renderd starts. I'm not sure on the packaging.

[pnorman]

Launchpad issue for backporting: https://bugs.launchpad.net/trusty-backports/+bug/1544413

Mapnik 3 conflicts with monav, which has been dropped from more recent debian and ubuntu versions, so I'm not sure how they'll feel about the backport.

[gravitystorm]

Do we have any guesses as to whether the backport is likely to happen before Xenial is released?

If not, is the plan to wait until the end of April and upgrade the tileservers, or are we going to get mapnik3 (and mod_tile etc) before then?

[pnorman]

If not, is the plan to wait until the end of April and upgrade the tileservers, or are we going to get mapnik3 (and mod_tile etc) before then?

It'd be pretty easy to get mapnik 3 on any ppa before then - backportpackage -u ppa:<lp username>/<ppa name> -s xenial -d trusty mapnik works.

mod_tile/renderd is a different matter, and I've never packaged that

[pnorman]

Given that it's April, I'd suggest doing this as part of an Xenial upgrade.

A few thoughts

• I'll start upgrading my dev box to 16.04 LTS in the next month
• mod_tile/renderd is the only software I expect problems. PostgreSQL, osm2pgsql, and Mapnik are all well tested with their Debian packaging, and that's where the Xenial packages are from
• Do we have the capacity to bring down one of the render servers for the time needed to reinstall, reimport, and iron out bugs?
  ** Should we be looking at a new rendering server anyways? This would give us more capacity during style updates, and allow us to important maintenance like clustering more freely.

[tomhughes]

We would normally wait for 16.04.1 before starting to do upgrades (not sure when that is due) but we can probably wave that in this case.

I will likely do some other not particularly critical machine first, to sort out any general issues with our chef recipes, though I think I have already done a lot of that for the 15.10 machine we have

Ideally I would say that we start by upgrading the machines but using existing data and style sheet, and then develop a plan for reloading the database - will that work? ie does the currently style sheet work with mapnik 3?

The other question is how long a database load will take - it's been a while since I've done one - and what sort of "debugging time" you think we might need... Very little I would hope if it's all basically tested beforehand.

[lonvia]

How about repurposing poldi to a tile server for a while (or even permanently)? It's a bit short on disk space for tile caching but otherwise has the same specs as yevaud. And it's sitting there idle at the moment. Having pummelzacken as nominatim backup machine is imho sufficient.

[gravitystorm]

Do we have the capacity to bring down one of the render servers for the time needed to reinstall, reimport, and iron out bugs?

Woah, lets not get to hasty here. We can talk about database reloads later on. If we make this a bigger project than just getting mapnik3 working we're making it less likely to happen. :-)

ie does the currently style sheet work with mapnik 3?

Yes, it works as-is and provides a huge improvement in text rendering in South-East Asia, so I'm keen to see it happen as soon as possible.

[pnorman]

Woah, lets not get to hasty here. We can talk about database reloads later on. If we make this a bigger project than just getting mapnik3 working we're making it less likely to happen. :-)

It depends if we do an in-situ upgrade or not. If we don't, it's easiest to reload at the same time. I'm not proposing a schema change or anything on the style side, just a reload with existing settings.

The other question is how long a database load will take - it's been a while since I've done one - and what sort of "debugging time" you think we might need... Very little I would hope if it's all basically tested beforehand

The style is developed with Mapnik 3, so I'm not particularly worried about bugs there. I see likely sources of bugs as

• Full planet + Mapnik 3 interactions
• Chef recipes and 16.04
• Something that breaks under heavy load under 16.04

Those three things are hard to test in advance

[pnorman]

I can confirm there were no issues with renderd+mod_tile and Mapnik 3 on Debian strech

[matthijsmelissen]

I was wondering about the current status of this issue?

[tomhughes]

I was talking to @Firefishy about this last week and my suggestion was that we repurpose poldi as a third tile server, getting that setup with Ubuntu 16.04 and mapnik 3 and then we upgrade the other two in turn.

First step is openstreetmap/operations#79 to get poldi sorted out, which @Firefishy is working on.

[matkoniecz]

What version of mapnik is currently used? Is it defined at

chef/cookbooks/tile/templates/default/renderd.conf.erb

Line 10 in 92837c0

plugins_dir=/usr/lib/mapnik/2.2/input

?

[tomhughes]

Whatever's in the repo I believe.

[tomhughes]

2.2.0

[matthijsmelissen]

Is there any news already? Anything we can do to help perhaps?

[tomhughes]

I believe @Firefishy is still working on getting poldi sorted, which is the first step.

[zerebubuth]

I've got a PPA with mapnik3 & mod_tile packaged up. At the moment it's under ~zerebubuth, but I'd be happy to push it to ~osmadmins if that makes it easier.

[pnorman]

vial is running with 16.04 and Mapnik 3, so I'm keeping my eye out for potentially related problem reports.

[tomhughes]

Assuming nothing horrible happens this week I'll probably look at upgrading one of the other machines next weekend and then the other one a week or two after that...

[tomhughes]

All three tile servers are now running mapnik 3.0.9 with postgres 9.5 and postgis 2.2.

[matthijsmelissen]

Great news, thanks a lot!

[sommerluk]

Thanks!

[kocio-pl]

And what Carto version is installed now? If it's 0.16+, we would be able to drop the MML from osm-carto.

[tomhughes]

I updated carto to the latest release when doing the upgrades so it is 0.16.3 currently. That is installed from npm anyway so is easily upgraded when necessary.

[kocio-pl]

Thanks, Tom!