Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updating lake centerlines #354

Open
ache051 opened this issue Aug 1, 2021 · 18 comments · May be fixed by #426
Open

Updating lake centerlines #354

ache051 opened this issue Aug 1, 2021 · 18 comments · May be fixed by #426

Comments

@ache051
Copy link
Contributor

ache051 commented Aug 1, 2021

Hi,
The current OMT vectortiles generation routine is still pulling data from Lukas' osm-lakeline released in 2016. Granted lake centerlines don't change much but would still good to update periodically. At the moment I forked Lukas' repository and changed a few lines so centerlines are generated when I run quickstart.sh. Not pretty but does the job.

My question is if there is any plan on either updating the data or incorporating functionality from osm-lakeline into openmaptiles-tools or vectortiles generation routine?

@TomPohys
Copy link
Member

TomPohys commented Aug 2, 2021

Thanks. Yes, there is a plan to figure it out. Do you have some measured time, how long it takes to regenerated osm-lakeline on your use case?

@ache051
Copy link
Contributor Author

ache051 commented Aug 3, 2021

I have started doing a run from a planet.osm.pbf-loaded database on AWS (m5.24xlarge, 96 cores) and also on local machine (AMD Ryzen9 3950x, 12 cores) to get some numbers. Will post them when they are done.

@ache051
Copy link
Contributor Author

ache051 commented Aug 3, 2021

The AWS instance finished generating the centerlines (36370 of them in total) in 132 minutes. Will check the local machine tomorrow.

@ache051
Copy link
Contributor Author

ache051 commented Aug 3, 2021

The local machine finished in 155 minutes with 37544 centerlines (planet.osm.pbf is 2 months newer than AWS instance's).
For reference the AWS instance needed 8 days to run quickstart.sh down to zoom level 14, and the local machine needed 31 days.

@nyurik
Copy link
Member

nyurik commented Aug 3, 2021

@ache051 ideally we should NOT use the lakeline centerlines like we do now. My long term hope was to get rid of it entirely, and instead use the usual PostGIS functions to compute it, possibly generating a materialized view if needed for performance reasons. The current implementation adds significant complexity to the project's ecosystem.

@ache051
Copy link
Contributor Author

ache051 commented Aug 3, 2021

@nyurik I agree with you 100%. Exporting to shapefile then generate lines is cumbersome. At the time being though it is a stop gap solution for my project. If anybody thinks there is merit, then I will look at if ST_ApproximateMedialAxis can do the trick.
I have posted the centerlines generated from last night's run for your amusement.

@nyurik
Copy link
Member

nyurik commented Aug 3, 2021

@ache051 this looks like a perfect path as far as I'm concerned! Will gladly review/discuss/merge any PR work in that direction, and I'm sure @TomPohys will agree with me on this :)

@ache051
Copy link
Contributor Author

ache051 commented Aug 4, 2021

Great! I will dive in a bit more seriously, but here are the results from my crude attempt with PostGIS internal functions (green is from last night's run, red is from PostGIS). A lot of work to be done with voronois...
Screenshot from 2021-08-04 16 50 04
Screenshot from 2021-08-04 16 50 42
Screenshot from 2021-08-04 16 51 11

@nyurik
Copy link
Member

nyurik commented Aug 4, 2021

LOL, the last one is just priceless... I keep thinking what kind of an animal it reminds me of... some dinosaur I suspect :D But the last image does highlight the broken nature of the PostGIS algo it seem? The docs say that ST_ApproximateMedialAxis will use ST_StraightSkeleton if SFCGAL is not available - was that the case in your examples?

@ache051
Copy link
Contributor Author

ache051 commented Aug 4, 2021

Hah! It looks like my 2 years old's drawing!
You are right about the SFCGAL part. I didn't want to modify the postgis installation so adapted some of the codes from this stackexchange post. I am going to look at the code closer tomorrow.

@TomPohys
Copy link
Member

TomPohys commented Aug 4, 2021

Is the SFCGAL extension problem? There are some licence blockers? Some comments in openmaptiles/openmaptiles#1150 (comment)

Anyway, thanks for looking at it!

@ache051
Copy link
Contributor Author

ache051 commented Aug 5, 2021

Not much that it is a problem. It's just that I already have a planet-loaded database and didn't want to fiddle around with it. But if there are other components that need SFCGAL, then I am happy to build a docker image and try it out.

@ZeLonewolf
Copy link
Contributor

@TomPohys I noted in openmaptiles/openmaptiles#1150 (comment) with link to the original source documentation, that there is no issue with using the SFCGAL extension from a license perspective. Also, it is the same licensing as PostGIS itself, so if PostGIS is OK, then so is SFCGAL :)

@ZeLonewolf
Copy link
Contributor

@ache051, it would be very useful if we had the SFCGAL extensions available. The lack of this extension is a blocker for #1150 and #380.

@TomPohys
Copy link
Member

No problem to add SFCGAL into openmaptiles-tools/postgis docker. PR is welcome.

@ache051
Copy link
Contributor Author

ache051 commented Aug 16, 2021

I have been chipping it away slowly. ST_StraightSkeleton ST_ApproximateMedialAxis (edited) does a better job than my initial effort. But in order to replicate what Lukas and Ungarj did (routing between the furthest nodes in a network built by Voronois with maximum angle of turns defined), we will also need pgrouting enable as well. I have built a modified openmaptiles-tools/postgis with pgrouting enabled and will try to implement Ungarj's inside Postgis. To be honest I have no idea whether it is the best method to construct centerlines, I just replicate 😛 Comments welcome.

@nyurik
Copy link
Member

nyurik commented Aug 16, 2021

@klokan might know the most about the actual math needed for this

@nyurik
Copy link
Member

nyurik commented Aug 16, 2021

My opinion on the overall approach -- it is better to have a "good enough" solution for lake centerlines as part of the OMT SQL than to have a "perfect" solution that uses a complex external processing outside of the project. The SQL can be incrementally improved in subsequent pull requests, whereas the external project is essentially frozen, as it requires far more work to make any changes.

@quincylvania quincylvania linked a pull request Feb 28, 2023 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants