Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallelize interpolation build #264

Open
orangejulius opened this issue Sep 15, 2020 · 1 comment
Open

Parallelize interpolation build #264

orangejulius opened this issue Sep 15, 2020 · 1 comment

Comments

@orangejulius
Copy link
Member

This issue is to record the fact that the process of building the interpolation database is single threaded and can be quite slow.

For a full planet build you can expect it to take 6+ days. We recommend avoiding setting up the interpolation database when initially experimenting with Pelias to save yourself some time.

We welcome efforts to parallelize it, but beware: it's not as easy as it sounds. Feel free to shoot an email to team@pelias.io if you are interested in helping out.

Over time we may add more details here about the architecture of the interpolation service and why it's single threaded :)

@joacub
Copy link

joacub commented Nov 7, 2020

Why it is single thread, i can see that could be mutlithread processed bounch of files at the same time will reduce a lot the times

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants