-
Notifications
You must be signed in to change notification settings - Fork 912
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add docker support #2272
Add docker support #2272
Changes from all commits
ad237cd
3fe73f0
4eeb36a
caaa350
b6d373f
30ad01e
447c55e
0cf0d0c
18ef6b6
f75eea4
0d15f1c
c4c450f
076073b
fc139b0
29f6833
ce38a10
b03bfd1
c6edb29
8383453
cb96ddd
a1413af
02851d2
6ce9e77
004227e
96383be
35ad5bd
4900551
7dd9810
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
* | ||
!Gemfile | ||
!Gemfile.lock | ||
!db/functions | ||
!db/docker_postgres.sh |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,99 @@ | ||
# Using Docker to run OpenStreetMap | ||
|
||
Using [Docker](https://www.docker.com/) will allow you to install the OpenStreetMap application and all its' dependencies in a container, almost with a single command. | ||
|
||
These instructions gloss over the precise details of the dependencies and their configuration but these can be found in full detail at [INSTALL.md](INSTALL.md). | ||
|
||
The first step is to fork/clone the repo to your local machine. The repository is reasonably large (~150MB) and it's unlikely that you need the full history. If you are happy to wait for it all to download, run: | ||
``` | ||
git clone https://github.com/openstreetmap/openstreetmap-website.git | ||
``` | ||
|
||
To clone only the most recent version (~23MB), instead use a 'shallow clone': | ||
|
||
``` | ||
git clone --depth=1 https://github.com/openstreetmap/openstreetmap-website.git | ||
``` | ||
|
||
Now change working directory to the `openstreetmap-website`: | ||
|
||
``` | ||
cd openstreetmap-website | ||
``` | ||
|
||
### Storage setup | ||
|
||
``` | ||
cp config/example.storage.yml config/storage.yml | ||
``` | ||
|
||
### Database | ||
|
||
``` | ||
cp config/example.database.yml config/database.yml | ||
``` | ||
|
||
Set `username` to postgres and `host` to db leave the `password` blank | ||
|
||
|
||
### App configuration | ||
|
||
``` | ||
cp config/settings.yml config/settings.local.yml | ||
``` | ||
|
||
### Installation | ||
|
||
In the root directory run: | ||
|
||
``` | ||
docker-compose -f docker/docker-compose.yml up | ||
``` | ||
Now if this is your first time running or you have removed cache this will take some time to complete. So grab tea/coffee and seat tight. Upon successfull build it should show | ||
|
||
### Migrations | ||
While `docker-compose up` is running, open another terminal windows and run: | ||
|
||
``` | ||
docker-compose -f docker/docker-compose.yml exec web bundle exec rake db:migrate | ||
``` | ||
|
||
### Node.js modules | ||
We use Yarn to manage the Node.js modules required for the project.: | ||
|
||
``` | ||
docker-compose -f docker/docker-compose.yml exec web bundle exec rake yarn:install | ||
``` | ||
|
||
Once these are complete you should be able to visit the app at http://localhost:3000 | ||
|
||
If localhost does not work, you can use the IP address of the docker-machine. | ||
|
||
### Tests | ||
|
||
``` | ||
docker-compose -f docker/docker-compose.yml exec web bundle exec rake test:db | ||
``` | ||
|
||
### Bash | ||
|
||
If you want to get onto the web container and run specific commands you can fire up bash via: | ||
|
||
``` | ||
docker-compose -f docker/docker-compose.yml exec web /bin/bash | ||
``` | ||
|
||
Similarly, if you want to get onto the db container use: | ||
|
||
``` | ||
docker-compose -f docker/docker-compose.yml exec db /bin/bash | ||
``` | ||
|
||
### Populating the database | ||
This installation comes with no geographic data loaded. You can either create new data using one of the editors (Potlatch 2, iD, JOSM etc) or by loading an OSM extract. | ||
|
||
After installing but before creating any users or data, import an extract with [Osmosis](https://wiki.openstreetmap.org/wiki/Osmosis) and the `--write-apidb` task. The `web` container comes with `osmosis` pre-installed. So to populate data with a `.osm.pbf` use the following command: | ||
|
||
``` | ||
docker-compose -f docker/docker-compose.yml exec web osmosis --read-pbf /path/to/file.osm.pbf --write-apidb host="db" database="openstreetmap" user="postgres" validateSchemaVersion="no" | ||
``` |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
FROM ruby:2.5-slim | ||
|
||
# install packages | ||
# fixes dpkg man page softlink error while installing postgresql-client [source: https://stackoverflow.com/a/52655008/5350059] | ||
RUN mkdir -p /usr/share/man/man1 && mkdir -p /usr/share/man/man7 | ||
RUN apt-get update && apt-get install curl -y | ||
RUN apt-get clean && rm -rf /var/lib/apt/lists/* | ||
|
||
|
||
#npm is not available in Debian repo so following official instruction [source: https://github.com/nodesource/distributions/blob/master/README.md#debinstall] | ||
RUN curl -sL https://deb.nodesource.com/setup_10.x -o nodesource_setup.sh && bash nodesource_setup.sh && rm nodesource_setup.sh | ||
RUN apt-get install -y --no-install-recommends ruby-dev libarchive-dev libmagickwand-dev libxml2-dev libxslt1-dev build-essential libpq-dev libsasl2-dev imagemagick libffi-dev locales postgresql-client nodejs osmosis | ||
RUN apt-get clean && rm -rf /var/lib/apt/lists/* | ||
RUN npm install yarn phantomjs-prebuilt -g --unsafe-perm | ||
|
||
|
||
# Setup app location | ||
RUN mkdir -p /app | ||
WORKDIR /app | ||
|
||
# Install gems | ||
ADD Gemfile* /app/ | ||
RUN bundle install | ||
# Setup local | ||
RUN sed -i -e 's/# en_GB.UTF-8 UTF-8/en_GB.UTF-8 UTF-8/' /etc/locale.gen && \ | ||
echo 'LANG="en_GB.UTF-8"'>/etc/default/locale && \ | ||
dpkg-reconfigure --frontend=noninteractive locales && \ | ||
update-locale LANG=en_GB.UTF-8 | ||
|
||
ENV LANG en_GB.UTF-8 |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
FROM postgres:11 | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Production still uses 9.5 as of now. |
||
ADD docker/docker_postgres.sh docker-entrypoint-initdb.d/docker_postgres.sh | ||
|
||
# compiling ruby from source as official debian has ruby 2.3 and we need => 2.4 | ||
RUN apt-get update | ||
RUN apt-get install -y curl build-essential libreadline-dev libssl-dev libcurl4-openssl-dev postgresql-server-dev-all zlib1g-dev libxml2-dev | ||
RUN curl -L https://cache.ruby-lang.org/pub/ruby/2.6/ruby-2.6.4.tar.gz | tar zx | ||
RUN cd ruby-2.6.4 && ./configure && make && make install | ||
RUN cd .. && rm -r ruby-2.6.4 | ||
RUN apt-get clean && rm -rf /var/lib/apt/lists/* | ||
|
||
|
||
# Setup app location | ||
RUN mkdir -p /app | ||
WORKDIR /app | ||
|
||
# add database functions directory | ||
ADD db/functions/ /app/db/functions/ | ||
|
||
# Install gems | ||
ADD Gemfile* /app/ | ||
RUN bundle install | ||
|
||
# change ownership to postgres as while running docker_postgres.sh postgres will need write access | ||
RUN chown -R postgres /app/db |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
version: '2' | ||
services: | ||
web: | ||
image: openstreetmap-website:v1 | ||
build: | ||
context: .. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why not put the Dockerfiles in this directory? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Which directory? Right now all docker related files are in one directory named There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oh wait, nevermind. |
||
dockerfile: docker/Dockerfile | ||
volumes: | ||
- ..:/app | ||
ports: | ||
- "3000:3000" | ||
command: bundle exec rails s -p 3000 -b '0.0.0.0' | ||
depends_on: | ||
- db | ||
environment: | ||
DATABASE_URL: postgres://postgres@db:5432 | ||
db: | ||
image: openstreetmap-db:v1 | ||
build: | ||
context: .. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same here as above? |
||
dockerfile: docker/Dockerfile.postgres | ||
ports: | ||
- "5432:5432" | ||
environment: | ||
POSTGRES_DB: openstreetmap |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
#!/bin/bash | ||
set -e | ||
psql -v ON_ERROR_STOP=1 -U "$POSTGRES_USER" -c "CREATE EXTENSION btree_gist" openstreetmap | ||
make -C db/functions libpgosm.so | ||
psql -v ON_ERROR_STOP=1 -U "$POSTGRES_USER" -c "CREATE FUNCTION maptile_for_point(int8, int8, int4) RETURNS int4 AS '/app/db/functions/libpgosm', 'maptile_for_point' LANGUAGE C STRICT" openstreetmap | ||
psql -v ON_ERROR_STOP=1 -U "$POSTGRES_USER" -c "CREATE FUNCTION tile_for_point(int4, int4) RETURNS int8 AS '/app/db/functions/libpgosm', 'tile_for_point' LANGUAGE C STRICT" openstreetmap | ||
psql -v ON_ERROR_STOP=1 -U "$POSTGRES_USER" -c "CREATE FUNCTION xid_to_int4(xid) RETURNS int4 AS '/app/db/functions/libpgosm', 'xid_to_int4' LANGUAGE C STRICT" openstreetmap |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Each time apt-get is run you should update and then remove the APT cache files at the end of the command.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So that the cache files don't make it into the docker image
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Isn't this just building a container for local use though? Not anything that's going to be published?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, but what I've read indicates it as a best practice in general