Allow creating multiple databases on startup.

[tonio-ramirez]

This can be useful in some cases. For example, if you're auditing entities, and want to store the audit information on a separate database.

[yosifkit]

I like it (as long as we go document how to do it with the proper syntax in docker run).

[yosifkit]

Ping @tianon

[tianon]

Seems reasonable, since database names can't have spaces in them.

[thaJeztah]

since database names can't have spaces in them.

Well, in fact, they can: docs. TL;DR MySQL supports any Unicode character when quoted using back ticks. Not saying it's good practice, but it is supported.

[tianon]

I'm not sure I'm comfortable with our script supporting such an obscene
practice! :)

(but really only because I don't have a better idea for making this work
cleanly with just environment variables :P)

[thaJeztah]

@tianon yup agree, maybe document that it isn't supported is sufficient? Eg "known issues" or just "not supported".

Just to prevent support questions 😄

[md5]

Perhaps an approach like the /docker-entrypoint-initdb.d/ thing we did in the Postgres image would be appropriate: docker-library/postgres#23

I was just looking at this entrypoint and comparing it to https://github.com/cpuguy83/docker-mysql while thinking about how to make a simple, replication capable MySQL container for testing purposes.

My first thought was that it would be nice to be able to add to TEMP_FILE; seeing this PR, it seems that the entrypoint.d approach would allow additional DBs to be created just as easily.

[md5]

Actually, it looks like mysqld has a --bootstrap option that might be a better approach than the current --init-file approach.

I got hip to --bootstrap in the GoogleCloudPlatform/compute-docker-mysql repo, which shows how to use it: https://github.com/GoogleCloudPlatform/compute-docker-mysql/blob/master/run-mysqld#L22-L35

[thaJeztah]

Nice find on the --bootstrap, didn't know that one

[md5]

Unfortunately, after playing with it last night, I found that --bootstrap implies --skip-grant-tables, which makes the GRANT ALL statements break.

It's possible to work around this by manipulating the mysql.user table directly in the case of the GRANT ALL *.* statement and by manipulating mysql.db in the case of the GRANT ALL ${MYSQL_DATABASE}.* statement, but that sounds like a recipe for future breakage. I also discovered that GRANT ALL statements work correctly with --bootstrap/--skip-grant-tables in MariaDB, but that doesn't help with MySQL.

[thaJeztah]

Maybe we should just deprecate MySQL and go for MariaDB 😄

[md5]

As long as the image is still called "mysql".

[yosifkit]

Oh, that sounds fun, though I think the MySQL people might get a bit grumpy if we just drop it. We could do mysql:maria or mysql:mariadb and then change latest to be MariaDB, but that may create inconsistency. (We should probably open a separate issue about adding/migrating to Maria, since that is not the purpose of this PR)

[md5]

I thought we were joking!

I see no problem having a maria image though.

[thaJeztah]

Yeah, thought we were joking as well, but adding an official MariaDB image to the repository might be a good thing (haven't checked if an official one exists to be honest)

[md5]

Looks like @tianon has an unofficial image here: https://github.com/tianon/dockerfiles/tree/master/mariadb

[thaJeztah]

There are others as well probably, but no (semi) official?

[yosifkit]

Nothing in the Official Images (previously know as "stackbrew").

[tianon]

Totally off-topic, but I agree that MariaDB would need to be separate.
It's a fork of MySQL, and MySQL is definitely still a thing, so it would be
dishonest IMO to call "MariaDB" pure "MySQL".

[thaJeztah]

@tianon reading back my comment; I meant to say;
"Adding MariaDB to the registry", didn't mean this repository;
they're clearly a different thing, different maintainers so shouldn't
be mixed up

[yosifkit]

Back on topic, I like the "create multiple databases", though we should now add it to 5.6 as well, and then add docs to tell users how to use it (and not allow spaces in db names).

@tianon?

[yosifkit]

Off topic: we need MariaDB. docker-library/official-images#294

[tianon]

Yeah, SGTM

[thaJeztah]

on topic; I like the "create multiple databases" as well. Is this all working correctly (looking back at the --bootstrap sidesteps?)

[yosifkit]

I think it should work fine in --init-file, as long and the format with newlines is correct. This way we still get to use GRANT rather than mucking with the mysql.user table directly.

[tfwright]

Hi all, I'm new to docker and trying out this image and I ran into a need to multiple databases. I've been sifting through this and other linked issues and for the life of me I can't parse what was settled on for best practice here. Do I need a new dockerfile that extends this one or can I specify multiple databases (or initdb scripts) in docker-compose.yml? Or something else entirely? Thanks for any guidance you can offer!

[yosifkit]

@tfwright, you can have any number of initdb sql scripts, so that should be the easiest way to create multiple databases.

[tfwright]

@yosifkit Thanks. To confirm, right now I'm just using the image in my compose config, so I'll need to change that to point to a custom dockerfile, right?

[yosifkit]

@tfwright, not necessarily, you could just have the sql file with the database creation in the directory with the compose file and add a volume directive to the yaml with ./create-databases.sql:/docker-entrypoint-initdb.d/. I think even ./*.sql:/docker-entrypoint-initdb.d/ should work if you have multiple scripts or even just put them in a folder: ./sql-setup/:/docker-entrypoint-initdb.d/.

[tfwright]

@yosifkit I tried adding a simple script and volume directive

#   new file:   src/sql-setup.sql

@@ -15,6 +15,7 @@ mysql:
   volumes:
      - "./src/my.cnf:/etc/mysql/conf.d/z_my.cnf"
      - /var/lib/mysql
+     - "./src/sql-setup.sql:/docker-entrypoint-initdb.d/"

but docker complains when trying to start the container

Recreating 1632cce382_spectre_mysql_1...
Cannot start container 45b56877b97225ccd74c06bd3bfe09c7c06161b099922889ba0d4ba09bb0ad77: [8] System error: not a directory

What did I miss?

[tfwright]

I was able to fix the above by changing the volume directive to "./src/sql-setup.sql:/docker-entrypoint-initdb.d/sql-setup.sql" which doesn't jive with what I understand about docker's syntax...but who cares? It works!

Thanks again!

[paulredmond]

@tfwright not sure if there is a better way, but your suggestion worked for me ❤️

[solars]

Came across this ticket, but saw that it is still open. I'm using a mysql container for rails development like this:

docker run --name cpc-mysql -p 127.0.0.1:3307:3306 -e MYSQL_RANDOM_ROOT_PASSWORD=true -e MYSQL_USER=google_report -e MYSQL_PASSWORD=asdf -e MYSQL_DATABASE=app-dev -d mysql

I would need a second database app-test for testing, what is currently the cleanest way to do this?

[thaJeztah]

@solars have you tried adding .sql or .sh files to the docker-entrypoint-initdb.d directory (or mounting a volume in that location)? #90

[solars]

Thanks for the pointer, I didn't see this option yet! In this case I will create a clean Dockerfile and add sql as needed. Thanks a lot!

[solars]

@thaJeztah one more thing, I read here (first answer): http://stackoverflow.com/questions/32482780/how-to-create-populated-mysql-docker-image-on-build-time

that in the Dockerfile (build) it doesn't work - is this still true? If so, how should I do it?

[thaJeztah]

@solars correct; it's possible, but the VOLUME declaration in the Dockerfile complicates things. (also see moby/moby#8177, moby/moby#18286, moby/moby#18287)

[solars]

@thaJeztah so I need to specify it in the run command above? or what is the suggested way to do it?

[yosifkit]

Agreeing with #86 (comment), I think this is solved. Comment if you feel there is more to discuss.

[Glideh]

So how do we do multiple databases after all ?
Separate container ? Seems to need quite a lot of config.
MYSQL_DATABASE space/coma separated ? Would be handy.

[yosifkit]

@Glideh, either a new container or add a script to /docker-entrypoint-initdb.d/. An .sql file would be the simplest, I think.

[skytreader]

I'm trying to follow the discussion here but it seems that the script in docker-entrypoint-initdb.d is not being run at all.

My Dockerfile is as follows:

FROM ubuntu:16.04
ADD . ./app
WORKDIR ./app

RUN apt-get update
RUN apt-get install -y python python-pip
RUN pip install -r requirements.txt

Mounted in docker-compose.yml via

version: '2'
services:
  db:
    image: "mysql:5.7"
    environment:
      - MYSQL_ALLOW_EMPTY_PASSWORD=yes
    ports:
      - "3308:3306"
    volumes:
      - "./docker-setup/init.sql:/docker-entrypoint-initdb.d/init.sql"
  web:
    build: .
    command: python run.py
    ports:
      - "80:7070"
    volumes:
      - .:/app
    depends_on:
      - db

With init.sql containing

CREATE DATABASE IF NOT EXISTS app DEFAULT CHARACTER SET = utf-8;
CREATE DATABASE IF NOT EXISTS app_test DEFAULT CHARACTER SET = utf-8;

However, when I run docker-compose up, I ultimately end up with the error:

web_1  | OperationalError: (_mysql_exceptions.OperationalError) (1049, "Unknown database 'app'")

To be clear, using MYSQL_DATABASE env var allows my web and db to run harmoniously. It's just that, if possible, I want to run app and app_test on the same MySQL service.

Anything I'm doing wrong?

[mrts]

Anything I'm doing wrong?

Yes, you are attempting to mount files as volumes.

Here's a working example for PosgreSQL that has similar structure:

myapp-postgresql:
    image: postgres:9.6.2
    volumes:
        - ../docker-posgresql-multi-db:/docker-entrypoint-initdb.d
    environment:
        - POSTGRES_MULTIPLE_DATABASES=db1,db2
        - POSTGRES_USER=myapp
        - POSTGRES_PASSWORD=

The docker-posgresql-multi-db directory contains a script for creating multiple databases, see https://github.com/mrts/docker-posgresql-multiple-databases for the script that you can reuse.