ℹ️ GitHub issues are disabled on this repository on purpose. Exasol Docker DB is a project mainly intended for automatic testing of Exasol deployments.
Exasol is a high-performance, in-memory, MPP database specifically designed for analytics. This repository contains a dockerized version of the Exasol DB for testing purposes.
The dockerized version of Exasol in this repository can be used with up to 10GiB of data. If you need more please get in contact with us via https://exasol.com/get-in-touch.
Your use of this repository is subject to the Exasol Terms & Conditions
Please note that this is an open source project which is not officially supported by Exasol. We will try to help you as much as possible, but can't guarantee anything since this is not an official Exasol product.
Currently supported features:
- create / start / stop a database in a virtual cluster
- use the UDF framework
- expose ports from containers on the local host
- update the virtual cluster
- create backups on archive volumes
Creating a stand-alone Exasol container (docker run)
Creating a multi-host Exasol cluster (by connecting multiple containers)
Installing custom JDBC drivers
The Exasol Docker image has been developed and tested with Docker 18.03.1-ce (API 1.37) on Fedora 27. It may also work with earlier versions, but that is not guaranteed.
Please see the Docker installation documentation for details on how to upgrade your Docker daemon.
Docker privileged mode is required for permissions management, UDF support and environment configuration and validation (sysctl, hugepages, block-devices, etc.).
Exasol currently only supports 64-bit x86-64 platforms with SSSE3 featured CPUs (Intel64 or AMD64) and is for this architecture optimized. See also the System Requirements.
We currently only support Docker on Linux. If you are using a Windows host you'd have to create a Linux VM.
The host OS must support O_DIRECT access for the Exasol containers (see Troubleshooting).
Each database instance needs at least 2 GiB RAM. We recommend that the host reserves at least 4 GiB RAM for each running Exasol container.
Make sure that NTP is configured correctly on the host. Also, the RNG daemon must be running in order to provide enough entropy for the Exasol services in the container.
We strongly recommend to set the CPU governor on the host to performance, in order to avoid serious performance problems. There are various tools to do that, depending on your distribution. Usually, the following command works:
for F in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor; do echo performance >$F; doneWe recommend to enable hugepages for hosts with at least 64GB RAM. In order to do so, you have to set the Hugepages option in EXAConf to either auto, host or the nr. of hugepages per container.
If you set it auto, the nr. of hugepages will be determined automatically, depending on the DB settings.
When setting it to host the nr. of hugepages from the host system will be used (i. e. /proc/sys/vm/nr_hugepages will not be changed). However, /proc/sys/vm/hugetlb_shm_group will always be set to an internal value!
It's possible to limit the resources of your Exasol container with the following docker run options:
docker run --cpuset-cpus="1,2,3,4" --memory=20g --memory-swap=20g --memory-reservation=10g exasol/docker-db:<version>This is especially recommended if you have multiple Exasol containers (or other services) on the same host. In that case, you should evenly distribute the available CPUs and memory throughout your Exasol containers.
See https://docs.docker.com/config/containers/resource_constraints/ for more options.
You can create an Exasol container from the Exasol docker image using the following command:
$ docker run --name exasoldb -p 127.0.0.1:9563:8563 --detach --privileged --stop-timeout 120 exasol/docker-db:<version>In this example port 8563 (within the container) is exposed on the local port 9563. Use this port to connect to the DB.
All data, configuration and logfiles of an Exasol container are stored below /exa. With the command above, this data is lost when the container is removed. However, you can make it persistent by mounting a volume into the container at /exa, for example:
$ docker run --name exasoldb -p 127.0.0.1:9563:8563 --detach --privileged --stop-timeout 120 -v exa_volume:/exa exasol/docker-db:<version>See the Docker volumes documentation for more examples on how to create and manage persistent volumes.
It is important to make sure that the database has been shut down correctly before stopping a container. A high stop-timeout (see example above) increases the chance that the DB can be shut down gracefully before the container is stopped, but it's not guaranteed. It's better to stop the DB manually by executing the following command within the container (after attaching to it):
$ dwad_client stop-wait DB1Or from outside the container:
$ docker exec -ti exasoldb dwad_client stop-wait DB1An existing persistent volume can be updated (for use with a later version of an Exasol image) by calling the following command with the new image:
$ docker run --rm -v exa_volume:/exa exasol/docker-db:<new version> update-scIf everything works correctly, you should see output similar to this:
Updating EXAConf '/exa/etc/EXAConf' from version '7.0.2' to '7.0.3'
Container has been successfully updated!
- Image ver. : 7.0.2 --> 7.0.3
- DB ver. : 7.0.2 --> 7.0.3
- OS ver. : 7.0.2 --> 7.0.3After that, a new container can be created (from the new image) using the old / updated volume.
It is possible to create multiple containers on different hosts and connect them to a cluster (one container per host).
First you have to create the configuration for the cluster. There are two possible ways to do so:
Execute the following command (--num-nodes is the number of containers in the cluster):
$ export CONTAINER_EXA="$HOME/container_exa/"
$ docker run -v "$CONTAINER_EXA":/exa --rm -i exasol/docker-db:<version> init-sc --template --num-nodes 3After the command has finished, the directory $CONTAINER_EXA contains all subdirectories as well as an EXAConf template (in /etc).
NOTE: you need to add --privileged if the host directory belongs to root.
You can create a template file and redirect it to wherever you want by executing:
$ docker run --rm -i exasol/docker-db:<version> -q init-sc --template --num-nodes 3 -p > ~/MyExaConfNOTE: we recommend creating an /exa/ template directory and the following steps assume that you did so. If you choose to only create the EXAConf file, you have to build a new Docker image with it and create the EXAStorage devices files within that image.
The configuration has to be completed before the cluster can be started.
[Node : 11]
PrivateNet = 10.10.10.11/24 # <-- replace with the real networkYou can also change the IP using the exaconf CLI tool from the Exasol image:
docker run --rm -v $CONTAINER_EXA:/exa exasol/docker-db:<version> exaconf modify-node -n 11 -p 10.10.10.11/24[[Disk : disk1]]
Devices = dev.1 #'dev.1' must be located in '/exa/data/storage'For more information on device management, see Managing disks and devices.
NOTE: You can leave this entry as it is if you create the devices as described below.
[EXAVolume : DataVolume1]
Type = data
Nodes = 11, 12, 13
Disk = disk1
# Volume size (e. g. '1 TiB')
Size = # <-- enter volume size hereYou can also change the volume size using the exaconf CLI tool from the Exasol image:
docker run --rm -v $CONTAINER_EXA:/exa exasol/docker-db:<version> exaconf modify-volume -n DataVolume1 -s 1TiBIf you are using the host network mode (see "Start the cluster" below), you may have to adjust the port numbers used by the Exasol services. The one that's most likely to collide is the SSH daemon, which is using the well-known port 22. You can change it in EXAConf:
[Global]
SSHPort = 22 # <-- replace with any unused port numberThe other Exasol services (e. g. Cored, BucketFS and the DB itself) are using port numbers above 1024. However, you can change them all by editing EXAConf.
[Global]
...
# Comma-separated list of nameservers for this cluster.
NameServers =Copy $CONTAINER_EXA to all cluster nodes (the exact path is not relevant, but should be identical on all nodes).
You can create device files by executing (on each node):
$ truncate -s 1G $CONTAINER_EXA/data/storage/dev.1This will create a sparse file of 1GB (1000 blocks of 1 MB) that holds the data. Adjust the size of the data file to your needs. Repeat this step to create multiple file devices.
IMPORTANT: Each device should be slightly bigger (~1%) than the required space for the volume(s), because a part of it will be reserved for metadata and checksums.
For more information on device management, see Managing disks and devices.
The cluster is started by creating all containers individually and passing each of them its ID from the EXAConf. For n11 the command would be:
$ docker run --detach --network=host --privileged -v $CONTAINER_EXA:/exa exasol/docker-db:<version> init-sc --node-id 11NOTE: this example uses the host network stack, i. e. the containers are directly accessing a host interface to connect to each other. There is no need to expose ports in this mode: they are all accessible on the host.
All EXAStorage devices have to be located below /exa/data/storage/ (within the container).
You can create file devices by executing (on each node):
$ truncate -s 1G $CONTAINER_EXA/data/storage/dev.2or (alternatively):
$ dd if=/dev/zero of=$CONTAINER_EXA/data/storage/dev.2 bs=1M count=1 seek=999This will create a sparse file of 1GB (1000 blocks of 1 MB) that holds the data. Adjust the size to your needs.
NOTE: you can also create the files in any place you like and mount them to /exa/data/storage/ with docker run -v /path/to/dev.x:/exa/data/storage/dev.x.
IMPORTANT: Each device should be slightly bigger (~1%) than the required space for the volume(s), because a part of it will be reserved for metadata and checksums.
After the devices have been created, they need to be added to EXAConf. You can either do this manually by editing EXAConf:
[Node : 11]
[[Disk : disk1]]
Devices = dev.1, dev.2or use the exaconf CLI tool from the Exasol image:
docker run --rm -v $CONTAINER_EXA:/exa exasol/docker-db:<version> exaconf add-node-device -D disk1 -d dev.2 -n 11You can add additional disks to each node in EXAConf (a disk is a group of devices that is assigned to a volume). This may be useful if you want to use different devices for different volumes.
Similar to adding devices, you can either manually edit EXAConf:
[Node : 11]
[[Disk : disk1]]
Devices = dev.1, dev.2
[[Disk : disk2]]
Devices = dev.3, dev.4or use the exaconf CLI tool from the Exasol image:
docker run --rm -v $CONTAINER_EXA:/exa exasol/docker-dev-7.0.0:juk exaconf add-node-disk -D disk2 -n 11
docker run --rm -v $CONTAINER_EXA:/exa exasol/docker-db:<version> exaconf add-node-device -D disk2 -d dev.3 -n 11
docker run --rm -v $CONTAINER_EXA:/exa exasol/docker-db:<version> exaconf add-node-device -D disk2 -d dev.4 -n 11NOTE: Don't forget to copy the modified EXAConf to all other nodes.
If you need to enlarge a file device of an existing Exasol container, you can use the following commands to do so:
$ docker exec -ti /bin/bash
$ truncate --size=+10GB /exa/data/storage/dev.1
$ cshdd --enlarge --node-id 11 -h /exa/data/storage/dev.1[.data]
Custom JDBC drivers can be added by uploading them into a bucket. The bucket and path for the drivers can be configured in each database section of EXAConf. The default configuration is:
[DB : DB1]
...
# OPTIONAL: JDBC driver configuration
[[JDBC]]
BucketFS = bfsdefault
Bucket = default
# Directory within the bucket that contains the drivers
Dir = drivers/jdbcIn order for the database to find the driver, you need to upload it into a subdirectory of drivers/jdbc of the default bucket (which is automatically created if you don't modify EXAConf). See the section Installing Oracle drivers for help on how to upload files to BucketFS.
In addition to the driver file(s), you also have to create and upload a file called settings.cfg , that looks like this:
DRIVERNAME=MY_JDBC_DRIVER
JAR=my_jdbc_driver.jar
DRIVERMAIN=com.mydriver.jdbc.Driver
PREFIX=jdbc:mydriver:
FETCHSIZE=100000
INSERTSIZE=-1Change the variables DRIVERNAME, JAR, DRIVERMAIN and PREFIX according to your driver and upload the file (into the same directory as the driver itself). Please make sure that every line, including the final line ends with a newline (LF) character.
IMPORTANT: Do not modify the last two lines!
If you use the default bucket and the default path, you can add multiple JDBC drivers during runtime. The DB will find them without having to restart it (as long as they're located in a subfolder of the default path). Otherwise, a container restart is required.
Oracle drivers can be added by uploading them into a bucket. The bucket and path for the drivers can be configured in each database section of EXAConf. The default configuration is:
[DB : DB1]
...
# OPTIONAL: Oracle driver configuration
[[ORACLE]]
BucketFS = bfsdefault
Bucket = default
# Directory within the bucket that contains the drivers
Dir = drivers/oracleIn order for the database to find the driver, you have to upload it to drivers/oracle of the default bucket (which is automatically created if you don't modify EXAConf).
You can use curl for uploading, e. g.:
$ curl -v -X PUT -T instantclient-basic-linux.x64-12.1.0.2.0.zip http://w:PASSWORD@10.10.10.11:2580/default/drivers/oracle/instantclient-basic-linux.x64-12.1.0.2.0.zip
Replace PASSWORD with the WritePasswd for the bucket. See Connecting to BucketFS for details.
NOTE: The only currently supported driver version is 12.1.0.2.0. Please download the package instantclient-basic-linux.x64-12.1.0.2.0.zip from oracle.com and upload it as described above.
Connecting to the default Exasol DB inside a Docker container is not different from the "normal" version. You can use any supported client and authenticate with username sys and password exasol.
Please refer to the offical manual for further information.
The default port of the BucketFS inside of the docker container is 2580. You must however expose this port in the docker command so that you can access it from outside of the container. For example by adding -p 2580:2580 to your docker run command.
Read and write passwords for the BucketFS are autogenerated. You can find them in the EXAConf. They are base64-encoded and can be decoded like this:
$ awk '/WritePasswd/{ print $3; }' EXAConf | base64 -d
Please refer to the BucketFS manual for information on how to access files in the BucketFS.
ERROR::EXAConf: Integrity check failed! The stored checksum 'a2f605126a2ca6052b5477619975664f' does not match the actual checksum 'f9b9df0b9247b4696135c135ea066580'. Set checksum to 'COMMIT' if you made intentional changes.
If you see a message similar to the one above, you probably modified an EXAConf that has already been used by an Exasol container or exadt. It is issued by the EXAConf integrity check that protects EXAConf from accidental changes and detects file corruption.
In order to solve the problem you have to set the checksum within EXAConf to 'COMMIT'. It can be found in the 'Global' section, near the top of the file:
[Global]
...
Checksum = COMMIT
...WORKER::ERROR: Failed to open device '/exa/data/storage/dev.1.data'! WORKER:: errno = Invalid argument
If the container does not start up properly and you see an error like this in the logfiles below /exa/logs/cored/, your filesystem probably does not support O_DIRECT I/O mode.
We strongly recommend to always use O_DIRECT, but if you really can't, then you can disable O_DIRECT mode by adding a line to each disk in EXAConf:
[Node : 11]
...
[[Disk : disk1]]
DirectIO = FalseIMPORTANT: Disabling O_DIRECT mode may cause significantly higher memory usage and fluctuating I/O throughput!
Could not start database: system does not have enough active nodes or DWAd was not able to create startup parameters for system
If all containers started successfully but the database did not and you see a message similar to this in the output of docker logs, you may not have enough memory in your host(s). The DB needs at least 2 GiB RAM per node (that's also the default value in EXAConf).
Please read the Contribution guidelines for this project before submitting a bug report or pull request!