Support for Erlang OTP on Docker #1075

c0b · 2015-09-24T01:11:46Z

There is no official erlang images yet, I'm trying to make it happen; please let me know if any additional work need to do

+This is used as docker base image for Erlang OTP.
+
+One latest version for each of Last four major version: 18, 17, R16, R15 will be supported.

http://www.erlang.org/download.html

Erlang is a programming language used to build massively scalable soft real-time systems with requirements on high availability.

c0b · 2015-09-24T06:45:37Z

I'm aware of https://hub.docker.com/r/correl/erlang/ which provides additional popular tools like rebar and relx, I will try to add them in more variants once this base can be merged.

c0b · 2015-09-24T17:02:05Z

docs PR at docker-library/docs#348 please review

tianon · 2015-09-28T21:50:27Z

Does it make sense to consider the pre-built packages hosted by Erlang Solutions, since they are officially recommended by upstream? I admit I don't know nearly enough about the Erlang ecosystem to know whether they're kept well up-to-date. One way of phrasing this that I find helpful is whether a production deployment of an Erlang project is recommended to be compiling Erlang from source, or whether it ought to be downloading pre-built packages from somewhere.

I'm also curious as to the number of versions supported here -- are these all still actively receiving updates/fixes upstream?

Additionally (as a final point from my initial review), I'm curious whether anyone from upstream is interested in being involved here (and if you've reached out to them, what their thoughts were)? 👍

md5 · 2015-09-28T22:14:53Z

Back when I was working for Synctree and created the Dockerfiles over at synctree/docker-erlang, I had some interaction with some folks over at Erlang Solutions Limited regarding their *.deb packages. I'll see if I can dig up who I was working with and cc them here.

md5 · 2015-09-28T22:49:35Z

Looks like I was working with @dexsda

c0b · 2015-09-30T00:35:18Z

that's on my schedule, I just made erlang:18-esl variant, to pull install the binary erlang-18.1 deb package from erlang-solutions install on to debian:jessie, which results a smaller image,

https://github.com/c0b/docker-erlang-otp/blob/master/18/esl/Dockerfile

$ docker images |grep ^erlang
erlang     18.1-esl            138c797adec7        5 days ago          286.9 MB
erlang     18.1                27ad0fc44644        5 days ago          741.5 MB
erlang     R16B03-1            e0deec5e1e72        6 days ago          740.2 MB
erlang     18.0.3              52d4a7a4a281        6 days ago          743.7 MB

c0b · 2015-09-30T05:40:46Z

I have researched some existing effort like this list, looks like many ones are already on this just haven't communicated yet

https://hub.docker.com/r/correl/erlang/
this might be the earliest and currently have the most stars by docker search erlang, provided erlang-otp-17.5 and rebar and relx in a single image, and compiling each one from source code, I haven't tried it, but presumably would be close to 1GB image;
https://hub.docker.com/r/unbalancedparentheses/erlang
this one support all versions of erlang R6B-0, R7, R8, ... R16, up to 17.4 from https://github.com/unbalancedparentheses/docker-erlang/blob/master/17.4/install.sh it also provided erlang & rebar & relx all compiled from source code
https://hub.docker.com/r/msaraiva/erlang/
this one is providing erlang-17.5 and 18, on top of Alpine Linux
docker image, it's very slim, as small as 16.78 MB, while erlang lib
is broken into very small packages, like most OS distro does, broken
into erlang-compiler, erlang-dialyzer, erlang-otp, erlang-snmp, and
etc. its base erlang:18 image provided 5 packages (compared a full erlang-otp has 52 lib
packages under /usr/lib/erlang/lib/...)
http://dl-4.alpinelinux.org/alpine/edge/main/x86_64/ (search erlang)
this one also provided elixir images in different Dockerfile,
presumably also slim but not full featured;
https://hub.docker.com/r/voidlock/erlang
is very similar to my way, support erlang R16 thru 18.1; I would not start my project if I know this earlier, but it installed update-locale for UTF-8 I'm not sure for what
https://github.com/synctree/docker-erlang/blob/master/R17/Dockerfile
this one installs erlang solutions precompiled deb on top of debian:8 (or debian:jessie) image.

$ docker search erlang
NAME                           DESCRIPTION         STARS     OFFICIAL   AUTOMATED
correl/erlang                  Erlang/OTP for Docker        14                   [OK]
...

While I just started from scratch, my goal is to provide images for a few last erlang releases (the last 4), close to full feature erlang, and relatively slim images;
before this PR to official images got merged, you can try it with cloning this repo, and build images locally,

https://github.com/c0b/docker-erlang-otp

So there are two ways to make images:

build from source code, from standard debian:8 image, start with apt-get install build-essential and gcc and some lib...-dev and download erlang-otp source code and build, this usually ends up with a fat image, close to 1GB;
install from some binary erlang packages, like the one from erlang-solutions, could end up with smaller image; packages from most OS distributions also provided erlang but relatively not up to date like debian https://packages.debian.org/source/sid/erlang fedora centos similar

Here from my repo I maintained one each for latest 4 erlang releases, (R15, R16, 17, 18), each with different variant, following the best practices from docker official images guideline, should end up with
full feature while relatively slim images: https://github.com/docker-library/official-images

the standard variant erlang:18, erlang:17, erlang:R16, erlang:R15 builds from source code, on top of https://hub.docker.com/_/buildpack-deps/ :jessie, it covered gcc compiler and some popular -dev packages, for port driver written in C; while it doesn't have java compiler so out of the standard erlang-otp provided 52 packages under /usr/lib/erlang/lib/... from this one odbc / jinterface / wxwidgets won't work, I assume to run GUI programs in docker is not popular, so here we can save space; jinterface is similar, the java dependencies are too fat, I assume demand is low;
the -onbuild variant for each erlang version, to utilize ONBUILD instruction from Dockerfile, those are for starters
-esl variant is to pull erlang-solutions deb package to install on top of debian:jessie, results in relatively slim image, but I am trying to avoid wxwidgets / jinterface dependencies, reasons same as
above.

All these images are almost full featured Erlang-OTP images (except wxwidgets & jinterface), you can run it like this once build locally,

$ docker run -it --rm erlang:18.1
Erlang/OTP 18 [erts-7.1] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V7.1  (abort with ^G)
1> uptime().                                          # the new uptime() shell command since OTP 18
3 seconds
ok
2> application:which_applications().
[{stdlib,"ERTS  CXC 138 10","2.6"},
 {kernel,"ERTS  CXC 138 10","4.1"}]
3>
User switch command
 --> q
$ docker run -it --rm erlang:18.1 /bin/bash
root@6d2c1ad52ae8:/# ls /usr/local/lib/erlang/lib/
asn1-4.0     cosProperty-1.2  edoc-0.7.17     gs-1.6    observer-2.1
 public_key-1.0.1 stdlib-2.6   xmerl-1.3.8 common_test-1.11
cosTime-1.2  eldap-1.2     hipe-3.13    orber-3.8    reltool-0.7
syntax_tools-1.7 compiler-6.0.1     cosTransactions-1.3
erl_docgen-0.4     ic-4.4    os_mon-2.4    runtime_tools-1.9.1
test_server-3.9 cosEvent-2.2     crypto-3.6.1  erl_interface-3.8
inets-6.0.1    ose-1.1    sasl-2.6 tools-2.8.1 cosEventDomain-1.2
debugger-4.1.1  erts-7.1     kernel-4.1     otp_mibs-1.1    snmp-5.2
typer-0.9.9 cosFileTransfer-1.2  dialyzer-2.8.1  et-1.5.1
megaco-3.18    parsetools-2.1  ssh-4.1 webtool-0.9 cosNotification-1.2
 diameter-1.11  eunit-2.2.11     mnesia-4.13.1  percept-0.8.11
ssl-7.1 wx-1.5
root@4cfd3172e8cc:/# ls /usr/local/lib/erlang/lib/ | wc -l
50

Size:

$ docker images |grep ^erlang
erlang     18.1-esl            138c797adec7        5 days ago          286.9 MB
erlang     18.1                27ad0fc44644        5 days ago          741.5 MB
erlang     R16B03-1            e0deec5e1e72        6 days ago          740.2 MB
erlang     18.0.3              52d4a7a4a281        6 days ago          743.7 MB

Comments are welcome.

c0b · 2015-10-04T10:01:24Z

there are not much discussions yet, while I got -esl variant:

$ docker images |awk 'NR == 1 || /^erlang/'
REPOSITORY         TAG                 IMAGE ID            CREATED          VIRTUAL SIZE
erlang             18.1-esl            9140e1a9913e        3 days ago       286.9 MB
erlang             18.1                f4ea1ad78f3f        3 days ago       741.5 MB
erlang             R16B03-1            46fe912a808c        3 days ago       737.9 MB
erlang             17.5                5b01e12ab52e        3 days ago       741 MB
erlang             18.0.3              52d4a7a4a281        10 days ago      743.7 MB

http://erlang.org/pipermail/erlang-questions/2015-October/thread.html#86266

c0b · 2015-10-07T22:13:09Z

/cc @rvirding hi Robert, could you comment here? as Compile from erlang-otp source code, vs install binaries from erlang-solutions, what is the preferred way?

c0b · 2015-10-09T21:06:38Z

when Robert Virding comes here in LA in this week for a training as Principal Language Expert @ Erlang Solutions I had a little conversation asked him to comment on this, while he may not have time to comment here, I just quote his words when I asked "Compile from erlang-otp source code, vs install binaries": /cc @rvirding

"Erlang Solutions provide pre-packaged erlang binary packages for most of the popular OS / Distributions it's convenient to use; while if you want more flexibility, sure you can also compile from source code"

[1] https://www.erlang-solutions.com/about/news/erlang-solutions-announces-appointment-robert-virding-principal-language-expert
[2] https://packages.erlang-solutions.com/erlang/ has pre-packaged binaries for Ubuntu, CentOS, Mac OS X, Debian, Windows, Fedora, Raspbian, as well as the source tar ball for each release.

tianon · 2015-10-13T21:23:10Z

That sounds to me like we don't gain much by compiling from source in this case? 😄 👍

c0b · 2015-10-14T22:59:21Z

Comparing to Python and Golang, python docker is always compiling from source code, because python.org not providing precompiled binaries (for Linux), while golang docker is installing binaries from golang.org if because that language team is officially providing in binaries;

I think here it's the same, erlang.org doesn't provide precompiled binaries either; although the commercial Erlang-Solutions Inc provided good binaries, but compile from source code still has the benefits of flexibility and support in time. have gcc in the image would be good for linkedin drivers written in C.

vovimayhem · 2015-11-06T19:15:16Z

Hi @c0b! I'm an Erlang/Elixir newbie "since today" :) @tianon redirected me back to this discussion.

Is it common for Erlang-based projects to use old Erlang releases (R15, R16)?

vovimayhem · 2015-11-06T21:04:14Z

At the moment, the erlang version at the apt repo (from packages.erlang-solutions.com) doesn't seem to be up to date with [https://github.com/erlang/otp/releases]...

This make the case for building from source, doesn't it? It would be very convenient to just install 'erlang-nox' from the debian repo, but again, it doesn't seem to be up to date.

vovimayhem · 2015-11-07T17:42:24Z

@c0b: Sent you a PR: "Feature: Improved variant names and builds", which should help :)

c0b · 2015-11-07T18:19:23Z

I agree, I can remove R15 / R16 from library/erlang in this PR and update docs PR. provide one last release for 17/18 each only.

vovimayhem · 2015-11-07T22:25:15Z

@c0b What happened to the LANG env var? I recall read somewhere that everything goes to funk if isn't set... but I'm not sure...

c0b · 2015-11-08T05:01:51Z

in my elixir implementation, that is the one really requiring utf-8, otherwise would give a warning; but the default erlang doesn't require that, the erlang default encoding is latin1 I think, and it works fine without LANG.

c0b · 2015-11-08T05:03:39Z

library/erlang

+
+17.5.6.4: git://github.com/c0b/docker-erlang-otp@7a86ea4d35b99d1c40a9f2fe1470615e096c952a 17
+17.5:     git://github.com/c0b/docker-erlang-otp@7a86ea4d35b99d1c40a9f2fe1470615e096c952a 17
+17:       git://github.com/c0b/docker-erlang-otp@7a86ea4d35b99d1c40a9f2fe1470615e096c952a 17


this PR is updated with one last version for 18 / 17 release only.

Did you want to bump this one more time before we merge to include some of these other changes? erlang/docker-erlang-otp@7a86ea4...master

vovimayhem · 2015-11-09T15:51:23Z

Coo �@c0b :) I was about to make another PR with rebar on it, but you already nailed it :)

yosifkit · 2015-11-09T21:49:07Z

The Dockerfiles look fine to me. And it looks like you contacted upstream on the mailing list, but there was not much response. It might be interesting if msaraiva wanted to join you in maintaining the images with his slim ones based on Alpine Linux or maybe those should be part of the Elixir discussion (up to you, but he seems to be already maintaining them and it might be useful to users).

LGTM, Build test of #1075; f63ba4e (erlang):

$ url="https://raw.githubusercontent.com/docker-library/official-images/f63ba4e28455b6143a8a7478e52e92828eab654a/library/erlang"
$ bashbrew build "$url"
Cloning erlang (git://github.com/c0b/docker-erlang-otp) ...
Processing erlang:18.1.3 ...
Processing erlang:18.1 ...
Processing erlang:18 ...
Processing erlang:latest ...
Processing erlang:17.5.6.4 ...
Processing erlang:17.5 ...
Processing erlang:17 ...
$ bashbrew list --uniq "$url" | xargs test/run.sh
testing erlang:18.1.3
    'utc' [1/4]...passed
    'cve-2014--shellshock' [2/4]...passed
    'no-hard-coded-passwords' [3/4]...passed
    'override-cmd' [4/4]...passed
testing erlang:17.5.6.4
    'utc' [1/4]...passed
    'cve-2014--shellshock' [2/4]...passed
    'no-hard-coded-passwords' [3/4]...passed
    'override-cmd' [4/4]...passed

vovimayhem · 2015-11-09T23:01:30Z

/cc @msaraiva

msaraiva · 2015-11-10T00:39:39Z

It might be interesting if msaraiva wanted to join you in maintaining the images with his slim ones based on Alpine Linux

It sounds interesting but there's a couple of things we should have in mind:

As I said on the mailing list, as far as I know, the OTP team doesn't build/test Erlang against musl libc. The build process, as it is right now, requires some patching. Is it ok for an official image to be based on a patched source code?
The images I maintain were originally designed to be as minimal as possible, so the user can choose later which erlang libraries should be installed according to his/her needs. Is this the approach we should have for the official image?

c0b · 2015-11-10T02:47:09Z

sure it should be shared maintained, if this can be merged and create a repo under https://github.com/docker-library/ that would be awesome.

link to musl libc is not what I am familiar. are there many docker images using it, and looks like its main consideration is code size, and targeting is embedded area? are there many success stories outside of embedded area, and how about mature level / compatibility review / performance benchmark comparison to other libc implementations
http://www.musl-libc.org/manual.html

for an official image for study purpose, may be still better to link with default glibc?

tianon · 2015-11-10T02:54:44Z

There are a number of images that have slim variants that are intended to be more minimal (not quite as minimal as alpine, but node is discussing an explicit alpine variant). I'd be wary of major upstream patching, since it'll ultimately be unsupported by upstream.

tianon · 2015-11-10T02:55:31Z

(more directly, I agree with @c0b that for now, we should probably stick to glibc and think about alpine as a potential future endeavor)

msaraiva · 2015-11-10T10:22:53Z

I agree with @tianon. For now, it's better to stick with glibc for the official image.

One latest version for each of Last two major releases supported, right there are 18.1, 17.5 from erlang otp releases: https://github.com/erlang/otp/releases

c0b · 2015-11-11T01:25:09Z

updated docker-erlang-otp/blob/master/18/Dockerfile for better /var/lib/apt/* cleanup and commit ids here in library/erlang

yosifkit · 2015-11-11T17:04:12Z

LGTM, Build test of #1075; 51334cd (erlang):

$ url="https://raw.githubusercontent.com/docker-library/official-images/51334cd945f4a94a07ca6d79702a902ff39f914a/library/erlang"
$ bashbrew build "$url"
Fetching erlang (git://github.com/c0b/docker-erlang-otp) ...
Processing erlang:18.1.3 ...
Processing erlang:18.1 ...
Processing erlang:18 ...
Processing erlang:latest ...
Processing erlang:17.5.6.4 ...
Processing erlang:17.5 ...
Processing erlang:17 ...
$ bashbrew list --uniq "$url" | xargs test/run.sh
testing erlang:18.1.3
    'utc' [1/4]...passed
    'cve-2014--shellshock' [2/4]...passed
    'no-hard-coded-passwords' [3/4]...passed
    'override-cmd' [4/4]...passed
testing erlang:17.5.6.4
    'utc' [1/4]...passed
    'cve-2014--shellshock' [2/4]...passed
    'no-hard-coded-passwords' [3/4]...passed
    'override-cmd' [4/4]...passed

tianon · 2015-11-11T17:18:39Z

LGTM 👍

Support for Erlang OTP on Docker

c0b · 2015-11-11T17:27:07Z

Nice! then how about docker-library/docs#348 ? and what else I can do to make it appearing at ?
https://hub.docker.com/_/erlang/

tianon · 2015-11-11T17:32:32Z

It has to build and push on the official server before it shows up there, but as soon as it's done it'll go out. We're taking a look at the docs now.

docker-library-bot added the library/erlang label Sep 24, 2015

c0b force-pushed the master branch from 5910c20 to bacfa13 Compare September 24, 2015 02:31

c0b mentioned this pull request Sep 24, 2015

docs for trying to be official docker images for Erlang OTP docker-library/docs#348

Merged

c0b force-pushed the master branch from bacfa13 to b3768e2 Compare September 24, 2015 06:39

c0b force-pushed the master branch 6 times, most recently from f7c6d70 to 090afd0 Compare October 2, 2015 06:29

c0b closed this Oct 2, 2015

c0b force-pushed the master branch from 090afd0 to 2515fde Compare October 2, 2015 06:30

c0b reopened this Oct 2, 2015

c0b force-pushed the master branch from 090afd0 to ea67cb4 Compare October 3, 2015 06:48

yosifkit added the moreinfo label Oct 13, 2015

tianon mentioned this pull request Nov 6, 2015

Base Erlang Docker Image - Request docker-library/rabbitmq#50

Closed

c0b force-pushed the master branch 2 times, most recently from d21a53c to f63ba4e Compare November 7, 2015 19:49

c0b reviewed Nov 8, 2015
View reviewed changes

c0b mentioned this pull request Nov 9, 2015

Move out Elixir to another repo erlang/docker-erlang-otp#3

Closed

c0b force-pushed the master branch 2 times, most recently from ec79293 to 862b66d Compare November 11, 2015 00:01

Support for Erlang OTP on Docker

51334cd

One latest version for each of Last two major releases supported, right there are 18.1, 17.5 from erlang otp releases: https://github.com/erlang/otp/releases

c0b force-pushed the master branch from 862b66d to 51334cd Compare November 11, 2015 00:05

tianon added a commit that referenced this pull request Nov 11, 2015

Merge pull request #1075 from c0b/master

fb00653

Support for Erlang OTP on Docker

tianon merged commit fb00653 into docker-library:master Nov 11, 2015

c0b mentioned this pull request Dec 9, 2015

Minimal image for Deployment to production erlang/docker-erlang-otp#6

Closed

yosifkit mentioned this pull request Mar 15, 2016

Erlang/OTP image #495

Closed

tianon added the new-image label Oct 5, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support for Erlang OTP on Docker #1075

Support for Erlang OTP on Docker #1075

c0b commented Sep 24, 2015

c0b commented Sep 24, 2015

c0b commented Sep 24, 2015

tianon commented Sep 28, 2015

md5 commented Sep 28, 2015

md5 commented Sep 28, 2015

c0b commented Sep 30, 2015

c0b commented Sep 30, 2015

c0b commented Oct 4, 2015

c0b commented Oct 7, 2015

c0b commented Oct 9, 2015

tianon commented Oct 13, 2015

c0b commented Oct 14, 2015

vovimayhem commented Nov 6, 2015

vovimayhem commented Nov 6, 2015

vovimayhem commented Nov 7, 2015

c0b commented Nov 7, 2015

vovimayhem commented Nov 7, 2015

c0b commented Nov 8, 2015

c0b Nov 8, 2015

tianon Nov 10, 2015

vovimayhem commented Nov 9, 2015

yosifkit commented Nov 9, 2015

vovimayhem commented Nov 9, 2015

msaraiva commented Nov 10, 2015

c0b commented Nov 10, 2015

tianon commented Nov 10, 2015

tianon commented Nov 10, 2015

msaraiva commented Nov 10, 2015

c0b commented Nov 11, 2015

yosifkit commented Nov 11, 2015

tianon commented Nov 11, 2015

c0b commented Nov 11, 2015

tianon commented Nov 11, 2015 via email

Support for Erlang OTP on Docker #1075

Support for Erlang OTP on Docker #1075

Conversation

c0b commented Sep 24, 2015

http://www.erlang.org/download.html

c0b commented Sep 24, 2015

c0b commented Sep 24, 2015

tianon commented Sep 28, 2015

md5 commented Sep 28, 2015

md5 commented Sep 28, 2015

c0b commented Sep 30, 2015

c0b commented Sep 30, 2015

c0b commented Oct 4, 2015

c0b commented Oct 7, 2015

c0b commented Oct 9, 2015

tianon commented Oct 13, 2015

c0b commented Oct 14, 2015

vovimayhem commented Nov 6, 2015

vovimayhem commented Nov 6, 2015

vovimayhem commented Nov 7, 2015

c0b commented Nov 7, 2015

vovimayhem commented Nov 7, 2015

c0b commented Nov 8, 2015

c0b Nov 8, 2015

Choose a reason for hiding this comment

tianon Nov 10, 2015

Choose a reason for hiding this comment

vovimayhem commented Nov 9, 2015

yosifkit commented Nov 9, 2015

vovimayhem commented Nov 9, 2015

msaraiva commented Nov 10, 2015

c0b commented Nov 10, 2015

tianon commented Nov 10, 2015

tianon commented Nov 10, 2015

msaraiva commented Nov 10, 2015

c0b commented Nov 11, 2015

yosifkit commented Nov 11, 2015

tianon commented Nov 11, 2015

c0b commented Nov 11, 2015

tianon commented Nov 11, 2015 via email