Support for sharing unix sockets

[BouncyLlama]

Expected behavior

When mounting a directory containing unix sockets the sockets should function the same as they do on a Linux host.

Actual behavior

The socket is 'there', but non-functional.

Information

After reading several forum threads, it appears that there is a workaround with socat over TCP, but this is rather slow.

The documentation has this to say: 'Socket files and named pipes only transmit between containers and between OS X processes -- no transmission across the hypervisor is supported, yet'
Hopefully this is a planned feature already, but I did not see any existing issues open in this tracker for this particular issue, although it relates to #410 which asks specifically for SSH_AUTH_SOCK to be supported.

Host OS: Mac OSX 10.10.5

Steps to reproduce the behavior

1. mount a directory containing unix sockets like so: '-v "/directorywithsockets:/otherdirectory"'
2. attempt to send data to/from the host/container via the socket

[dsheets]

This is on the roadmap.

[udondan]

Interestingly, even a socket created in the container is non-functional if it was created on a mounted volume of the host.

[jippi]

Any ETA on this? :)

Currently it's blocking hashicorp/nomad#1091 and other projects :)

[dsheets]

This is currently scheduled for resolution in November. Sorry for the delay.

[jippi]

@dsheets okay, will it be available in beta builds beforehand?

[dsheets]

Our goal is to ship it in beta builds in November.

[jippi]

amazing @dsheets - if you need any testing or otherwise beforehand, I'm willing to help out!

[Druotic]

Our goal is to ship it in beta builds in November.

@dsheets That statement makes me think the work for this is already underway. However, the status label is still "1-acknowledged". Is this currently being worked on, or still on the todo list?

[BlinkyStitt]

We are in November now :) Any guess when this will be in Beta? This feature blocks a lot for us.

[Druotic]

Any updates? Even a mention that this feature has been abandoned would be better than nothing.

[dsheets]

The feature has not been abandoned. It is still on the roadmap. Thanks for your patience.

[BlinkyStitt]

It is the last day of November. Any update on when this will be in the beta @dsheets?

[dsheets]

Work has begun but is currently delayed behind performance work. Sorry about that. Stay subscribed to get updates. Thanks for your patience!

[gugahoi]

Any updates on this?

[BlinkyStitt]

I've started using https://github.com/avsm/docker-ssh-agent-forward to work around this issue

[BlinkyStitt]

Any updates @dsheets ? Is there an issue tracking the performance work? If that is going to block for much longer, I am planning to spend some time cleaning up https://github.com/avsm/docker-ssh-agent-forward but if docker will have proper support soon I won't bother.

[KFoxder]

+1

[dsheets]

Support for this is planned and has been started but is delayed indefinitely behind other work. @wysenynja I would recommend cleaning up avsm/docker-ssh-agent-forward in the meantime.

[unixneo]

From here, I cannot get any process on macos outside the docker container to connect to a unix socket inside a Docker container (which is shared outside the container), see for example:

# socat -d -d UNIX-CONNECT:/Users/discourse/discourse/shared/socket-only/run/nginx.http.sock -
2020/04/12 09:46:19 socat[41943] N opening connection to LEN=67 AF=1 "/Users/discourse/discourse/shared/socket-only/run/nginx.http.sock"
2020/04/12 09:46:19 socat[41943] E connect(5, LEN=67 AF=1 "/Users/discourse/discourse/shared/socket-only/run/nginx.http.sock", 67): Connection refused
2020/04/12 09:46:19 socat[41943] N exit(1)

# ls -l /Users/discourse/discourse/shared/socket-only/run/nginx.http.sock
srw-rw-rw-  1 Tim  staff  0 Apr 12 08:42 /Users/discourse/discourse/shared/socket-only/run/nginx.http.sock

[unilynx]

It is my considered opinion that the only way this ever work completely right is if the Mac moves to a Linux base instead of a BSD base. I'm not holding my breath for that... The system calls to identify the peer on the connection rely on the two endpoints exist in the same process id space. Since Docker processes don't exist as far as MacOS is concerned, that's not happening.

But how is that different from native linux, where the other endpoint may be in a different PID namespace? SCM_CREDENTIALS (pid/uid/gid) is already meaningless. SCM_RIGHTS (file descriptors) might still be useful, although I don't see why you wouldn't be able to use proxy processes on either side of the virtual machine divide if you really wanted this.

Although it's probably still not worth the effort for docker. Most people will only want unix sockets for ssh agent forwarding, and specifically targetting that (properly integrating one of the workarounds listed in the issue) would probably suffice for many.

[Alan-R]

@unilynx Said it just right. Unless MacOS converts to running Linux instead of BSD, his comments are spot on. Pid space, user id space, group id space, etc are all disjoint between MacOS and the containers. Until MacOS converts to Linux this is going to stay the way it is. You can make your own estimation of how probable it is for MacOS to convert to Linux...

But ssh agent forwarding could be made to work with enough chewing-gum, baling wire and duct tape (and maybe a proxy or two).

[Frizlab]

macOS switching to Linux is, I think, not something we can count on, but having a workaround for SSH agent forwarding would be awesome.

[nopdotcom]

It would be appreciated if the mechanism for SSH agent forwarding were applicable to other services. Personally, I could really use the GPG agent socket (on macOS, usually at $HOME/.gnupg/S.gpg-agent.extra). Other people might appreciate the X11 socket (on macOS, pointed to by the contents of $DISPLAY; somewhere like /private/tmp/com.apple.launchd.fkXNRcDlhz/org.macosforge.xquartz:0).

[trinitronx]

There now exists an official workaround for this as of Docker for Mac >= 2.2.0.0.

The solution involves using a "magic path" specifically for SSH_AUTH_SOCK: /run/host-services/ssh-auth.sock

Mount this special path into the container via docker run args:

SSH_AUTH_SOCK_MAGIC_PATH='/run/host-services/ssh-auth.sock'
docker run -ti -v ${SSH_AUTH_SOCK_MAGIC_PATH}:${SSH_AUTH_SOCK_MAGIC_PATH} \
           -e SSH_AUTH_SOCK=${SSH_AUTH_SOCK_MAGIC_PATH} \
           some-container/needing-ssh-agent

Note that this does not resolve the main issue here: mounting arbitrary Unix sockets into a container. At least it works for ssh-agent / SSH_AUTH_SOCK!

[xpepermint]

Hum ... this issue was opened in 2016. Will this be resolved soon?

[Alan-R]

My advice to anyone new to this issue is to read all the comments. I think they describe it adequately. Specific use cases can be made to work with sufficient effort (as happened recently with ssh forwarding). I would not be shocked to see one or two more specific, narrow use cases taken care of in the next few years. The general problem is unlikely to be fully solved unless MacOS switches to a Linux base. I judge that to be highly unlikely unless something drastic happens in or to Apple. If you just want it to behave like a FIFO, then that's not so hard. However, UNIX domain sockets have some specific and useful properties which would be troublesome to implement between two dissimilar operating systems.

[adamnovak]

I originally came here for FIFO support; I don't recall seeing a FIFO workaround, besides "don't use a FIFO, use a network connection". Where's your source for the ability/method to share a plain FIFO? Ideally, I would think that Docker for Mac would watch the mounted directories, and smuggle the FIFO data between its own proxy processes in the VM and on the host, via the network. Has that or something like it been implemented?

…

On 5/10/20, Alan Robertson ***@***.***> wrote: My advice to anyone new to this issue is to read _all_ the comments. I think they describe it adequately. Specific use cases can be made to work with sufficient effort (as happened recently with ssh forwarding). I would not be shocked to see one or two more specific, narrow use cases taken care of in the next few years. The general problem is unlikely to be fully solved unless MacOS switches to a Linux base. I judge that to be highly unlikely unless something drastic happens in or to Apple. If you just want it to behave like a FIFO, then that's not so hard. However, UNIX domain sockets have some specific and useful properties which would be troublesome to implement between two dissimilar operating systems. -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: #483 (comment)

[fred-vogt]

@adamnovak - There is still no FIFO support as far as I know.

You can generally use a pair of FIFOs and a Mac listening port to proxy traffic between a pair of domain sockets.

This is facilitated by a pair of netcats in each environment.

This is as convoluted as it sounds - but it does work reliably.

Host:

# Mac - docker run wrapper - before launching the container
_PROXY_FIFO=`mktemp -u`
mkfifo $_PROXY_FIFO

PROXY_PORT=$(python -c 'import socket; s = socket.socket(); s.bind(("", 0)); print(s.getsockname()[1]); s.close()')

_PROXIED_SOCKET=
nc -lk 127.0.0.1 "$PROXY_PORT" <"$_PROXY_FIFO" | nc -U "$_PROXIED_SOCKET" >"$_PROXY_FIFO" &

trap "exit" INT TERM ERR
trap "{ kill 0; rm -f $_PROXY_FIFO; }" EXIT

Container: -e PROXY_PORT=...

# Called from inside the container entrypoint
_PROXY_FIFO=`mktemp -u`
mkfifo "$_PROXY_FIFO"

_PROXIED_SOCKET=
nc host.docker.internal "$PROXY_PORT" <"$_PROXY_FIFO" | nc -Ulk "$_PROXIED_SOCKET" >"$_PROXY_FIFO" &

Exhibit A Keybase client

NOTE: the client & server keybase versions must be an exact version match

keybase client usage from inside the container:

keybase --no-auto-fork --local-rpc-debug-unsafe A decrypt --infile ...

Unix domain sockets:

# Mac unix socket:
_PROXIED_SOCKET="${HOME}/Library/Group Containers/keybase/Library/Caches/Keybase/keybased.sock"

# Container unix socket:
_PROXIED_SOCKET="${HOME}/.config/keybase/keybased.sock"

Routing:

keybase client in container:
  "sends" - container:/keybased.sock => container:fifo => host:ephemeral-tcp-port
  "recv"  - host:ephemeral-tcp-port => ~/.config/keybase/keybased.sock
keybase server on Mac:
  "recv"  - host:ephemeral-tcp-port => host:/keybased.sock => host:fifo
  "sends" - host:fifo => host:ephemeral-tcp-port

[dancojocaru2000]

@unilynx

Most people will only want unix sockets for ssh agent forwarding

Most, but certainly not all.

The main issue for my personal use case is that I don't want ports exposed on the host. Instead, I would like a container port to be exposed on a UNIX socket and then use nginx to forward that UNIX socket to a certain hostname:

• app1.serveraddress.com
• app2.serveraddress.com
• app3.serveraddress.com

All these domains would point to the same computer and nginx would forward to a different UNIX socket based on the different hostname.

Exposing the 3 apps on ports - let's say - 7001, 7002 and 7003 and then doing some forwarding would mean that one would be able to access app1 from app2.serveraddress.com:7001, which isn't desirable.

[benlangfeld]

@unilynx

Most people will only want unix sockets for ssh agent forwarding

Most, but certainly not all.

The main issue for my personal use case is that I don't want ports exposed on the host. Instead, I would like a container port to be exposed on a UNIX socket and then use nginx to forward that UNIX socket to a certain hostname:

• app1.serveraddress.com

• app2.serveraddress.com

• app3.serveraddress.com

All these domains would point to the same computer and nginx would forward to a different UNIX socket based on the different hostname.

Exposing the 3 apps on ports - let's say - 7001, 7002 and 7003 and then doing some forwarding would mean that one would be able to access app1 from app2.serveraddress.com:7001, which isn't desirable.

You can use an overlay network for that, like Calico.

[dancojocaru2000]

How exactly would an overlay network help in my case?

[unilynx]

@unilynx

Most people will only want unix sockets for ssh agent forwarding

Most, but certainly not all.

The main issue for my personal use case is that I don't want ports exposed on the host. Instead, I would like a container port to be exposed on a UNIX socket and then use nginx to forward that UNIX socket to a certain hostname:

Why not publish the ports bound to only 127.0.0.1 ?

(I haven't personally tested with 127.0.0.1, but I do run a few dockers with their ports published to an alias 10.x.x.x address I've assigned to the localhost interface)

[dancojocaru2000]

The port would still be bounded.

Still, haven't thought about that approach! Thanks for reminding me I can probably do that!

[benlangfeld]

How exactly would an overlay network help in my case?

Your nginx instance would have interfaces on both the overlay and the external networks.

[trinitronx]

I definitely see another use case with $HOME/.gnupg/S.gpg-agent* sockets:

Building software inside a Docker container, and GPG signing the resulting build artifacts with a Yubikey plugged into the host OS. For example: Building a Debian package withdebuild + export DEBUILD_DPKG_BUILDPACKAGE_OPTS="-sa -k${DEBSIGN_KEYID}"

This would need access to the gpg-agent socket for signing the package & .dsc, .changes, and .buildinfo files.

[rkettelerij]

This is indeed very much needed in order to securely use GPG within Docker. GnuPG gpg-agent offers an extra domain socket which you can forward from the host to your container and hence use GPG inside the container. Lot's of stuff (e.g. pass or gopass, password manager) integrates with GPG so this isn't an exotic use case IMHO.

Since SSH forwarding works now, what would it take to make this work for GPG?

[AndreaCensi]

I am not sure I understand one thing: /var/run/docker.sock is a socket, right? and that one socket can be mounted in the containers, arbitrarily deep. Is that a special case that is different from user-created sockets?

[stephen-turner]

Sorry to disappoint everyone, but I'm closing this ticket as "Won't Fix". The main use case has been solved in #410. As described in recent comments, the general case is too hard / impossible, and is realistically never going to be done.

[Vanuan]

@stephen-turner what's the current state of osxfs? Is it stiil being used? There's been a talk about open sourcing it several years ago.

[stephen-turner]

It's still in the product at the moment, but is being replaced because of a lot of problems with it, particularly high CPU usage. There is no intent to open source it.

[docker-robott]

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked