Deprecate Docker Machine

[chris-crone]

Tell us about your request
The Docker Machine project started in 2014 to make it easier to setup and run Docker in a virtual machine. One of the major use cases for this was to make it easier to use Docker on macOS and Windows. Since its release in 2016, Docker Desktop has been seen as the replacement for Docker Machine to develop with containers on macOS and Windows. Not only does Docker Desktop provide a Docker API endpoint but it also provides transparent handling of file sharing and networking between the host and VM which make container development easier.

While originally Docker Desktop did not support all widely used versions of macOS and Windows, this is no longer the case. Docker Desktop supports macOS 10.14 or later and Windows 10 (1909) or later.

If you are an active user of Docker Machine, please let us know here about what you're using it for and how it's helping you.

Which service(s) is this request for?
Docker Machine

[Aeolun]

Hi, since I opened the other issue. I’ll respond here first.

We’re mainly using docker-machine as a way to control docker servers hosted on other linux machines from one machine.

In this case it’s the Gitlab CI controller node spawning AWS spot instances and installing docker on them so they can be used to execute CI jobs on demand.

This doesn’t seem to be even close to the stated use case, but it works really well (aside from the occasional bug that prompted me to write the original ticket).

[d33kayyy]

I'm using docker-machine for systems with incompatible processors, specifically non-Intel hackintosh system, for example, hackintosh which uses AMD processor.

Apple Hypervisor framework relies solely on Intel's VT-x that we need docker-machine and VirtualBox as a workaround to use Docker.

[afbjorklund]

@chris-crone :

If you are an active user of Docker Machine, please let us know here about what you're using it for and how it's helping you.

We have a fork of libmachine in minikube, where it is used to start virtual machines for running Kubernetes clusters on.

For now, only provisioners and drivers have been forked, and the library is using https://github.com/machine-drivers/machine
This vanilla fork is mostly needed for some Windows bug fixes, that couldn't be merged to https://github.com/docker/machine

Now when Kubernetes doesn't even support* Docker container runtime anymore, we plan of forking the library as well...

Our provisioner is extended to be install alternative container runtimes*, and drivers now support running in containers (DinD)
We also did fixes to some of the included drivers, such as Hyper-V and some external drivers, such as KVM (actually: libvirt).

See https://minikube.sigs.k8s.io/docs/

https://github.com/kubernetes/minikube

1. If possible, we will also try to support cri-dockerd as a replacement for dockershim.

2. We are using containerd and buildkitd, to replace the dockerd running in the machine.

[divad1196]

Hi,

I personally use docker for my own usage and learning. I use docker swarm and docker-machine is the simplest way i currently have to setup multiple nodes at once with the exact same configuration using a loop (being able to provide multiple nodes at once in the cli instead of relying on bash loop would be great).

I don't think i would use it a lot in production use cases.

[afbjorklund]

If you are an active user of Docker Machine, please let us know here about what you're using it for and how it's helping you.

There are some more user stories at the original docker-machine deprecation announcement (from 2018), also boot2docker

• https://github.com/docker/machine (see 4537)

• https://github.com/boot2docker/boot2docker (see 1408)

Neither project has seen any involvement since 2019, so we declared them both dead last year (2020) - officially or unofficially

[erikkri]

I guess the usage of machine might increase with yesterdays announcement.

[markus-seidl]

Additional to yesterdays announcement, also the possibility to use the more advanced Parallels Software as virtualisation provider is what I like with docker-machine.

[r4j4h]

I like using it for convenience on Windows 8.1 machines since that version was not graced with WSL or native Docker for Windows support. It is a little more to-the-point than some scripts manually driving VirtualBox or Hyper-V, or vagrant and the like.

I also use it on Mac in favor over Docker for Mac, which still has issues out of the box with bridge interfaces.

Both of these cases are for development work only and I have alternatives to docker machine so am not dependent on it, just wanted to report that it still actively meets my needs.

[lethargosapatheia]

I don't think it's normal that Docker still give users the impression that docker-machine is being actively developed by not stating clearly in their documentation that it's become obsolete. I see questions related to docker-machine constantly on the internet, community channels etc.

[afbjorklund]

It was updated on https://github.com/docker/toolbox but not on https://github.com/docker/machine

And it's more abandoned than obsolete, as you can see by the number of replies from machine users

[justincormack]

I have archived the project as we have not worked on it for two years. The code and forks remain available if people still wish to use it.

[Gemeinagent]

Just to add my 2 cent: We use docker-machine to provision AWS EC2 instances with docker-machine create --driver amazonec2. This creates an instance which is then used as docker host, configured/provisioned and saved as AMI to be used in CloudFormation autoscaling groups.

I know there is such a thing as ECS and Kubernetes, but we are not quite "there" yet.

Anybody has a similar use case and a replacement already integrated?

[coffeespice]

Im using docker-machine to provision and maintain instances on AWS EC2 and private cloud. Its easier/faster to setup than Ansible.

[blulic]

I am evaluating the docker-machine as the open source replacement for the Docker Desktop for Mac, which is not any more free for the larger businesses:

https://www.docker.com/products/docker-desktop#:~:text=It%20remains%20free%20for%20small,professional%20use%20in%20larger%20businesses.

[fxi]

We use docker-machine + docker swarm to create and manage instances on Exoscale : https://github.com/exoscale/docker-machine-driver-exo

This solution was very convenient

[MatteoOreficeIT]

I'am using docker-machine to bypass missing SLAT support on my DELL PRECISION M6400 intel Duo (2) Quad 9300 to setup a vmware workstation docker daemon

[jvitormts]

I'm using docker-machine to provision instances on AWS EC2. Its faster to setup than Ansible.

[vladkosinov]

I'm using docker-machine to bootstrap and manage a few of my pet projects, there is just no simpler tool for such purposes. It's a pity that such a tool has has been abandoned

[lmapii]

I'm working in the embedded industry and was always pushing towards docker since it had the potential to be a great development environment, especially with the variety of tools that are required when working on different hardware projects (some tools being even incompatible to eachother, i.e., cannot be installed at the same time).

After a lot of very tedious work I managed to work around the problem that accessing physical ports (even a simple USB port) within a docker container is only possible in linux: Using a setup of boot2docker with a known VirtualBox image and docker-machine I managed to set up a whole hardware End-Of-Line test environment, the machine running the setup only required docker and docker-machine to be installed.

3 years later all those projects have been killed, and even the original issue docker/for-mac#900 that was opened in 2016 (!) has essentially zero feedback. Now that also docker-machine is dead for me it is virtually impossible to continue using docker for this.

So my experience with docker was this: Originally I wanted to use it to have reproducible, stable environments, with long ter support, to avoid the "works on my machine" problem and to tackle compatibility problems. Now, after only three years I'm not even able to spin up these environments anymore - which is pure horror.

I guess docker will continue to be a great tool for "standard" software development, but if you're not fitting the profile (don't tell me embedded is not big enough an industry) you're out of luck.

Long story short I'm now in a situation where I have to tell my clients that using docker comes with a considerable risk for non-standard use-cases.

[fxi]

FYI, there seems to be a fork still maintained by gitlab : https://gitlab.com/gitlab-org/ci-cd/docker-machine

[afbjorklund]

@fxi : the gitlab fork is only intended for gitlab.

https://docs.gitlab.com/runner/executors/docker_machine.html#forked-version-of-docker-machine says:

The intent of this fork is to fix critical and bugs affecting running costs only. No new features will be added.

There is a similar fork of machine over at machine-drivers...

[Abathargh]

Easy to use tools are the best choices to start learning something, or for simple/small use-cases.

I used docker-machine so many times to set up nodes to use to host a swarm mode stack, and it's so easy and nice for small projects. I used it a lot expecially in university and for personal projects. I still use it for all of those use cases.

The way it works so nicely with drivers for local VMs or service providers is really cool and lets me concentrate on solving problems instead of setup and configuration. I usually just have a small script that automates the setup and everything is done.

[thaJeztah]

I know Rancher maintains a fork (but I have not kept up with that one to know what changes it has compared to the original version); https://github.com/rancher/machine

[danielm-codefresh]

We use docker-machine to commission DigitalOcean VPS's. Is there an alternative tool that does the same thing that's still being maintained?

I've been furiously trying to find an alternative, there is none quite like docker-machine. The best hope is we fork it, get it ready for ARM and keep it alive.

Any success yet?

[Xantios]

We use docker-machine to commission DigitalOcean VPS's. Is there an alternative tool that does the same thing that's still being maintained?

I've been furiously trying to find an alternative, there is none quite like docker-machine. The best hope is we fork it, get it ready for ARM and keep it alive.

Any success yet?

@trentmurray I saw you created a fork, you're not alone. I was considering the same thing. maybe we should just raise this project from the dead (considering Docker for Mac stil blows, and I don't ever see it getting better to be honest)

[miloabolaffio]

docker-machine was incredibly efficient for spot workloads on the cloud.

[trentmurray]

We use docker-machine to commission DigitalOcean VPS's. Is there an alternative tool that does the same thing that's still being maintained?

I've been furiously trying to find an alternative, there is none quite like docker-machine. The best hope is we fork it, get it ready for ARM and keep it alive.

Any success yet?

@trentmurray I saw you created a fork, you're not alone. I was considering the same thing. maybe we should just raise this project from the dead (considering Docker for Mac stil blows, and I don't ever see it getting better to be honest)

Yes, my plan is to convert boot2docker for ARM though. I don't see a reason to continue developing the x architecture anymore - current docker-machine is good enough for that.

[afbjorklund]

Doing some minor updates to the following:

• https://github.com/machine-drivers/machine

  • adding go.mod support

• https://github.com/machine-drivers/boot2docker

  • rebuilding for 19.03.15

• https://github.com/machine-drivers/docker.github.io

  • documentation for machine

• https://github.com/machine-drivers/docker-machine-kvm

  • fixes for network (e1000)

For minikube, we are doing some major changes:

• https://github.com/afbjorklund/buildroot4containers (name TBD)

  • rebuilding the iso with arm64 support, based on buildroot

• https://github.com/machine-drivers/docker-machine-driver-qemu

  • updating the "qemu" driver, with support for kvm and hvf

Currently done on a fork, but they could be backported to libmachine.
I don't think the new binary releases will come from "machine-drivers"*.

* the organization was created to host drivers for machine, not all of it

The updated binaries need a new organization, and some new branding.
Can't call the projects "docker-machine" and "boot2docker", any longer.

They can still use Docker® Engine and Moby, and still use Tiny Core Linux.

[trentmurray]

@afbjorklund - where did you fork this to?

[afbjorklund]

@afbjorklund - where did you fork this to?

The minikube source code can be found at:

https://github.com/kubernetes/minikube

See #245 (comment) for more details about it

The upstream libmachine contributions are at:

https://github.com/machine-drivers/machine

docker/machine@master...machine-drivers:master

[McDoit]

Late to the party, but we do use docker-machine to build linux docker containers from a windows host on EC2

Worked flawlessly until today when i got this error
Error checking TLS connection: Error checking and/or regenerating the certs: There was an error validating certificates for host
And started to research it, and found this thread :(

[srsridharan]

For anyone who stumbles across this searching for a replacement to docker-machine and/or rancher-machine (whose docker engine is not updated beyond the boot2docker's engine version- which is outdated) you may find the following useful: multipass

[ColonelPanic-42]

Late to the party, but we do use docker-machine to build linux docker containers from a windows host on EC2

Worked flawlessly until today when i got this error Error checking TLS connection: Error checking and/or regenerating the certs: There was an error validating certificates for host And started to research it, and found this thread :(

Similar here. Getting ERROR: error during connect: Get https://192.168.1.4:2376/v1.24/info: x509: certificate has expired or is not yet valid when trying to run any Docker commands. Every source on the internet tells me to regenerate certificates with docker-machine, except I can't because it's deprecated. How am I supposed to recover from this?

[McDoit]

Wayback machine to your rescue @ColonelPanic-42 !
they have the old setup script, which you can run with
docker-machine create --engine-install-url=https://web.archive.org/web/20220410153512/https://get.docker.com
But I would recommend that you A, look through it, and B, create your own gist with the content

[thaJeztah]

See this section in the docs about how to protect the remote API with TLS; https://docs.docker.com/engine/security/protect-access/

[nonsenseless]

I was following the OReilly Docker Cookbook (which is apparently ancient) and it has a recipe for Starting a Docker Host in the Cloud (Digital Ocean) by using Docker Machine. I was interested in scripting this. Is there a current tool that replaces this functionality?

[dipakparmar]

I was following the OReilly Docker Cookbook (which is apparently ancient) and it has a recipe for Starting a Docker Host in the Cloud (Digital Ocean) by using Docker Machine. I was interested in scripting this. Is there a current tool that replaces this functionality?

take a look at the docker-machine fork by rancher (https://github.com/rancher/machine) if you looking to stick to same docker-machine pattern

[foremtehan]

Excuse me, How would you deploy with Docker Desktop ? Did i misunderstood the docker-machine? I saw some tutorials that deployed an app using docker-machine into a VPS server.

[jeffdill2]

What are folks using as an alternative now for doing multi-node testing on your local machine, since Docker Desktop doesn't fulfill this role? Kubernetes?

I'm coming from @robisenberg 's "Docker for Rails Developers" (which utilizes docker-machine), so I'm just trying to figure out the most logical approach to continuing through my studies. 😄

[Gemeinagent]

Well, it's not the same... But maybe https://multipass.run/ might be an alternative for you.
(multi node on local machine)

[EskeRahn]

Sad to hear it is deprecated. It seems such a clean solution on a virtual server on amazon to do this

docker-machine create --driver amazonec2 aws01

As suggested here https://stackoverflow.com/a/50646504/1164009