Support RancherOS, Fixes #992

[ibuildthecloud]

This PR adds a RancherOS provisioner. As RancherOS is only officially supported on VirtualBox and AWS right now, this provisioner has been tested with those configurations. To try VirtualBox just do the following:

./docker-machine create -d virtualbox --virtualbox-boot2docker-url https://github.com/rancherio/os/releases/download/v0.3.1-rc2/machine-rancheros.iso dev

The way this provisioner works is two fold. First, we changed RancherOS to look and behave a lot like boot2docker. This makes it so that RancherOS ISO can be used as a drop in replacement for b2d.

Second, the provisioner code largely follows the approach of the Ubuntu provisioner. Instead of duplicating all the code I did a simple refactor of the Ubuntu provisioner and created a Generic provisioner in which I could share a lot of the code between the two provisioners.

Refs #905

[hairyhenderson]

I like the idea of a generic provisioner - that'll make life easier as others come along. But just be aware of the changes happening in #1090.

[ibuildthecloud]

It seems #1090 could benefit from this refactor too. Some of that code that that PR had to copy and paste is now in the GenericProvisioner.

[hairyhenderson]

@ibuildthecloud yeah - maybe a separate PR with the refactor would be easier to merge first?

/cc @ehazlett

[ibuildthecloud]

I agree. I can certainly do that.

[ibuildthecloud]

@hairyhenderson @ehazlett I created #1098 which this PR is based on. #1098 is the GenericProvisioner.

[nathanleclaire]

Cool. I'll have to review this when I get a chance. Thanks for going the extra mile with the GenericProvisioner refactor.

One thing this makes me think about is, we really need to change the --virtualbox-boot2docker-url flags to something more general like --virtualbox-image, since I'd like the flexibility for people to use it for this, or Ubuntu, etc. as well.

[ehazlett]

@ibuildthecloud awesome! i was wanting to add this in as well :)

Also, huge +1 to the Generic Provisioner!

Reviewing :)

[ehazlett]

Looking awesome! Tested with #1098 merged. Works well. Only warning i received was this during create:

Unsupported storage driver aufs, using overlay

Perhaps we should override the storage driver in the RancherOS provisioner config?

[nathanleclaire]

So this name is kind of a misnomer in this PR, huh? Since it looks like it gets used to get RancherOS ISO too.

[ibuildthecloud]

How about naming method "DownloadISOFromURL". Also the more misleading part is the message below that says "Downloading latest boot2docker release to ". I can change that to "Downloading %s to %s ", latestReleaseUrl, machineName

[ibuildthecloud]

Unsupported storage driver aufs, using overlay

@ehazlett I wasn't sure what the behavior you wanted here, so I put a warning. I notice in the RHEL drive you just silently set it to devicemapper, so I can do that same. I didn't want to fail because then the default invocation will always fail being that the default is aufs.

Should I just silently set it overlay?

[nathanleclaire]

I think I'd prefer a warning in both cases. There's also the possibility of what I mentioned here (#1090 (comment)) but it's a little more drawn out so warnings seem fine to me in the near-term.

[ibuildthecloud]

Just a heads up, I'm troubleshooting one issue with this provisioner. w/ VirtualBox I randomly get Error creating machine: ssh: handshake failed: read tcp 127.0.0.1:40048: connection reset by peer. I'm not too sure why, still looking at it.

[nathanleclaire]

Just a heads up, I'm troubleshooting one issue with this provisioner. w/ VirtualBox I randomly get Error creating machine: ssh: handshake failed: read tcp 127.0.0.1:40048: connection reset by peer. I'm not too sure why, still looking at it.

cc @sthulb

[ibuildthecloud]

Okay, the ssh error is a bug with RancherOS, not the provisioner. We'll fix on our side.

[ibuildthecloud]

@nathanleclaire @sthulb I change my mind. I think this might be an issue in the VirtualBox driver. At https://github.com/docker/machine/blob/master/drivers/virtualbox/virtualbox.go#L396 GetIP() is called but SSH might not be available at that time. There seems to be WaitForSSH() func that libmachine/host.go calls but that is after this call to GetIP. Seems like a WaitForSSH() should be called before GetIP().

[nathanleclaire]

Ah yes, this is why the virtualbox driver previously did its own waiting on SSH, in addition to the generic method, but we may have taken it out by accident.

It should definitely be added back in, I suppose in the Create function before we call for the IP address like you mentioned.

[ehazlett]

@ibuildthecloud @nathanleclaire i'm not sure I agree with the warnings on the storage provider. since we don't have a "recommended" how can we warn? To me, whatever is the best for the system should be specified and then potentially warned against if you specify an alternate.

[nathanleclaire]

To me, whatever is the best for the system should be specified and then potentially warned against if you specify an alternate.

I think the issue is mainly around, what happens if the user manually specifies --engine-storage-driver aufs. Because that is what the flag defaults to anyway, we have no way of checking inside of the provisioner whether or not the user has set it. So, we need to either warn them that they are going to be over-ruled, or figure out a way by which we can check (e.g. change the default to "" and make each provisioner decide what the default should be in their case, if they want to bail if user has set incompatible engine options, etc.).

[ehazlett]

@nathanleclaire i see -- thanks for clarification. I think I would rather have the provisioners set their recommended and then set the default to "". Since we don't recommend a specific driver for the engine, I think we should default to the provisioner and then warn if the user asks for a driver that is different.

Thoughts?

[ibuildthecloud]

I like the approach of defaulting to "". But if the user picks a driver that is unsupported we should probably fail as opposed to warning.

[nathanleclaire]

Yeah, failing seems more like the right thing to do here - I have a feeling that even if we log a warning, it's likely to get overlooked. Hopefully users confident enough to set the storage driver will understand that we have their best interests at heart.

[ibuildthecloud]

I created #1109 to default the storage driver to "". Also I updated this PR to match that behavior.

[nathanleclaire]

Nice - reviewing this PR is on my immediate todo list. Longer term, we should start a conversation about how we want to orient the VBox driver - personally, I don't think forcing you guys to provide an ISO in the format that our hardcoded-to-boot2docker driver expects is the right approach (I'd prefer the opposite - to force our code to be more general), and I'd like something more flexible to allow users to have Debian etc. VMs as well.

[ehazlett]

Reviewed / tested -- works great. LGTM

[nathanleclaire]

This needs #1108 before merge

[ehazlett]

#1108 merged

[ibuildthecloud]

@nathanleclaire I rebased this onto #1143. Now this PR should just run as is.

[nathanleclaire]

I'm getting sporadic errors on removal of a running RancherOS VM:

$ ./docker-machine_darwin-amd64 rm rancher
Error removing machine rancher: exit status 1
There was an error removing a machine. To force remove it, pass the -f option. Warning: this might leave it running on the provider.

Currently trying to investigate whether this is related to the Virtualbox driver or not.

[nathanleclaire]

Naturally as soon as I post that comment they vanish on me. Trying to dupe now.

[ibuildthecloud]

@nathanleclaire I see that error on machines that fail to create. for example if I get a panic, I do ctrl-c, or i manually delete in vbox gui.

[nathanleclaire]

Bingo

executing: /usr/bin/VBoxManage unregistervm --delete rancher
STDOUT:
STDERR: VBoxManage: error: Cannot unregister the machine 'rancher' while it is locked
VBoxManage: error: Details: code VBOX_E_INVALID_OBJECT_STATE (0x80bb0007), component Machine, interface IMachine, callee nsISupports
VBoxManage: error: Context: "Unregister(CleanupMode_DetachAllReturnHardDisksOnly, ComSafeArrayAsOutParam(aMedia))" at line 158 of file VBoxManageMisc.cpp

Error removing machine rancher: exit status 1
There was an error removing a machine. To force remove it, pass the -f option. Warning: this might leave it running on the provider.

Now to try and figure out if it also happens with boot2docker.

[nathanleclaire]

I see that error on machines that fail to create. for example if I get a panic, I do ctrl-c, or i manually delete in vbox gui.

Yeah, these are all finishing creation successfully though (ostensibly at least). Maybe there's a race causing issues.

[ehazlett]

@nathanleclaire do you think those random rm fails would have to do with my addition of the "wait for stopped" state? I don't know why it would -- just a thought.

[nathanleclaire]

do you think those random rm fails would have to do with my addition of the "wait for stopped" state? I don't know why it would -- just a thought.

Which wait for stopped?

[ehazlett]

In virtualbox: https://github.com/docker/machine/blob/master/drivers/virtualbox/virtualbox.go#L437 -- i had to add this for Windows.

[nathanleclaire]

The issue I described seems not easily dupe'able and I suspect either an issue with my VirtualBox setup, or something not directly related to this PR.

Tested and I'm in -- LGTM 👍

[nathanleclaire]

Are there still a few outstanding PRs which need to get merged in before this one?

[nathanleclaire]

Ah, also, I'd like to get something like https://github.com/docker/machine/pull/1090/files#diff-1a523bd9fa0dbf998008b37579210e12R1299 in either this or a followup PR depending on the order in which things resolve.

[ibuildthecloud]

@nathanleclaire I don't know of any other outstanding PRs. I can add docs, but it seem like it will overlap with the Red hat provisioner. Should I create a PR based on that one or want until red hat is merged?

[nathanleclaire]

I don't know of any other outstanding PRs. I can add docs, but it seem like it will overlap with the Red hat provisioner. Should I create a PR based on that one or want until red hat is merged?

I'd wait, I mostly just a want a commitment to do so once that's in / outlining the basic structure a little bit - I'm ready to merge this whenever you are @ehazlett.

[zchee]

🎉