bundle install requires sudo / root password or --path?

[rberger]

Ok I'm trying Chef-DK again with the assumption that I should recommend this for clients who don't have any other Ruby setup at a minimum and I see that Chef-DK is becoming a requirement for other tools. I've removed all my rvm stuff from my environment so I can pretend to be a new user with no special ruby setup. This is on Mac OS X 10.9.4.

My path and such:

/opt/chefdk/bin:/Users/rberger/.chefdk/gem/ruby/2.1.0/bin:/opt/chefdk/embedded/bin:/Users/rberger/anaconda/bin:/Users/rberger/bin:/usr/local/bin:/usr/local/sbin:/usr/local/share/npm/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/MacGPG2/bin:/usr/texbin:/usr/local/opt/go/libexec/bin

GEM_ROOT=/opt/chefdk/embedded/lib/ruby/gems/2.1.0
GEM_HOME=/opt/chefdk/embedded/lib/ruby/gems/2.1.0
GEM_PATH=/Users/rberger/.chefdk/gem/ruby/2.1.0:/opt/chefdk/embedded/lib/ruby/gems/2.1.0

which ruby
/opt/chefdk/embedded/bin/ruby
which gem
/opt/chefdk/embedded/bin/gem
which  bundle
/opt/chefdk/embedded/bin/bundle

If I do gem install install as a normal user it all works very nicely (seems to put them in /Users/rberger/.chefdk/gem/ruby/2.1.0/gems)

but if I do a bundle install I get:

Your user account isn't allowed to install to the system Rubygems.
You can cancel this installation and run:

    bundle install --path vendor/bundle

to install the gems into ./vendor/bundle/, or you can enter your password
and install the bundled gems to Rubygems using sudo.

Password:

Is there a way not to have to do the --path vendor/bundle, sudo or give it a password?

[rberger]

It seems that if I set export GEM_HOME=/Users/rberger/.chefdk/gem/ruby/2.1.0 then bundle install (and I presume gem install) installs new gems into my personal ~/.chefdk/gem/ruby/2.1.0. But it will still pick up and not reinstall gems already existing in /opt/chefdk/embedded/lib/ruby/gems/2.1.0.

This is the behavior I want / expect.

Wondering what that breaks though.

[sersut]

Chef achieves this by overriding those ENV variables with these before running a command.

Here is how they look like:

$ chef exec "ruby -e 'puts ENV[\"GEM_HOME\"]'"
/opt/chefdk/embedded/lib/ruby/gems/2.1.0
$ chef exec "ruby -e 'puts ENV[\"GEM_PATH\"]'"                                                
/Users/serdar/.chefdk/gem/ruby/2.1.0:/opt/chefdk/embedded/lib/ruby/gems/2.1.0
$ chef exec "ruby -e 'puts ENV[\"GEM_ROOT\"]'"                                                        
"/opt/chefdk/embedded/lib/ruby/gems/2.1.0"

I'm curious about the reason why you are going towards using a Gemfile especially in the non-power user scenario. One of the things we're trying to get to is to remove the need to use the Gemfiles when one is working with / around Chef. Can you share your use case maybe we're missing something bigger 😄

[rberger]

@sersut Not sure what you are asking? Many tools (and even cookbooks for testing) and such have the first step of using them is to say "bundle install" . I think just about anyone would be doing that pretty often.

But now that I think about it, is there any reason to just chown /opt/chefdk to my userid (or something similar for the group)?

[sersut]

chown'ing /opt/chefdk, using sudo or giving it a password will make bundle install the new gems into /opt/chefdk.

The problem with using Gemfiles with Chef DK is that the tools that come with Chef DK will not adhere to the gem versioning overrides in the Gemfiles. The binaries are modified to lock down the gems they are going to use to the gem versions that are shipped with Chef DK.

Going back to your previous question, I don't know of a way to install gems with bundle install into ~/.chefdk/gem/ruby/2.1.0 without using --path. @danielsdeleo might know of a way maybe?

[danielsdeleo]

I'd consider it a bundler bug. We customize rubygems to default to "user" installs, which I think bundler should respect. In my investigation for #146 I learned that bundler seems to have its own implementation for gem installation, which does not always work the way rubygems expects it to. I think the bundler and rubygems teams may be working together to have bundler reuse the rubygems installation code, which might fix the problem.

Poking around the issues on bundler, it seems to be their position that rubygems configuration shouldn't affect bundler, e.g., rubygems/bundler#2565 and rubygems/bundler#2667 and rubygems/bundler#710 . In any case there are some workarounds referenced in those tickets. Since we can't globally configure bundler for you, you will probably have to pick a workaround to use for now. You can file an issue with them and see if they're open to providing some configuration mechanism we could set globally ahead of time, even if they're unwilling to honor gemrc and similar.

I also found this issue, which says that sudo detection is broken on OS X 10.9: rubygems/bundler#3061 but it's not very informative.

[rberger]

As a point of feedback I've now tried to help two Chef Novices with no Ruby installed on their system other than the system ruby try to install chef dk and its not been pleasant. The getchef website process is very confusing. They couldn't find the Che DK downloader (UX issues). Once removing the standard chef installer that they accidently downloaded and installed because the website is so confusing, and found Chef-DK, The install goes smoothly.

And then we hit the bundler problem. Isn't it kind of common to have Gemfiles in the top level Chef repo for supporting any other code?

If the beginner follows the instructions to do a bundle install it then prompts them to enter their password, and then the gems get installed in the chef gem location instead of the user gem location. From that point on things are fubar. Get permission problems when running knife, and so on I presume because some gems got written into the chef gem locations.

I didn't quite get what the work arounds are for the bundler issue that @danielsdeleo mentioned.

[danielsdeleo]

@rberger generally installing gems to the omnibus bundle should work, but you are probably going to hit this rubygems bug: #146

Probably your best workaround is to create a bundler config in your home directory to set the path option to ~/.chefdk/gem/ruby/2.1.0. Since bundler doesn't have truly global config, only per-user config, we can't easily configure this in the omnibus package ahead of time.

Finally, it's our long term goal that your day-to-day work shouldn't require bundler at all, but it's going to take a bit of work to get to that point.

[rberger]

So remind me why I can't just set GEM_HOME="/Users/rberger/.chefdk/gem/ruby/2.1.0"
instead of "/opt/chefdk/embedded/lib/ruby/gems/2.1.0"?

Ie. instead of chef shell-init bash generating:

export PATH=/opt/chefdk/bin:/Users/rberger/.chefdk/gem/ruby/2.1.0/bin:/opt/chefdk/embedded/bin:/opt/chefdk/bin:/Users/rberger/.chefdk/gem/ruby/2.1.0/bin:/opt/chefdk/embedded/bin:/Users/rberger/anaconda/bin:/Users/rberger/bin:/usr/local/bin:/usr/local/sbin:/usr/local/share/npm/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/MacGPG2/bin:/usr/texbin:/usr/local/opt/go/libexec/bin
export GEM_ROOT="/opt/chefdk/embedded/lib/ruby/gems/2.1.0"
export GEM_HOME=/opt/chefdk/embedded/lib/ruby/gems/2.1.0
export GEM_PATH=/Users/rberger/.chefdk/gem/ruby/2.1.0:/opt/chefdk/embedded/lib/ruby/gems/2.1.0

Be the same but have GEM_HOME be set to /Users/rberger/.chefdk/gem/ruby/2.1.0

That seems to make chef-dk work the way I want/expect. I can use direnv to use pure Chef-DK in cookbooks / repos I'm doing chef work. When I use bunder it still knows and uses any official Chef Gems in /opt/chefdk/embedded/lib/ruby/gems/2.1.0 but then puts any new Gems in /Users/rberger/.chefdk/gem/ruby/2.1.0 and because GEM_PATH contains them both, it properly adds them all to the Ruby library search path.

Otherwise it trys to put the new gems from the bundle into /opt/chefdk/embedded/lib/ruby/gems/2.1.0 thus requiring sudo password (and screwing up chef-dk) or if I use the bundle install --path vendor/gems it duplicates everything each time.

So bottom line, any reason NOT to set GEM_HOME to /Users/rberger/.chefdk/gem/ruby/2.1.0 and all else set as chef shell-init does already?

[danielsdeleo]

That seems workable to me. The code is pretty simple, so it shouldn't be a difficult pull request, if you're up for it.

[rberger]

I created a pull request #160. As I noted. It works for me but I am not clear what potential impact it could have for others.

[danielsdeleo]

Closing this since #160 was merged. It'll be in 0.3.0