Change gemspec to not rely on the presence of git and the git repository

[courtland]

I'm hoping you will accept this pull request, or at least do something similar. Although having git ls-files is fairly common these days in gem specifications, its actually bad practice because it makes the gem dependent on the existence of git and the git repository. Not all deployment systems have git installed, even though one might want to use the latest git master of a particular gem.

Also, a new feature in the latest version of Bundler (1.2.x) enables "bundle package --all" to unpack git repositories specified in the Gemfile into vendor/cache, also removing the git repository (.git directory). This allows one to deploy gems for the latest master of a particular git repository without having to checkout the repository when deploying to production. Savon is currently broken when using this deployment methodology.

For example, in my Gemfile I have

gem 'savon', :git => 'git://github.com/savonrb/savon.git'

I run something like the following in my development environment

bundle package --all

Using savon (2.0.3) from git://github.com/savonrb/savon.git (at /Users/courtland/myapp/vendor/cache/savon-6de06caf367e) 

In my production environment I run

bundle install --deployment
fatal: Not a git repository (or any of the parent directories): .git

The "fatal: Not a git..." error comes from savon.

Thanks.

[rubiii]

hey @courtland,

i was able to reproduce your problem by running the two bundler commands without changing the .bundle and vendor folders. from what i can see, savon can be loaded even though bundler/rubygems prints the "fatal" warning.

i'm fine with changing the gemspec, but i'd like to understand whether this warning actually prevents you from using savon or has any other negative side effects.

cheers,
daniel

[courtland]

It definitely prevents me from using the gem, at least the latest master version.

I use a lot of other git-sourced gems from github, where I often track the latest master commit instead of the official gem, usually because they don't publish a gem. For most of them I have to fork the repository simply to change the git ls-files bit in the gemspec. I have an unusual production scenario involving hundreds of deployments where re-downloading gems from git repositories is not desirable and often not possible.

The same thing happens if you use the :path option in a Gemfile and point it at a local copy of the gem that doesn't have the git repository files (.git directory).

Here are some other examples of this issue:

postrank-labs/goliath#44

steveklabnik/require_relative#4

rubygems/bundler#1638

Thanks for your attention.

[rubiii]

i understand this is a problem to you, but i can also see why people like steve will probably not pay too much attention to your pull requests, because it's missing code to reproduce the workflow and problem. as i said, i'm willing to change savon's gemspec, but i'd like to learn about your workflow and the root cause of your problem before making the change.

if you could provide a simple commented script which describes your use case and reproduces this problem, i think people using "the bundler gemspec" would better understand the problem.

ps. i had a look at bundler's most recent template for creating gemspecs and noticed that it was changed by one of the referenced pull requests (bundler 2f84b9). i would definitely have to try this out, as i haven't noticed the gem.files method before and i'm not sure if this approach is better than yours, but what i would like to know is if this "new" bundler-default would work for you before comparing the two solutions.

[courtland]

Not sure who you mean by Steve? This is the only pull request I have out there of this nature.

The code/solution thats in this pull request I borrowed from one of the examples in my previous comment. I don't particularly like it, but its the most elegant thing I could come up with at the time and needed a solution immediately such that I could use the latest master version of savon in my application. It at least yields an Array that is identical to what git ls-files.split("\n") does. I thought that might give you a good starting point to figure out what you want to do, if anything.

I'm not familiar with the gem.files syntax or how it works, but that seems cleaner than doing a glob and would probably work for my environment.

I can understand why most people would not care to change this, especially with how prevalent the git ls-files strategy is, but I don't think the problem is all that complicated or requires a code example. If you want people to be able to use the latest source code of a gem in a Gemfile via bundler, and you want it to be compatible with as many platforms as possible, then you cannot assume the production platform is going to have git installed and/or have the git repository available for the particular gem.

My workflow is something like this...

I have a rails application that runs on FreeBSD. Its part of a commercial networking appliance product that uses rails as the framework for a web administration console. The rails application and various other source code are installed to a production system via a proprietary FreeBSD package. There is no Internet connection available during initial installation. Git is not installed on the system.

We develop the rails application using OS X. Source control is done using SVN.

The application's Gemfile contains

gem 'savon', :git => 'git://github.com/courtland/savon.git'

To upgrade my application to the latest version of savon, I would do something like...

bundle update
# savon master gets checked out via git
bundle package --all
# savon is now stored inside vendor/cache/savon-db02a5acbb24 (where db02... is the latest commit).
# This is done by bundler. vendor/cache/savon... does NOT contain the git repository (.git directory).
# I commit the contents of vendor/cache/savon... to my SVN tree.

Savon is then installed to a production system, still lacking the .git directory. Again, production system does not have a git binary.

To install my application's bundle in production I use

bundle install --deployment

The --deployment flag of bundler tells it to install gems from vendor/cache/... and NOT download anything from rubygems or any git repositories. This is extremely important to me because this often happens when there is no Internet connectivity. If the gemspec of a particular gem contains any git ... commands, bundler will fail to load the gem into the application's environment because it doesn't know what files are available. This is because git ls-files returns an empty Array.

The --all flag of "bundle package" and its affect on :git options is new to bundler 1.2. Before I was using :path in my Gemfile and manually checking out the latest version of a gem and pointing the Gemfile at it. For example...

# vendor/git/savon contains a checkout of the latest version of savon
gem 'savon', :path => 'vendor/git/savon'

Bundler 1.2 has made this unnecessary and essentially does this automatically now.
Frankly I wish "bundle package" would just build a gem from the latest git source and put it in vendor/cache.

Thanks again for your consideration. It continues to baffle me that people think its OK to rely on the existence and behavior of some external binary when describing the files that belong to a gem that supposedly works on any platform having ruby and other gem dependencies. You seem to want to do things "correctly" with savon so I thought I might be able to sway you in the other direction :)

[rubiii]

thanks for the detailed description. i'm going to merge your pull request later today.

ps. did you consider opening an issue or pull request at bundler to fix the root cause?

[courtland]

Nope. I didn't even realize that it was the default. I will go bug them. Thanks!

[rubiii]

fyi: savon.gemspec is changed on master and it's also included in the new v2.1.0 release.

[courtland]

Great. Thanks!

On Sun, Feb 3, 2013 at 4:39 AM, Daniel Harrington
notifications@gh.neting.ccwrote:

fyi: savon.gemspec is changed on master and it's included in the new
v2.1.0 release.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/380#issuecomment-13046445.