private git submodule fails

[michaelshobbs]

I'm trying to deploy a repo with a private submodule. It fails of course.

git push dokku@localhost:dokku.example.com master
Counting objects: 8, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (6/6), 655 bytes, done.
Total 6 (delta 3), reused 4 (delta 1)
-----> Cleaning up ...
-----> Building dokku.example.com ...
remote: HEAD is now at 2e28837... submodule for testing
remote: Host key verification failed.
remote: fatal: The remote end hung up unexpectedly
remote: Clone of 'git@github.com:example/nodecore' into submodule path 'nodecore' failed
To dokku@localhost:dokku.example.com
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'dokku@localhost:dokku.example.com'

I've read #314. However, embedding the username and password in the repo is not an option.

I'm running both dokku-deployment-keys and dokku-hostkeys-plugin. So, at first glance, I figured we just needed to run these plugins before the app gets built. Unfortunately there is currently no way of doing this. See here.

So, one approach might be to break lines 33-35 into a new built-in plugin and subsequently move the aforementioned key plugins into a new phase that is called by the build-in.

Thoughts on this? Is this interesting to anyone else or am I off my rocker?

[vmakhaev]

+1 though I don`t have an idea how to solve it

[motin]

So, at first glance, I figured we just needed to run these plugins before the app gets built. Unfortunately there is currently no way of doing this. See here.

By looking at those line of code, it seems that the plugin is built before the app gets built (which starts in https://github.com/progrium/dokku/blob/master/dokku#L38), so what am I missing?

[motin]

Ok now I see the problem, but it's not related to those lines the OP was referencing (https://github.com/progrium/dokku/blob/4e9d797d452d9e8499eec67ffad3e4140fc22736/dokku#L32-L37), but instead due to the fact that the submodule cloning happens way before the build hook is even triggered (technically git_archive_all is triggered before the dokku receive command which triggers the build).

Furthermore, it is not even related to any build logic nor any docker container, since all submodule cloning happens on the dokku host before it is sent to the buildpack.

Thus, if you can successfully authenticate by ssh key to github using the dokku user on the dokku host, your private submodules can then be checked out. Make sure the following works as logged in on the dokku host, and private submodules will thereafter be cloned successfully:

 su dokku
 ssh-keyscan -t rsa github.com >> ~/.ssh/known_hosts
 ssh -T git@github.com

[josegonzalez]

@michaelshobbs Mind making a documentation change for this?

[motin]

For inspiration: https://github.com/neam/dokku-host-provisioning#supporting-apps-that-have-submodules-that-reference-private-repositories

[davidlaprade]

I'm actually having the same difficulty, with a twist. I have a private repo referenced in the Gemfiles of two separate apps. One of these apps has no trouble bundling the gem from the private repo. The other does.

Here's the Gemfile (it's the same for both apps):

gem 'repo', git: 'git@github.com:private/repo.git', ref: 'd1be9f7'

The error I get is the following:

$ g push dokku master
Counting objects: 21, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (20/20), done.
Writing objects: 100% (21/21), 2.24 KiB | 0 bytes/s, done.
Total 21 (delta 13), reused 0 (delta 0)
-----> Cleaning up...
-----> Building myapp from herokuish...
-----> Adding BUILD_ENV to build environment...
-----> Ruby app detected
-----> Compiling Ruby/Rails
-----> Using Ruby version: ruby-2.0.0
-----> Installing dependencies using bundler 1.9.7
       Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment
       Fetching gem metadata from https://rails-assets.org/..
       Fetching gem metadata from https://rubygems.org/............
       Fetching version metadata from https://rails-assets.org/..
       Fetching version metadata from https://rubygems.org/...
       Fetching dependency metadata from https://rails-assets.org/..
       Fetching dependency metadata from https://rubygems.org/..
       Fetching git@github.com:private/repo.git
       Host key verification failed.
       fatal: Could not read from remote repository.
       Please make sure you have the correct access rights
       and the repository exists.
       Retrying git clone 'git@github.com:private/repo.git' "/tmp/build/vendor/bundle/ruby/2.0.0/cache/bundler/git/repo-1349f2a3af9920b5b235539271be4c0fbb2580a9" --bare --no-hardlinks --quiet due to error (2/3): Bundler::Source::Git::GitCommandError Git error: command `git clone 'git@github.com:private/repo.git' "/tmp/build/vendor/bundle/ruby/2.0.0/cache/bundler/git/repo-1349f2a3af9920b5b235539271be4c0fbb2580a9" --bare --no-hardlinks --quiet` in directory /tmp/build has failed.
       Host key verification failed.
       fatal: Could not read from remote repository.
       Please make sure you have the correct access rights
       and the repository exists.

But the other app bundles just fine:

$ g push production master
Counting objects: 5, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 618 bytes | 0 bytes/s, done.
Total 5 (delta 3), reused 0 (delta 0)
-----> Cleaning up...
-----> Building my_second_app from herokuish...
-----> Adding BUILD_ENV to build environment...
-----> Ruby app detected
-----> Compiling Ruby/Rails
-----> Using Ruby version: ruby-2.0.0
-----> Installing dependencies using bundler 1.9.7
       Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment
       Rubygems 2.0.14 is not threadsafe, so your gems must be installed one at a time. Upgrade to Rubygems 2.1.0 or higher to enable parallel gem installation.
       Using rake 10.3.2
       Using rails 4.0.5
       . . .
       Using repo 1.0.0 from git@github.com:private/repo.git (at d1be9f7)
       . . .

I've given SSH access to the dokku user, as described above. To confirm it was done correctly:

su - dokku
ssh -T git@github.com
Hi davidlaprade! You've successfully authenticated, but GitHub does not provide shell access.

Not only that, but I can also successfully clone the private repo that's causing the trouble:

su - dokku
cd
git clone git@github.com:private/repo.git
Cloning into 'repo'...
remote: Counting objects: 284, done.
remote: Compressing objects: 100% (249/249), done.
remote: Total 284 (delta 31), reused 277 (delta 24), pack-reused 0
Receiving objects: 100% (284/284), 136.45 KiB | 0 bytes/s, done.
Resolving deltas: 100% (31/31), done.
Checking connectivity... done.

Any idea why one might be able to connect but not the other?

[michaelshobbs]

FWIW, commenting on year old issues is not the best way to get heard. Fortunately, I was paying attention today.

To your particular issue, I think you will want to use https://github.com/cedricziel/dokku-deployment-keys and/or https://github.com/cedricziel/dokku-hostkeys-plugin

[davidlaprade]

FWIW, commenting on year old issues is not the best way to get heard. Fortunately, I was paying attention today.

Fair enough, @michaelshobbs. Thanks for the help! I've been experimenting with those plugins and will open an issue there if I can't get my problem resolved.

[mauron85]

I'm trying to make this work with nodejs and private npm packages hosted on bitbucket.

dokku$ ssh -T git@bitbucket.org

Reports:

authenticated via a deploy key.


You can use git or hg to connect to Bitbucket. Shell access is disabled.

This deploy key has read access to the following repositories:

...

All repos mentioned there are exactly those which fail, when I do git push to dokku.

npm ERR! git clone --template=/app/.npm/_git-remotes/_templates --mirror ssh://git@bitbucket.org/myco/somemodule.git /app/.npm/_git-remotes/ssh-git-bitbucket-org-myco-somemodule.git-git-ea958bef: Please make sure you have the correct access rights

[michaelshobbs]

Your issue is different than the original reported here. Have a look at the dokku-deploykeys and dokku-hostkeys plugins.

[mauron85]

@michaelshobbs Ach just installed both, and my problem is resolved. Thanks and sorry to be off.

[michaelshobbs]

No worries. Glad to help.

[epipheus]

So I have this same issue. hostkeys and deploykeys added

As dokku, passwordless access to github:

dokku@dataninja-1:~$ ssh -T git@github.com
Hi epipheus! You've successfully authenticated, but GitHub does not provide shell access.

I can even login and clone the repository I'm having an issue as a subcomponent.

log:

$ git push dokku master
Counting objects: 272, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (256/256), done.
Writing objects: 100% (272/272), 479.23 KiB | 5.27 MiB/s, done.
Total 272 (delta 125), reused 0 (delta 0)
-----> Cleaning up...
-----> Building dash from herokuish...
-----> Adding BUILD_ENV to build environment...
-----> Adding shared deployment-keys to build environment ...
-----> Adding host-keys to build environment ...
       Adding shared keys
-----> Fetching custom buildpack
-----> Ruby app detected
-----> Compiling Ruby/Rails
-----> Using Ruby version: ruby-2.4.0
-----> Installing dependencies using bundler 1.15.2
       Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment
       Fetching gem metadata from https://rubygems.org/.........
       Fetching version metadata from https://rubygems.org/..
       Fetching dependency metadata from https://rubygems.org/.
       Fetching git@github.com:cocoonventures/mountain_view.git
       Host key verification failed.
       fatal: Could not read from remote repository.
       Please make sure you have the correct access rights
       and the repository exists.

simple login as dokku and cloning yields:

dokku@dataninja-1:~/test_if_im_stupid$ git clone git@github.com:cocoonventures/mountain_view.git
Cloning into 'mountain_view'...
remote: Counting objects: 1871, done.
remote: Total 1871 (delta 0), reused 0 (delta 0), pack-reused 1871
Receiving objects: 100% (1871/1871), 112.88 MiB | 31.22 MiB/s, done.
Resolving deltas: 100% (495/495), done.
Checking connectivity... done.
Checking out files: 100% (1472/1472), done.

Am i missing something here?

[epipheus]

i notice that the hostkeys plugin mentions key generation being 2048b but github docs use 4096bit keys

[rasodu]

Cloning private repository was failing for me. I did following.

1. Switch to dokku user: sudo su - dokku
2. Add key for dokku user: ssh-keygen -t rsa -b 4096 -C "Dokku Host"
3. Then added the public key to my account
4. ssh-keyscan github.com >> home/dokku/.ssh/known_hosts

After doing the above steps, I was able to clone private repository.