Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: A check box for specifying whether to pull the corresponding repository before launching the container #79

Closed
ghost opened this issue Aug 11, 2014 · 50 comments
Assignees
Labels
RFE A Request for Enhancement. Usually linked to a PR that follows it.
Milestone

Comments

@ghost
Copy link

ghost commented Aug 11, 2014

Hi, i want my docker cloud instances to always use the latest image for a resulting container. Therefore it would be nice, to first pull the repository on the docker host.

So every time i build a new image and push it to the registry it will soon be used by all the cloud instances.

@michaelheyvaert
Copy link

+1

@larrycai
Copy link

+1, so far it is just pulled if it doesn't exist, pulling every time is good feature to have

@magnayn
Copy link
Contributor

magnayn commented Aug 21, 2014

Yes, definitely want this.

@halvorgb
Copy link

Great idea.

@ghost
Copy link
Author

ghost commented Feb 13, 2015

Does the java docker client lib already offer the ability to just pull a repo and do nothing else? So it would only be a checkbox within docker-plugin, that triggers the pull?

@halvorgb
Copy link

I began to create a pull request branching from master, but I encountered problems building the project. (Some docker-java dependency problems.) I tried to bump the docker-java version from 0.10.5-SNAPSHOT to 0.10.5, but that didn't work either (some packages referenced no longer exist in that version it seems.)

My (untested) fix for this issue is here in a .diff format. https://gist.github.com/halvorgb/c19c5076ffe6c23f65c0

Is there some known commit i could check out that builds? If my understanding is correct, this fix would be a simple one and I've already done most of it.

@thomassuckow
Copy link
Contributor

You can use the @magnayn docker-java if you add a snapshot repo #143 (comment)

@pjdarton
Copy link
Member

+1

FYI, when I did some work on a Jenkins enhancement, I found that there was a cloudbees server looking for pull requests on the source such that all one needed to do was to issue a pull request (via GitHub) and it'd automatically build and test your source for you, and let you know if it thought the pull request looked good or not.
I believe that updating the plugin version from a -SNAPSHOT build to an official build is something that the official plugin owner would do as part of the official release process (and result in a new version appearing in the Jenkins update page).

As for the code itself, it seemed to me like the proposed code changes would result in it fetching the image whether or not it needed to. Personally, I'd prefer to see it doing some form of up-to-date check rather than a brute-force "just re-fetch it all anyway" approach. Not sure how easy that would be to achieve though...

@KostyaSha
Copy link
Member

@pjdarton sorry i don't understand what do you mean and what are you talking about.

If there is different use cases, then feature should have different modes/strategies.

@pjdarton
Copy link
Member

At present, the official plugin only fetches the image if it doesn't have a copy of that image. It does not check that copy it has matches the one in the repository, so if it has previously fetched "foo:latest" then even if there's a new "foo:latest" it will continue using the old version of foo.

With the proposed changes, I think that the code will re-fetch "foo:latest" whether or not the repository has a new version of foo. This means that, in the (common) situation where foo has not changed and the cached copy of foo:latest is the same as the one in the repository, the code would still re-fetch foo:latest and overwrite what it has with an identical copy.
What I would suggest is that the normal use-case is that the code should ask the repository what foo:latest is, and only re-download it if the repository's answer indicated that the repository's copy of foo:latest is different from the one in the cache.

i.e. at present, the code does "download only if we don't have any copy", the proposed change is to "download a new copy every time", and my suggestion is to "check for an update every time and only download if a new copy is available or if we don't have a copy at all".

The point is that, if your specification is that a slave should use "foo:latest" then it should run on the latest version of foo (I think we're in agreement on this point), and so each time a new version of foo is created and the "latest" tag is updated to point at that new version then the new version should be downloaded (and I think we agree on this too), but (and this is where we differ) if foo doesn't change then there shouldn't be any unnecessary downloading.

@KostyaSha
Copy link
Member

@pjdarton i know what is this issue/request about and how this can be implemented. I don't understand this comments:

FYI, when I did some work on a Jenkins enhancement, I found that there was a cloudbees server looking for pull requests on the source such that all one needed to do was to issue a pull request (via GitHub) and it'd automatically build and test your source for you, and let you know if it thought the pull request looked good or not.
I believe that updating the plugin version from a -SNAPSHOT build to an official build is something that the official plugin owner would do as part of the official release process (and result in a new version appearing in the Jenkins update page).

@pjdarton
Copy link
Member

One creates a fork of the code on GitHub, do work, putting your changes into that fork, and then one issues a "pull request" (using the GitHub web interface).
On the core Jenkins code, there was a server (run by cloudbees) monitoring for pull requests that built the code (from the pull request) and automatically added a comment to the pull-request, indicating whether or not it compiled.
i.e. It tested that the pull-request would not break the code, thereby removing uncertainty caused by problems purely local to the developer's own environment.

I don't know if such a facility is set up for this plugin's code, but there was (and probably still is) one for the core Jenkins code, and as the proposed fix mentioned "problems building the project" that were preventing the contributor from testing their fix, I thought this could be relevant.

Regarding the -SNAPSHOT comment - responsibility for doing that typically lies with the person/people who have the authority/access to action a "pull request". You appear to have "commit" rights to this repository, so I assume that includes you.

@KostyaSha
Copy link
Member

Your are commenting "feb 13" repository state and now 23 apr. Library issues were resolved, master is buildable, pr builds works for all plugins hosted under @jenkinsci.

@pjdarton
Copy link
Member

The latest version of the plugin (according to the Jenkins update site, http://updates.jenkins-ci.org/download/plugins/docker-plugin/) is 0.8, and that was released on Oct 2014, which implies that nothing has been released since then. That's why I assumed that the "feb 13" comments are still relevant.

If there is an updated version of the plugin, please publish it.

@KostyaSha
Copy link
Member

It available in experimental plugin update center, there is also #177 new beta will be published soon. Release will be done when build will be stable.

@pjdarton
Copy link
Member

What is the URL for the "experimental plugin update center" that you refer to?
I'm not keen on using something that is not "stable" (normally I would wait for the release), but if we (myself and my work colleagues) can try out the experimental releases then we could evaluate whether or not the newer version is sufficiently stable for our purposes (and, if you wish us to, we may be able to provide you with reasonably detailed bug reports should things fail).

Also, can you give me an estimate when the next official release will be?

@KostyaSha
Copy link
Member

http://jenkins-ci.org/content/experimental-plugins-update-center
I'm not a payed developer, i do coding in out-of-work time, so i have no estimates. And this becomes offtopic for this issue. I hope close more issues and update beta release on weekend.

@pjdarton
Copy link
Member

Thanks for the info. I'll take a look and see if that solves the problem (that this thread was about) for us.

Re: estimates - no problem - I understand.

@KostyaSha
Copy link
Member

@magnayn As temp solution i think it makes sense to change default behaviour to pull ":latest" every time and other only when doesn't exist. Later this can be moved to default pulling strategy with some other variants like "never pull strategy" and etc.

@KostyaSha
Copy link
Member

Hi all, does anyone wants test #289 ?

@DanielHouston
Copy link

Hi @KostyaSha.
I installed the plugin manually from https://jenkins.ci.cloudbees.com/job/plugins/job/docker-plugin/442/com.nirima$dockerplugin/.

I get the following error on Jenkins start up:

Jul 29, 2015 1:32:07 AM INFO org.eclipse.jetty.util.log.JavaUtilLog info
Started SelectChannelConnector@0.0.0.0:8080
Jul 29, 2015 1:32:07 AM INFO winstone.Logger logInternal
Winstone Servlet Engine v2.0 running: controlPort=disabled
Jul 29, 2015 1:32:08 AM INFO jenkins.InitReactorRunner$1 onAttained
Started initialization
Jul 29, 2015 1:32:09 AM INFO jenkins.InitReactorRunner$1 onAttained
Listed all plugins
Jul 29, 2015 1:32:09 AM INFO jenkins.InitReactorRunner$1 onAttained
Prepared all plugins
Jul 29, 2015 1:32:10 AM INFO jenkins.InitReactorRunner$1 onAttained
Started all plugins
Jul 29, 2015 1:32:10 AM INFO jenkins.InitReactorRunner$1 onAttained
Augmented all extensions
Jul 29, 2015 1:32:22 AM WARNING hudson.ExtensionFinder$GuiceFinder$FaultTolerantScope$1 error
Failed to instantiate Key[type=com.nirima.jenkins.plugins.docker.builder.DockerBuilderPublisher$DescriptorImpl, annotation=[none]]; skipping this component
com.google.inject.ProvisionException: Guice provision errors:

1) Error injecting constructor, java.lang.NoClassDefFoundError: org/jenkinsci/plugins/tokenmacro/MacroEvaluationException
  at com.nirima.jenkins.plugins.docker.builder.DockerBuilderPublisher$DescriptorImpl.<init>(DockerBuilderPublisher.java:317)

1 error
    at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:52)
    at com.google.inject.Scopes$1$1.get(Scopes.java:65)
    at hudson.ExtensionFinder$GuiceFinder$FaultTolerantScope$1.get(ExtensionFinder.java:428)
    at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41)
    at com.google.inject.internal.InjectorImpl$3$1.call(InjectorImpl.java:1005)
    at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1051)
    at com.google.inject.internal.InjectorImpl$3.get(InjectorImpl.java:1001)
    at hudson.ExtensionFinder$GuiceFinder._find(ExtensionFinder.java:390)
    at hudson.ExtensionFinder$GuiceFinder.find(ExtensionFinder.java:381)
    at hudson.ClassicPluginStrategy.findComponents(ClassicPluginStrategy.java:388)
    at hudson.ExtensionList.load(ExtensionList.java:349)
    at hudson.ExtensionList.ensureLoaded(ExtensionList.java:287)
    at hudson.ExtensionList.getComponents(ExtensionList.java:167)
    at hudson.DescriptorExtensionList.load(DescriptorExtensionList.java:185)
    at hudson.ExtensionList.ensureLoaded(ExtensionList.java:287)
    at hudson.ExtensionList.iterator(ExtensionList.java:156)
    at org.apache.commons.jelly.expression.ExpressionSupport.evaluateAsIterator(ExpressionSupport.java:101)
    at org.apache.commons.jelly.tags.core.ForEachTag.doTag(ForEachTag.java:89)
    at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
    at org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98)
    at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
    at org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98)
    at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
    at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
    at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:120)
    at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
    at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
    at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:95)
    at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63)
    at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53)
    at org.kohsuke.stapler.jelly.JellyRequestDispatcher.forward(JellyRequestDispatcher.java:55)
    at hudson.util.HudsonIsLoading.doDynamic(HudsonIsLoading.java:45)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:298)
    at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:161)
    at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:96)
    at org.kohsuke.stapler.MetaClass$12.dispatch(MetaClass.java:391)
    at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
    at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
    at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649)
    at org.kohsuke.stapler.Stapler.service(Stapler.java:238)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:686)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1494)
    at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:132)
    at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:123)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
    at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:49)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
    at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:169)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
    at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
    at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
    at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1474)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:533)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
    at org.eclipse.jetty.server.Server.handle(Server.java:370)
    at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
    at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:949)
    at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1011)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:644)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
    at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:668)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
    at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoClassDefFoundError: org/jenkinsci/plugins/tokenmacro/MacroEvaluationException
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(Class.java:2615)
    at java.lang.Class.getMethod0(Class.java:2856)
    at java.lang.Class.getMethod(Class.java:1668)
    at hudson.model.Descriptor.<init>(Descriptor.java:284)
    at hudson.tasks.BuildStepDescriptor.<init>(BuildStepDescriptor.java:56)
    at com.nirima.jenkins.plugins.docker.builder.DockerBuilderPublisher$DescriptorImpl.<init>(DockerBuilderPublisher.java:317)
    at com.nirima.jenkins.plugins.docker.builder.DockerBuilderPublisher$DescriptorImpl$$FastClassByGuice$$fbd60072.newInstance(<generated>)
    at com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)
    at com.google.inject.internal.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:61)
    at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:108)
    at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:88)
    at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:269)
    at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
    at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1058)
    at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
    ... 82 more
Caused by: java.lang.ClassNotFoundException: org.jenkinsci.plugins.tokenmacro.MacroEvaluationException
    at jenkins.util.AntClassLoader.findClassInComponents(AntClassLoader.java:1376)
    at jenkins.util.AntClassLoader.findClass(AntClassLoader.java:1326)
    at jenkins.util.AntClassLoader.loadClass(AntClassLoader.java:1079)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    ... 98 more

Jul 29, 2015 1:32:23 AM INFO jenkins.InitReactorRunner$1 onAttained
Loaded all jobs
Jul 29, 2015 1:32:23 AM INFO hudson.model.AsyncPeriodicWork$1 run
Started Download metadata
Jul 29, 2015 1:32:23 AM INFO hudson.model.AsyncPeriodicWork$1 run
Finished Download metadata. 36 ms
Jul 29, 2015 1:32:24 AM INFO org.jenkinsci.main.modules.sshd.SSHD start
Started SSHD at port 44493
Jul 29, 2015 1:32:24 AM INFO jenkins.InitReactorRunner$1 onAttained
Completed initialization
Jul 29, 2015 1:32:24 AM INFO hudson.WebAppMain$3 run
Jenkins is fully up and running 

Am I missing something in addition to the .hpi?

@KostyaSha
Copy link
Member

Yes, you need token-macro plugin. Better install docker-plugin from update center and then update docker-plugin hpi. Either you need install all dependencies manually.

@DanielHouston
Copy link

Thanks for the quick reply. Manually installing the token-macro plugin fixed the initial issue. I appear to have further dependency issues

Jul 29, 2015 1:46:49 AM INFO com.nirima.jenkins.plugins.docker.DockerCloud provision
Asked to provision 1 slave(s) for: test
Jul 29, 2015 1:46:49 AM INFO com.nirima.jenkins.plugins.docker.DockerCloud provision
Will provision 'evarga/jenkins-slave', for label: 'test', in cloud: 'SmallerVPS'
Jul 29, 2015 1:46:49 AM SEVERE hudson.triggers.SafeTimerTask run
Timer task hudson.slaves.NodeProvisioner$NodeProvisionerInvoker@46d09390 failed
java.lang.NoClassDefFoundError: com/google/common/base/Preconditions
    at com.github.dockerjava.core.DockerClientConfig$DockerClientConfigBuilder.withUri(DockerClientConfig.java:445)
    at com.github.dockerjava.core.DockerClientConfig$DockerClientConfigBuilder.withProperties(DockerClientConfig.java:421)
    at com.github.dockerjava.core.DockerClientConfig.createDefaultConfigBuilder(DockerClientConfig.java:233)
    at com.github.dockerjava.core.DockerClientConfig.createDefaultConfigBuilder(DockerClientConfig.java:222)
    at com.nirima.jenkins.plugins.docker.client.ClientConfigBuilderForPlugin.<init>(ClientConfigBuilderForPlugin.java:36)
    at com.nirima.jenkins.plugins.docker.client.ClientConfigBuilderForPlugin.dockerClientConfig(ClientConfigBuilderForPlugin.java:42)
    at com.nirima.jenkins.plugins.docker.DockerCloud.getClient(DockerCloud.java:151)
    at com.nirima.jenkins.plugins.docker.DockerCloud.countCurrentDockerSlaves(DockerCloud.java:430)
    at com.nirima.jenkins.plugins.docker.DockerCloud.addProvisionedSlave(DockerCloud.java:453)
    at com.nirima.jenkins.plugins.docker.DockerCloud.provision(DockerCloud.java:188)
    at hudson.slaves.NodeProvisioner$StandardStrategyImpl.apply(NodeProvisioner.java:685)
    at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:286)
    at hudson.slaves.NodeProvisioner.access$000(NodeProvisioner.java:57)
    at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:782)
    at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:51)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

This is after a fresh install of Jenkins and docker-plugin 0.10.2. I have tried updating the plugin to 0.10.3 through the advanced tab under Manage Plugins settings. I've also tried simply replacing the .jpi in my JENKINS_HOME/plugins/ folder.

If I am missing further dependencies, I don't see why they were not installed when I installed version 0.10.2?

@KostyaSha
Copy link
Member

Remove jpi and hpi for docker-plugin and put new hpi

@DanielHouston
Copy link

I have done this and have the same
java.lang.NoClassDefFoundError: com/google/common/base/Preconditions error

@KostyaSha
Copy link
Member

Strange, because docker-plugin is not using this classes.
UPD, Found, seems it bad shaded library

@KostyaSha
Copy link
Member

Could you provide list of installed plugins?

@DanielHouston
Copy link

If it is still useful, these are the installed plugins
http://i.imgur.com/7xwnlf4.png.

If there is a new plugin built after the shaded library has been fixed, I will be happy to test it asap

@KostyaSha
Copy link
Member

Please try https://jenkins.ci.cloudbees.com/job/plugins/job/docker-plugin/443/com.nirima$docker-plugin/ but firstly remove old jpl/jpi/hpi files

@DanielHouston
Copy link

I removed the old hpi and the docker-plugin folder in JENKINS_HOME/plugins and replaced it with the hpi from the build. But I still have the same NoClassDefFound error

@DanielHouston
Copy link

It would be good to see if this problem can be recreated by someone else to ensure it's not only my environment. Is anyone else able to test the plugin?

@DanielHouston
Copy link

I have had a colleague test build 443 on a different machine, by installing 0.10.2 and replacing the hpi with the 0.10.3 snapshot. He also gets the same java.lang.NoClassDefFoundError: com/google/common/base/Preconditions error.

@KostyaSha
Copy link
Member

@KostyaSha
Copy link
Member

@DanielHouston
Copy link

What I assume was your local build (from dropbox) gave Bad template 'evarga/jenkins-slave:latest' in cloud 'SmallerVPS': 'null'. Trying next template... errors in Jenkins log (template was working in 0.10.2)

<hudson>
  <clouds>
    <com.nirima.jenkins.plugins.docker.DockerCloud plugin="docker-plugin@0.10.3-SNAPSHOT">
      <name>SmallerVPS</name>
      <templates>
        <com.nirima.jenkins.plugins.docker.DockerTemplate>
          <configVersion>2</configVersion>
          <labelString>test</labelString>
          <launcher class="com.nirima.jenkins.plugins.docker.launcher.DockerComputerSSHLauncher">
            <sshConnector plugin="ssh-slaves@1.9">
              <port>22</port>
              <credentialsId>xxxxxxxxx</credentialsId>
              <jvmOptions></jvmOptions>
              <javaPath></javaPath>
              <maxNumRetries>0</maxNumRetries>
              <retryWaitTime>0</retryWaitTime>
            </sshConnector>
          </launcher>
          <remoteFsMapping></remoteFsMapping>
          <remoteFs>/home/jenkins</remoteFs>
          <instanceCap>3</instanceCap>
          <mode>EXCLUSIVE</mode>
          <retentionStrategy class="com.nirima.jenkins.plugins.docker.strategy.DockerOnceRetentionStrategy">
            <idleMinutes>10</idleMinutes>
            <idleMinutes defined-in="com.nirima.jenkins.plugins.docker.strategy.DockerOnceRetentionStrategy">10</idleMinutes>
          </retentionStrategy>
          <numExecutors>1</numExecutors>
          <dockerTemplateBase>
            <image>evarga/jenkins-slave:latest</image>
            <dockerCommand></dockerCommand>
            <lxcConfString></lxcConfString>
            <hostname></hostname>
            <dnsHosts/>
            <volumes/>
            <volumesFrom2/>
            <environment/>
            <bindPorts></bindPorts>
            <bindAllPorts>false</bindAllPorts>
            <privileged>true</privileged>
            <tty>false</tty>
            <extraHosts class="java.util.Collections$UnmodifiableRandomAccessList" resolves-to="java.util.Collections$UnmodifiableList">
              <c class="list"/>
              <list reference="../c"/>
            </extraHosts>
          </dockerTemplateBase>
          <removeVolumes>false</removeVolumes>
          <pullStrategy>PULL_LATEST</pullStrategy>
        </com.nirima.jenkins.plugins.docker.DockerTemplate>
      </templates>
      <serverUrl>http://xxxxxxxxxxxxx</serverUrl>
      <connectTimeout>600</connectTimeout>
      <readTimeout>600</readTimeout>
      <version></version>
      <credentialsId></credentialsId>
      <containerCap>6</containerCap>
    </com.nirima.jenkins.plugins.docker.DockerCloud>
  </clouds>
  <quietPeriod>5</quietPeriod>
  <scmCheckoutRetryCount>0</scmCheckoutRetryCount>
  <views>
    <hudson.model.AllView>
      <owner class="hudson" reference="../../.."/>
      <name>All</name>
      <filterExecutors>false</filterExecutors>
      <filterQueue>false</filterQueue>
      <properties class="hudson.model.View$PropertyList"/>
    </hudson.model.AllView>
  </views>
  <primaryView>All</primaryView>
  <slaveAgentPort>0</slaveAgentPort>
  <label></label>
  <nodeProperties/>
  <globalNodeProperties/>
</hudson>

Config.xml for reference

I will try build 445

@DanielHouston
Copy link

Same Bad Template error on pr build 445

@DanielHouston
Copy link

I removed the cloud configuration and re-added it after installing build 445 to see if it was old 0.10.2 configurations not working with the new plugin. Same error

@KostyaSha
Copy link
Member

Thanks, @DanielHouston could your provide log from system jenkins log?

@DanielHouston
Copy link

You're welcome. Logs since latest reboot:

Jul 30, 2015 4:05:59 PM winstone.Logger logInternal
INFO: Winstone Servlet Engine v2.0 running: controlPort=disabled
Jul 30, 2015 4:06:00 PM jenkins.InitReactorRunner$1 onAttained
INFO: Started initialization
Jul 30, 2015 4:06:01 PM jenkins.InitReactorRunner$1 onAttained
INFO: Listed all plugins
Jul 30, 2015 4:06:01 PM jenkins.InitReactorRunner$1 onAttained
INFO: Prepared all plugins
Jul 30, 2015 4:06:02 PM jenkins.InitReactorRunner$1 onAttained
INFO: Started all plugins
Jul 30, 2015 4:06:02 PM jenkins.InitReactorRunner$1 onAttained
INFO: Augmented all extensions
Jul 30, 2015 4:06:20 PM jenkins.InitReactorRunner$1 onAttained
INFO: Loaded all jobs
Jul 30, 2015 4:06:20 PM hudson.model.AsyncPeriodicWork$1 run
INFO: Started Download metadata
Jul 30, 2015 4:06:20 PM hudson.model.AsyncPeriodicWork$1 run
INFO: Finished Download metadata. 16 ms
Jul 30, 2015 4:06:21 PM org.jenkinsci.main.modules.sshd.SSHD start
INFO: Started SSHD at port 46178
Jul 30, 2015 4:06:21 PM jenkins.InitReactorRunner$1 onAttained
INFO: Completed initialization
Jul 30, 2015 4:06:21 PM hudson.WebAppMain$3 run
INFO: Jenkins is fully up and running
Jul 30, 2015 4:09:00 PM com.nirima.jenkins.plugins.docker.DockerCloud provision
INFO: Asked to provision 1 slave(s) for: test
Jul 30, 2015 4:09:00 PM com.nirima.jenkins.plugins.docker.DockerCloud provision
INFO: Will provision 'evarga/jenkins-slave', for label: 'test', in cloud: 'SmallerVPS'
Jul 30, 2015 4:09:11 PM com.nirima.jenkins.plugins.docker.DockerCloud provision
WARNING: Bad template 'evarga/jenkins-slave' in cloud 'SmallerVPS': 'null'. Trying next template...
Jul 30, 2015 4:09:11 PM com.nirima.jenkins.plugins.docker.DockerCloud provision
INFO: Asked to provision 1 slave(s) for: test
Jul 30, 2015 4:09:11 PM com.nirima.jenkins.plugins.docker.DockerCloud provision
INFO: Will provision 'evarga/jenkins-slave', for label: 'test', in cloud: 'SmallerVPS'
Jul 30, 2015 4:09:11 PM com.nirima.jenkins.plugins.docker.DockerCloud provision
WARNING: Bad template 'evarga/jenkins-slave' in cloud 'SmallerVPS': 'null'. Trying next template...
Jul 30, 2015 4:09:20 PM com.nirima.jenkins.plugins.docker.DockerCloud provision
INFO: Asked to provision 1 slave(s) for: test
Jul 30, 2015 4:09:20 PM com.nirima.jenkins.plugins.docker.DockerCloud provision
INFO: Will provision 'evarga/jenkins-slave', for label: 'test', in cloud: 'SmallerVPS'
Jul 30, 2015 4:09:20 PM com.nirima.jenkins.plugins.docker.DockerCloud provision
WARNING: Bad template 'evarga/jenkins-slave' in cloud 'SmallerVPS': 'null'. Trying next template...
Jul 30, 2015 4:09:30 PM com.nirima.jenkins.plugins.docker.DockerCloud provision
INFO: Asked to provision 1 slave(s) for: test
Jul 30, 2015 4:09:30 PM com.nirima.jenkins.plugins.docker.DockerCloud provision
INFO: Will provision 'evarga/jenkins-slave', for label: 'test', in cloud: 'SmallerVPS'
Jul 30, 2015 4:09:30 PM com.nirima.jenkins.plugins.docker.DockerCloud provision
WARNING: Bad template 'evarga/jenkins-slave' in cloud 'SmallerVPS': 'null'. Trying next template...

Unfortunately not a lot of information here!

@KostyaSha
Copy link
Member

And system logger for com.nirima.jenkins.plugins.docker with level ALL :)

@DanielHouston
Copy link

I don't seem to get any more output with level ALL

Jul 30, 2015 4:39:30 PM INFO com.nirima.jenkins.plugins.docker.DockerCloud provision
Asked to provision 1 slave(s) for: test
Jul 30, 2015 4:39:30 PM INFO com.nirima.jenkins.plugins.docker.DockerCloud provision
Will provision 'evarga/jenkins-slave', for label: 'test', in cloud: 'SmallerVPS'
Jul 30, 2015 4:39:30 PM WARNING com.nirima.jenkins.plugins.docker.DockerCloud provision
Bad template 'evarga/jenkins-slave' in cloud 'SmallerVPS': 'null'. Trying next template...
Jul 30, 2015 4:39:40 PM INFO com.nirima.jenkins.plugins.docker.DockerCloud provision
Asked to provision 1 slave(s) for: test
Jul 30, 2015 4:39:40 PM INFO com.nirima.jenkins.plugins.docker.DockerCloud provision
Will provision 'evarga/jenkins-slave', for label: 'test', in cloud: 'SmallerVPS'
Jul 30, 2015 4:39:40 PM WARNING com.nirima.jenkins.plugins.docker.DockerCloud provision
Bad template 'evarga/jenkins-slave' in cloud 'SmallerVPS': 'null'. Trying next template...
Jul 30, 2015 4:39:50 PM INFO com.nirima.jenkins.plugins.docker.DockerCloud provision
Asked to provision 1 slave(s) for: test
Jul 30, 2015 4:39:50 PM INFO com.nirima.jenkins.plugins.docker.DockerCloud provision
Will provision 'evarga/jenkins-slave', for label: 'test', in cloud: 'SmallerVPS'
Jul 30, 2015 4:39:50 PM WARNING com.nirima.jenkins.plugins.docker.DockerCloud provision
Bad template 'evarga/jenkins-slave' in cloud 'SmallerVPS': 'null'. Trying next template...

@KostyaSha
Copy link
Member

Ok, i will verify myself later

@DanielHouston
Copy link

@KostyaSha
Good news. I have found the cause of the problem to be a NPE when adding the first provisioned slave . addProvisionedSlave(DockerTemplate t) will attempt to .get() against provisionedImages before any slaves have been added. Line 463 in DockerCloud.java.
Checking state of provisionedImages and initialing currentProvisioning if the list is empty fixes the error.

More good news, the pull strategy changes also appear to be working :) 👍

@KostyaSha
Copy link
Member

Maybe dirty hack that i removed was related to this NPE, i will check on weekend.

@KostyaSha
Copy link
Member

@DanielHouston thanks, fixed.

@KostyaSha
Copy link
Member

Merged, will be released after @DanielHouston master testing

@KostyaSha KostyaSha added this to the 0.11.0 milestone Aug 2, 2015
@DanielHouston
Copy link

Everything is looking like it is working :)
The only thing I would say is the logging does not reflect the pull strategy
Pulling image 'IMAGE' since one was not found. This may take awhile... is not true when you are pulling to only update latest, or always pulling. There is an image but it is being updated. Similarly, If you use a tag which does not exist in the repository and the pull strategy is to update the image, you still get Pulling image 'IMAGE' since one was not found. This may take awhile... and Finished pulling image 'IMAGE' even though no pulling happens.

@KostyaSha
Copy link
Member

Good catch, i forgot about nice logging

@fraz3alpha
Copy link
Contributor

If you are in there changing the log message, if at all possible, please could you add in a note to say how long the pull took? The total number of seconds (s) or in milliseconds (ms) would be ideal

@KostyaSha
Copy link
Member

ok

@pjdarton pjdarton added RFE A Request for Enhancement. Usually linked to a PR that follows it. and removed enhancement A PR providing an enhancement to existing functionality. labels Mar 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
RFE A Request for Enhancement. Usually linked to a PR that follows it.
Projects
None yet
Development

No branches or pull requests

9 participants