Can't make plugin work over unix socket #1032
Comments
|
Today we started experiencing the same issue running Jenkins 2.426.1 |
|
Same here, I had to switch to a tcp connection on 127.0.0.1:2375 to work around the issue. I also have another connection problem with a remote tcp docker socket secured via SSL (tcp://<remote_ip>:2376) : I don't know if this is the same bug, but the 2 issues appeared simultaneously for me. I can create another issue if necessary. |
|
Maybe this is related to Apache HttpComponents Client 5.x API plugin? We rolled back to version 5.2.1-1.1 and the docker plugin runs as expected. |
|
For the second issue, a workaround is to disable SSL on the remote Docker TCP socket and disable server credentials in Jenkins configuration. But this is insecure... @a-st I will try to rollback as it seems safer, thanks ! |
|
We will need to look into this https://plugins.jenkins.io/apache-httpcomponents-client-5-api compatibility issue to see if we could come up with a more sustainable fix, possibly over the winter holidays. |
Confirmed this works as workaround, thanks for sharing it |
This has probably different root cause, seems to be related to this |
Maintainer of jenkinsci/apache-httpcomponents-client-5-api-plugin here. I can second @rkosegi supposition. |
|
Thanks @strangelookingnerd for the follow-up! |
|
same too |
|
I just released 5.2.3-1.0 of the plugin which includes the latest http client dependency version with the fix. |
|
Just updated to apache-httpcomponents-client-5-api:5.2.3-1.0 but it didn't fix it for me, reverted to 5.2.1-1.1 and it works |
|
Digging a little deeper into the issue it seems like docker-java (which is used by this plugin) is not compatible with the latest httpclient5 versions - see docker-java/docker-java#2262 |
|
I had a slightly different error (https://issues.jenkins.io/browse/JENKINS-72415):
but fortunately latest version ( |
|
Same here, Docker socket works only with apache-httpcomponents-client-5-api |
|
Still dosnt works even on newest version of Log output |
|
Finally fixed in just released version |
|
Tested with |
|
I got same issue on jenkins 2.462.2 with apache-httpcomponents-client-5-api:5.4-118.v199115451c4d So I rolled back apache-httpcomponents-client-5-api's version to 5.3.1-110.v77252fb_d4da_5 and it was fixed!
|
|
Nice one, Is rolling back the version of the apache-httpcomponents-client-5-api's just a work around? How can this be fixed? |
Unfortunately yes. From what I can tell the problem lies in https://github.com/docker-java/docker-java (which is wrapped by this plugin) and it must be fixed there. Previously the error was caused by either a missing dependency during runtime in docker-java as well as an incompatible change in Apache HttpClient5. The latter was rolled back and this fixed the issue at hand. With the new major version of HttpClient5 it seems the issue is re-introduced. |
|
Ah okay, should an issue be created in that repo? |
Yep, It was fixed by simply rolling back to apache-httpcomponents-client-5-api version. |
|
The most recent version of apache-httpcomponents-client-5-api library, api 5.4-118.v199115451c4d , breaks one or more automated tests of the docker plugin. Ubuntu 22.04 and Debian without Docker listening on port 2375 fail tests when testing with apache-httpcomponents-client-5-api 5.4-118.v199115451c4d . The same tests pass if tested with apache-httpcomponents-client-5-api 5.3.1-110.v77252fb_d4da_5 Ubuntu 22.04 with Docker listening on port 2375 passes tests with both versions of apache-httpcomponents-client-5. The issue report that tracks that surprise is: |
|
@MarkEWaite I've just updated, and seems that in not ontly fails the test but effectively can't connect to the socket in a running instance of Jenkins. |
The short term workaround is to downgrade the apache-httpcomponents-client-5-api plugin. The long term solution is for an interested contributor to replace apache-httpcomponents-client-5-api with something like the HTTP libraries that are now included with the JDK. |
Just to clarify, the fix would need to happen in https://github.com/docker-java/docker-java. Unfortunately there has been very little traction in the recent months. So there is not too much hope this is going to happen anytime soon. A rewrite of the docker-plugin to not use docker-java or another implementation of it surely is an option but would require some serious effort. |
|
Thanks for the answers, and I can confirm that pinning the plugin did the job. works. |
|
for me only downgrading to this version works https://updates.jenkins.io/download/plugins/apache-httpcomponents-client-5-api/5.3.1-1.0/apache-httpcomponents-client-5-api.hpi hope it help someone like me spending countless hours trying to debug the jenkins error |
MarkEWaite
changed the title
|
Can anyone provide more information w.r.t. the root cause instead of lots of talk of pinning to old versions? The original root cause identified by @rkosegi has merge requests into Apache HTTPComponents which seem to have dried up in January but the issue remains... |
No, but I replied to a similar request that I'll attempt to investigate more deeply later this week or early next week so that a bug report can be submitted to the upstream repository. If you have capacity to investigate it, I'd love to hear the results of your investigation. |
Reverts 1991154 from jenkinsci#57 jenkinsci/docker-plugin#1032 describes the failure that end users encounter with httpclient 5.4 when connecting to the Docker daemon over a Unix domain socket. jenkinsci/docker-plugin#1103 describes the test automation failure that happens with httpclient 5.4. jenkinsci#29 notes that there may have been a breaking change in httpclient 5.4. jenkinsci#59 also notes that there is an issue in httpclient 5.4.
|
A new 5.4-124.v31e2987e48f4 release of Apache httpcomponents client 5 API is available that switches the library back to httpclient 5.3.1 while retaining the version number "5.4" in the version string. |
Jenkins and plugins versions report
Environment
What Operating System are you using (both controller, and any agents involved in the problem)?
OS: Ubuntu LTS
Docker: 20.10.7
Reproduction steps
run jenkins controller using docker-compose:
Configure docker-local cloud using script from this repo
Expected Results
Docker plugin to connect to docker daemon
Actual Results
Error message that plugin can't connect:
Connect to unix://localhost:2375 [localhost/127.0.0.1] failed: Connection refused (Connection refused)Anything else?
Here are detailed logs
I tried to connection from jenkins master to socket using curl, no issue:
I was also able to attach
straceand trace down actual attempt for connection:Clearly, there should be AF_UNIX, but there is AF_INET.
Are you interested in contributing a fix?
Sure, happy to help if you give me pointers.
The text was updated successfully, but these errors were encountered: