-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] Proxy usage for https broken in 3006.5 #65824
Comments
Minor versions are available https://repo.saltproject.io/salt/py3/ubuntu/22.04/amd64/minor/ |
You can also set an environment variable in the state file as a workaround:
However the bug that the proxy from the minion config isn't used should be fixed. |
Ah this is very helpful, thank you! I guess I didn't realize /latest/ repo is latest packages exclusively. |
As another workaround I set env variables for
Also I've noticed that you shouldn't remove |
I was wondering if someone can test a possible fix for me. before i add this change to the code. on line 364, of the
change that to
I'm wondering if it is just cause the https scheme was left off and requests doesn't know what to use for https proxies. If that works for one of the testers I'll add a quick bug fix for it. |
Nice, I can confirm that this resolves the issue! This is the new minion debug log: 2024-01-12 16:39:29,406 [salt.state :2326][INFO ][110295] Executing state archive.extracted for [/usr/local/bin]
2024-01-12 16:39:29,407 [salt.utils.http :240 ][DEBUG ][110295] Switching to request backend due to the use of proxies.
2024-01-12 16:39:29,408 [salt.utils.http :288 ][DEBUG ][110295] Requesting URL https://get.helm.sh/helm-v3.13.3-linux-amd64.tar.gz.sha256sum using GET method
2024-01-12 16:39:29,408 [salt.utils.http :289 ][DEBUG ][110295] Using backend: requests
2024-01-12 16:39:29,411 [urllib3.connectionpool:1019][DEBUG ][110295] Starting new HTTPS connection (1): get.helm.sh:443
2024-01-12 16:39:30,327 [urllib3.connectionpool:474 ][DEBUG ][110295] https://get.helm.sh:443 "GET /helm-v3.13.3-linux-amd64.tar.gz.sha256sum HTTP/1.1" 200 98
2024-01-12 16:39:30,333 [salt.loaded.int.module.file:4925][DEBUG ][110295] file.extract_hash: Extracting any supported hash for file matching: https://get.helm.sh/helm-v3.13.3-linux-amd64.tar.gz
2024-01-12 16:39:30,335 [salt.loaded.int.module.file:4967][DEBUG ][110295] file.extract_hash: Line 'bbb6e7c6201458b235f335280f35493950dcd856825ddcfd1d3b40ae757d5c7d helm-v3.13.3-linux-amd64.tar.gz' matches source 'https://get.helm.sh/helm-v3.13.3-linux-amd64.tar.gz'
2024-01-12 16:39:30,336 [salt.loaded.int.module.file:5057][DEBUG ][110295] file.extract_hash: Returning sha256 hash 'bbb6e7c6201458b235f335280f35493950dcd856825ddcfd1d3b40ae757d5c7d' as a match of https://get.helm.sh/helm-v3.13.3-linux-amd64.tar.gz
2024-01-12 16:39:30,339 [salt.loader.lazy :977 ][DEBUG ][110295] The functions from module 'file' are being loaded by dir() on the loaded module
2024-01-12 16:39:30,339 [salt.utils.lazy :99 ][DEBUG ][110295] LazyLoaded file.cached
2024-01-12 16:39:30,340 [salt.loaded.int.module.file:4925][DEBUG ][110295] file.extract_hash: Extracting any supported hash for file matching: https://get.helm.sh/helm-v3.13.3-linux-amd64.tar.gz
2024-01-12 16:39:30,341 [salt.loaded.int.module.file:4967][DEBUG ][110295] file.extract_hash: Line 'bbb6e7c6201458b235f335280f35493950dcd856825ddcfd1d3b40ae757d5c7d helm-v3.13.3-linux-amd64.tar.gz' matches source 'https://get.helm.sh/helm-v3.13.3-linux-amd64.tar.gz'
2024-01-12 16:39:30,341 [salt.loaded.int.module.file:5057][DEBUG ][110295] file.extract_hash: Returning sha256 hash 'bbb6e7c6201458b235f335280f35493950dcd856825ddcfd1d3b40ae757d5c7d' as a match of https://get.helm.sh/helm-v3.13.3-linux-amd64.tar.gz
2024-01-12 16:39:30,342 [salt.utils.http :240 ][DEBUG ][110295] Switching to request backend due to the use of proxies.
2024-01-12 16:39:30,343 [salt.utils.http :288 ][DEBUG ][110295] Requesting URL https://get.helm.sh/helm-v3.13.3-linux-amd64.tar.gz using GET method
2024-01-12 16:39:30,343 [salt.utils.http :289 ][DEBUG ][110295] Using backend: requests
2024-01-12 16:39:30,344 [urllib3.connectionpool:1019][DEBUG ][110295] Starting new HTTPS connection (1): get.helm.sh:443
2024-01-12 16:39:30,516 [urllib3.connectionpool:474 ][DEBUG ][110295] https://get.helm.sh:443 "GET /helm-v3.13.3-linux-amd64.tar.gz HTTP/1.1" 200 16188560
2024-01-12 16:39:31,053 [salt.loaded.int.states.archive:1075][DEBUG ][110295] file.cached: {'changes': {'hash': {'old': None, 'new': 'bbb6e7c6201458b235f335280f35493950dcd856825ddcfd1d3b40ae757d5c7d'}}, 'comment': 'File is already cached to /var/cache/salt/minion/extrn_files/base/get.helm.sh/helm-v3.13.3-linux-amd64.tar.gz with hash bbb6e7c6201458b235f335280f35493950dcd856825ddcfd1d3b40ae757d5c7d', 'name': 'https://get.helm.sh/helm-v3.13.3-linux-amd64.tar.gz', 'result': True}
2024-01-12 16:39:31,054 [salt.loaded.int.states.archive:85 ][DEBUG ][110295] Using checksum file /var/cache/salt/minion/archive_hash/extrn_files/base/get.helm.sh/helm-v3.13.3-linux-amd64.tar.gz.hash for cached archive file /var/cache/salt/minion/extrn_files/base/get.helm.sh/helm-v3.13.3-linux-amd64.tar.gz
2024-01-12 16:39:31,057 [salt.loader.lazy :977 ][DEBUG ][110295] The functions from module 'archive' are being loaded by dir() on the loaded module
2024-01-12 16:39:31,057 [salt.utils.lazy :99 ][DEBUG ][110295] LazyLoaded archive.list
2024-01-12 16:39:31,572 [salt.loaded.int.states.archive:1388][DEBUG ][110295] Extracting /var/cache/salt/minion/extrn_files/base/get.helm.sh/helm-v3.13.3-linux-amd64.tar.gz to /usr/local/bin/
2024-01-12 16:39:31,574 [salt.loader.lazy :977 ][DEBUG ][110295] The functions from module 'cmd' are being loaded by dir() on the loaded module
2024-01-12 16:39:31,574 [salt.utils.lazy :99 ][DEBUG ][110295] LazyLoaded cmd.run_all
2024-01-12 16:39:31,575 [salt.loaded.int.module.cmdmod:438 ][INFO ][110295] Executing command tar in directory '/usr/local/bin/'
2024-01-12 16:39:31,962 [salt.loaded.int.module.cmdmod:922 ][DEBUG ][110295] stdout: linux-amd64/helm
2024-01-12 16:39:31,963 [salt.loaded.int.module.cmdmod:438 ][INFO ][110295] Executing command tar in directory '/root'
2024-01-12 16:39:31,966 [salt.loaded.int.module.cmdmod:922 ][DEBUG ][110295] stdout: tar (GNU tar) 1.34
.... etc |
On a side note, there should be some doco pointing to the library doco as NO_PROXY/no_proxy does not have any standards. |
closing since merged. will release with 3006.7 |
Description
Using an archive.extracted / file.managed to pull down a remote file where a proxy is needed to get out stopped working in salt version 3006.5
Setup
/etc/salt/minion.d/f_defaults.conf :
Steps to Reproduce the behavior
e.g. to download the latest version of helm:
Expected behavior
The file(s) should download and extract as expected
Logs
Here is a debug minion log of the failure occurring:
Versions Report
salt --versions-report
(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)Additional context
I noticed that this commit related to this issue was pulled in on 3006.5, although I don't see anything mentioned about proxy/tornado/pycurl in the changelog.
I am definitely in favor of this change - having to install pycurl+tornado was a bit of an annoyance we could do without, but I also need the new method to function for my environment :)
I do happen to have a similarly built Ubuntu 3006.4 minion (with pycurl+tornado) installed, and using the sls example above works perfectly fine there.
My ugly 'workaround' for 3006.5 minions was to
# salt-call pip.install pkgs=tornado,pycurl force_reinstall=true upgrade=true proxy=http://my.local.proxy:3128/
/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/http.py
to the 3006.4 version.As an aside rant, I don't like how the official salt package repos only have the latest minion package present, e.g. I can't simply rollback affected systems to the 3006.4 using apt unless I happen to have it cached somewhere.
The text was updated successfully, but these errors were encountered: