Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error: curl_easy_setopt: 48 #149

Closed
SamuelBadr opened this issue Sep 26, 2021 · 38 comments
Closed

Error: curl_easy_setopt: 48 #149

SamuelBadr opened this issue Sep 26, 2021 · 38 comments

Comments

@SamuelBadr
Copy link

Hi,
I posted this problem on Slack originally and was asked to post here as well after we failed to find a fix.

Every time I upgrade my Julia packages, Pkg.update() gives me this same error multiple times:

┌ Error: curl_easy_setopt: 48
└ @ Downloads.Curl /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36

Note that #123 concerns the same problem, but seeing as I'm using an official build of Julia, the discussion there doesn't help I think.

julia> versioninfo()
Julia Version 1.6.3
Commit ae8452a9e0 (2021-09-23 17:34 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin19.5.0)
  CPU: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, skylake)
Environment:
  JULIA_NUM_THREADS = 12

This doesn't affect functionality AFAICT, has been happening for a couple of weeks now and persists even when I try with a nightly build, for example.
LD_LIBRARY_PATH , DYLD_FALLBACK_LIBRARY_PATH and DYLD_LIBRARY_PATH all aren't set.

@StefanKarpinski
Copy link
Member

Does it still happen if you disable multithreading?

@SamuelBadr
Copy link
Author

Screenshot 2021-09-28 at 20 13 33

Yep, still happens.

@StefanKarpinski
Copy link
Member

Ok, well, I guess it's good that's not the problem, but that means I need to think of what the problem might be. Did this problem happen with Julia 1.6.2 as well?

@SamuelBadr
Copy link
Author

Yes; it's been happening for a while now. Can't say for how long exactly, but certainly with 1.6.2 and even before that.

@StefanKarpinski
Copy link
Member

Can you run this in your shell?:

DYLD_PRINT_LIBRARIES=YES julia -e0 2>&1 | grep curl

For me it outputs this:

dyld: loaded: <A7AB4DA2-C48F-3AAD-8418-558F37097C08> /Users/stefan/dev/julia/usr/lib/libcurl.4.dylib

That's for my source build.

@SamuelBadr
Copy link
Author

That gives

dyld[88800]: <F686F2FD-558C-3BD7-B499-49676DC0520D> /usr/lib/libcurl.4.dylib

for me. I'm guessing that means julia uses the system curl instead of the one it ships with or something?

@garrekstemo
Copy link

I'll add a data point. I found this open issue when trying to solve the same problem. It seems other people have had the same issue but I never saw a fix other than maybe installing an updated version of curl alongside the system curl? I always see this error when updating packages, ever since I first installed Julia at version 1.6.1 a few months ago.

(@v1.6) pkg> up
    Updating registry at `~/.julia/registries/General`
┌ Error: curl_easy_setopt: 48
└ @ Downloads.Curl /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_setopt: 48
└ @ Downloads.Curl /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_setopt: 48
└ @ Downloads.Curl /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
  No Changes to `~/.julia/environments/v1.6/Project.toml`
  No Changes to `~/.julia/environments/v1.6/Manifest.toml`

Current config:

versioninfo()
Julia Version 1.6.3
Commit ae8452a9e0 (2021-09-23 17:34 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin19.5.0)
  CPU: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, skylake)

Output from

DYLD_PRINT_LIBRARIES=YES julia -e0 2>&1 | grep curl

gives

dyld[11354]: <434491C5-7B0A-310F-A2F3-BB83685A5D91> /usr/lib/libcurl.4.dylib

@StefanKarpinski
Copy link
Member

StefanKarpinski commented Sep 30, 2021

I'm guessing that means julia uses the system curl instead of the one it ships with or something?

Yep, that's what it means. The question is why? It shouldn't, afaiu, be doing that. cc @staticfloat

@staticfloat
Copy link
Member

First off, where did your version of julia come from? Is it downloaded from the official website, or coming from somewhere else?

Download the 1.6.3 dmg, extract the .app, then run the following:

using Libdl; filter(x -> occursin("curl", x), dllist())

And show me what it says.

@SamuelBadr
Copy link
Author

I install julia via homebrew. I already tried downloading from the website directly and the problem persisted.

Following your instructions (with the binary from the website), I get

julia> using Libdl; filter(x -> occursin("curl", x), dllist())
1-element Vector{String}:
 "/usr/lib/libcurl.4.dylib"

@staticfloat
Copy link
Member

staticfloat commented Oct 1, 2021

Can you give the full output of DYLD_PRINT_LIBRARIES=YES julia -e0? I want to see what other libraries are being loaded. And I'm interested in seeing what the official binary gives, rather than the one from Homebrew, to reduce variation first. If we can fix the issue for the official binary, we can see if there's anything different for the Homebrew build.

@SamuelBadr
Copy link
Author

❯ DYLD_PRINT_LIBRARIES=YES julia -e0                  
dyld[27147]: <D1BCC17C-FF7B-39EE-AF8E-B44ACD941BF6> /Applications/Julia-1.6.app/Contents/Resources/julia/bin/julia
dyld[27147]: <7898EFC9-E6A2-30F6-883E-704E81E5FBA3> /Applications/Julia-1.6.app/Contents/Resources/julia/lib/libjulia.1.6.dylib
dyld[27147]: <608D2B1D-0B2B-3F59-8681-005985E821EB> /usr/lib/libSystem.B.dylib
dyld[27147]: <A8ED3268-FE9C-3D3B-BE33-DECEE62D359D> /usr/lib/system/libcache.dylib
dyld[27147]: <443AF62A-1CD8-3F47-9FC7-AAA24BF94EC1> /usr/lib/system/libcommonCrypto.dylib
dyld[27147]: <55BA195A-2E71-38DB-9410-7E9DEFB39AFC> /usr/lib/system/libcompiler_rt.dylib
dyld[27147]: <EE1F7E55-8DE5-3A5A-A8AF-FC515AB05538> /usr/lib/system/libcopyfile.dylib
dyld[27147]: <30B7108B-5997-3F38-B3AB-95191FBACCC8> /usr/lib/system/libcorecrypto.dylib
dyld[27147]: <E717D3AE-5B16-3930-A0D6-CC23430FBF58> /usr/lib/system/libdispatch.dylib
dyld[27147]: <E421B2A3-0EF9-328C-81CF-A42C00DD88E6> /usr/lib/system/libdyld.dylib
dyld[27147]: <0B7AEE83-CEC8-3097-BAD1-D05C92CDEFCA> /usr/lib/system/libkeymgr.dylib
dyld[27147]: <CD147AF0-AFB3-3B34-BAF5-DA72BAC45366> /usr/lib/system/libmacho.dylib
dyld[27147]: <5CB9C11E-7765-364D-9531-BB019D43A393> /usr/lib/system/libquarantine.dylib
dyld[27147]: <2262201A-6C87-36A1-9ECD-7375B958E8FD> /usr/lib/system/libremovefile.dylib
dyld[27147]: <AC8244E6-8B16-3874-9638-45AB64494645> /usr/lib/system/libsystem_asl.dylib
dyld[27147]: <EC05973E-9A9E-370E-967E-BA4105BBBAEF> /usr/lib/system/libsystem_blocks.dylib
dyld[27147]: <462299CC-B4DB-3062-A818-E9ECC5C7188B> /usr/lib/system/libsystem_c.dylib
dyld[27147]: <E759FE08-A62F-3749-A7FE-1D6BCD31A888> /usr/lib/system/libsystem_collections.dylib
dyld[27147]: <2A52329B-5D32-334A-A5ED-874957332F39> /usr/lib/system/libsystem_configuration.dylib
dyld[27147]: <1660C6CA-5015-34C3-9C7C-85370206B49C> /usr/lib/system/libsystem_containermanager.dylib
dyld[27147]: <C3A73CA8-91A7-3E1A-B1B3-215FEF292EAC> /usr/lib/system/libsystem_coreservices.dylib
dyld[27147]: <1C5A1E21-ACA3-3ACC-8B4B-813EE87EB8F7> /usr/lib/system/libsystem_darwin.dylib
dyld[27147]: <42D2466E-01E0-3B4C-8518-8E7CF25DEA37> /usr/lib/system/libsystem_dnssd.dylib
dyld[27147]: <B29AFAD9-A4DB-353E-A867-CCE5F0BB6152> /usr/lib/system/libsystem_featureflags.dylib
dyld[27147]: <27CD98A9-C4E8-38A4-BB59-D67566FABDD3> /usr/lib/system/libsystem_info.dylib
dyld[27147]: <AF175E64-9F85-3480-8E37-6A293EAEBFAD> /usr/lib/system/libsystem_m.dylib
dyld[27147]: <3003E822-237A-336C-87D3-83F5808E99FE> /usr/lib/system/libsystem_malloc.dylib
dyld[27147]: <4367A754-5F37-3D36-905B-3250B82A14B6> /usr/lib/system/libsystem_networkextension.dylib
dyld[27147]: <1B7E107F-22F2-3F69-B4CB-1FACDDFEB8E6> /usr/lib/system/libsystem_notify.dylib
dyld[27147]: <4D661D92-4F8C-318A-826C-41CC869E53C5> /usr/lib/system/libsystem_product_info_filter.dylib
dyld[27147]: <8BC002E2-0EDF-33B5-AA5A-CAE183CBEA3D> /usr/lib/system/libsystem_sandbox.dylib
dyld[27147]: <D311D067-FE49-34A3-84CF-E7A88C4FFECA> /usr/lib/system/libsystem_secinit.dylib
dyld[27147]: <3ED03EE9-7E46-3A2D-808E-319FCBAF8AF8> /usr/lib/system/libsystem_kernel.dylib
dyld[27147]: <07B2F4DF-E55E-3DC5-865B-1A241C10AC07> /usr/lib/system/libsystem_platform.dylib
dyld[27147]: <45263E93-2DC0-32BA-9BB7-C3FBD34D7195> /usr/lib/system/libsystem_pthread.dylib
dyld[27147]: <BC7587F5-5C4F-3D24-BF25-C830B3009F06> /usr/lib/system/libsystem_symptoms.dylib
dyld[27147]: <D6BB24EC-3187-33DB-9A7D-5A987C07B509> /usr/lib/system/libsystem_trace.dylib
dyld[27147]: <4C22187A-F7F4-3BAA-AF99-F876D35A6CEF> /usr/lib/system/libunwind.dylib
dyld[27147]: <BBA2F5CC-63EC-389E-8343-4058CA0F2C7F> /usr/lib/system/libxpc.dylib
dyld[27147]: <A5458C2B-7912-3C3C-83DF-F8F3EFE9B9AA> /usr/lib/libc++abi.dylib
dyld[27147]: <00864E0C-0325-380E-BF76-B3FC47F3D5E8> /usr/lib/libobjc.A.dylib
dyld[27147]: <14760BC8-DDEA-38E7-B420-E934D01C7E08> /usr/lib/liboah.dylib
dyld[27147]: <127C425D-A82E-31C1-912D-DC1DFA025205> /usr/lib/libc++.1.dylib
dyld[27147]: <22A1E58B-8C2B-3164-8BF4-2B3E99C9B921> /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libgcc_s.1.dylib
dyld[27147]: <BA8F7A9B-1520-33EB-B1B7-26F7526EF650> /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libopenlibm.3.0.dylib
dyld[27147]: <66C18380-089C-371E-8DF3-F1ADDA67C576> /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib
dyld[27147]: <1F91D7B1-E37D-36EA-8ABE-4F92605A5A2D> /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libosxunwind.dylib
dyld[27147]: <7A309D42-BF15-38EE-96FD-A53A2749AFC4> /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libLLVM.dylib
dyld[27147]: <DF7DC92F-54BB-30F8-A020-589EF156EA06> /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
dyld[27147]: <5C9F3626-D042-3619-9532-434D927A20F3> /usr/lib/libfakelink.dylib
dyld[27147]: <02DEEF55-4D1D-3173-8F32-40C83A54DD33> /usr/lib/libicucore.A.dylib
dyld[27147]: <ECE8CC94-234D-315B-B4E1-5EADAFA36B09> /System/Library/PrivateFrameworks/SoftLinking.framework/Versions/A/SoftLinking
dyld[27147]: <B14C4124-35A3-3D65-B568-B561DA395510> /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/sys.dylib
dyld[27147]: <F1074014-32B3-3ED7-A6FD-3F4861503BC3> /usr/local/Cellar/pcre2/10.37_1/lib/libpcre2-8.0.dylib
dyld[27147]: <185B6429-AE7F-3DAB-841C-B1C2C1736C6B> /usr/local/Cellar/gmp/6.2.1_1/lib/libgmp.10.dylib
dyld[27147]: <055F5288-E073-3956-8745-8F15B985F57F> /usr/local/Cellar/mpfr/4.1.0/lib/libmpfr.6.dylib
dyld[27147]: <922830A3-B725-36B4-92B1-B73EE7A97BB1> /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libopenblas64_.0.3.10.dylib
dyld[27147]: <98C960E1-0AF7-37DE-A082-EFE038E35F9D> /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libgfortran.4.dylib
dyld[27147]: <06D90A5F-CAF4-3383-BAA8-E94BAFE99CBE> /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libquadmath.0.dylib
dyld[27147]: <BA618A2B-3CDD-3299-AC72-D53DC66FBE36> /usr/local/Cellar/suite-sparse/5.10.1/lib/libcholmod.3.0.14.dylib
dyld[27147]: <9A3C1A49-212B-3343-A54F-BC10D8DB6FA0> /usr/local/Cellar/suite-sparse/5.10.1/lib/libamd.2.4.6.dylib
dyld[27147]: <55C8A7D7-CAF3-3331-97B1-ABCBDA9BF814> /usr/local/Cellar/suite-sparse/5.10.1/lib/libcolamd.2.9.6.dylib
dyld[27147]: <218A4D41-CEB6-3ABF-B0A0-EE55FD60B25B> /usr/local/Cellar/suite-sparse/5.10.1/lib/libsuitesparseconfig.5.10.1.dylib
dyld[27147]: <FC27F115-CC00-3A9F-AEC3-450B5FF196F5> /usr/local/Cellar/suite-sparse/5.10.1/lib/libccolamd.2.9.6.dylib
dyld[27147]: <2033AB89-6EAC-3010-87D7-599E5C807077> /usr/local/Cellar/suite-sparse/5.10.1/lib/libcamd.2.4.6.dylib
dyld[27147]: <6391B682-8290-3E29-9973-95AB058E9F31> /usr/local/Cellar/metis/5.1.0/lib/libmetis.dylib
dyld[27147]: <E8AD73B3-559A-3C36-88DA-E50F7C364C2B> /usr/local/Cellar/openblas/0.3.17/lib/libopenblasp-r0.3.17.dylib
dyld[27147]: <02B8FEE7-9679-3C78-BEEB-0B249A341085> /usr/local/Cellar/gcc/11.2.0/lib/gcc/11/libgfortran.5.dylib
dyld[27147]: <C02E7D2E-0A76-35A5-B5D0-3799A3C7B644> /usr/local/Cellar/gcc/11.2.0/lib/gcc/11/libgomp.1.dylib
dyld[27147]: <18445726-01A0-3EC8-9497-F0E83C0EF7F8> /usr/local/Cellar/gcc/11.2.0/lib/gcc/11/libquadmath.0.dylib
dyld[27147]: <F5198FEF-6E4A-3789-9690-29212BD54C2A> /usr/local/Cellar/gcc/11.2.0/lib/gcc/11/libgcc_s.1.dylib
dyld[27147]: <113CA6DC-1E0F-310F-B50C-56DAFAC7F47C> /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libsuitesparse_wrapper.dylib
dyld[27147]: <1C6E76AE-6BAB-3154-A79F-E5E3CB2BCBC3> /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libcholmod.3.0.13.dylib
dyld[27147]: <67C92F27-AF28-3382-A044-9682A9BAA9D7> /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libamd.2.4.6.dylib
dyld[27147]: <A8216989-685C-3C62-B67B-65E9051DFADC> /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libcolamd.2.9.6.dylib
dyld[27147]: <36905660-DB31-3DA8-ABBB-4AAB301DF701> /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libsuitesparseconfig.5.4.0.dylib
dyld[27147]: <60C1F719-4A77-37F7-875B-1AF5A72028AD> /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libccolamd.2.9.6.dylib
dyld[27147]: <136087CE-4D31-3BE2-A01B-308A94C77EB2> /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libcamd.2.4.6.dylib
dyld[27147]: <A9777302-4DA7-3FF9-9095-56CC58ADA217> /usr/local/Cellar/nghttp2/1.45.1/lib/libnghttp2.14.dylib
dyld[27147]: <434491C5-7B0A-310F-A2F3-BB83685A5D91> /usr/lib/libcurl.4.dylib
dyld[27147]: <C54311B0-C44A-33CB-A718-8D027A9EF210> /System/Library/Frameworks/Security.framework/Versions/A/Security
dyld[27147]: <C4CE6D06-58B2-30B7-AF02-19EF750D6283> /usr/lib/libcrypto.44.dylib
dyld[27147]: <40150C6A-7279-3B2D-B73E-4F0A663132D1> /usr/lib/libssl.46.dylib
dyld[27147]: <08F0DD1C-D924-38F4-9FD7-BC9BC3BFF075> /System/Library/Frameworks/LDAP.framework/Versions/A/LDAP
dyld[27147]: <3CEA27FB-E468-30CD-97DA-DCE706180F7A> /System/Library/Frameworks/Kerberos.framework/Versions/A/Kerberos
dyld[27147]: <13550210-DAFF-3BF2-A7CF-48C44D10DB9C> /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration
dyld[27147]: <44D1858E-9FB3-3D95-89C2-928C0CBD5A83> /usr/lib/libapple_nghttp2.dylib
dyld[27147]: <B5B8D804-38D9-397A-A779-200B87FF7561> /usr/lib/libz.1.dylib
dyld[27147]: <D5C23BD9-932A-3EBC-9852-A938615CABE3> /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
dyld[27147]: <3D67151D-3FF1-314B-B45A-200627F0A2DE> /usr/lib/libCoreEntitlements.dylib
dyld[27147]: <52ADB158-024A-3841-8E0D-7BA641879E48> /usr/lib/libMobileGestalt.dylib
dyld[27147]: <194B78BB-6EC3-3FDB-BDF2-9D1BA8A03884> /System/Library/PrivateFrameworks/AppleFSCompression.framework/Versions/A/AppleFSCompression
dyld[27147]: <22252217-63BF-3448-93CD-0A259E561AFD> /usr/lib/libDiagnosticMessagesClient.dylib
dyld[27147]: <B6EAC2DE-8501-3174-A7B7-7821D6B7DF12> /usr/lib/libbsm.0.dylib
dyld[27147]: <C35B0EA9-9226-3F55-8BAD-9EF15DF49F93> /usr/lib/libcoretls.dylib
dyld[27147]: <04D4EDDE-2AD4-3E1A-8BF8-81667EB1CC4C> /usr/lib/libcoretls_cfhelpers.dylib
dyld[27147]: <6BC899CE-CBE1-3556-B4C0-5044549E10FE> /usr/lib/libpam.2.dylib
dyld[27147]: <37C75CF1-3B74-362A-85D3-F2397FA98988> /usr/lib/libsqlite3.dylib
dyld[27147]: <F2265D95-5C96-3252-852A-B60352C205EA> /usr/lib/libxar.1.dylib
dyld[27147]: <5079012D-0EDA-352E-BF39-F665661F08F3> /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
dyld[27147]: <CE7A925D-EA04-32BC-ACF3-9D40120150D3> /System/Library/PrivateFrameworks/CoreAutoLayout.framework/Versions/A/CoreAutoLayout
dyld[27147]: <BAEB22B2-B467-38A8-9067-2B82A7A2DD2D> /usr/lib/libcompression.dylib
dyld[27147]: <1F24BBA2-5C79-303B-A429-ADF046318EC3> /System/Library/Frameworks/CFNetwork.framework/Versions/A/CFNetwork
dyld[27147]: <E8BAED43-F2A9-3E8C-BC30-19F0CA518D66> /System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration
dyld[27147]: <2CA7C053-6DA9-33BD-9B64-11220138B519> /usr/lib/libarchive.2.dylib
dyld[27147]: <F1E3AA22-66F3-31AF-BAA6-DA14283423D5> /usr/lib/libxml2.2.dylib
dyld[27147]: <11119F78-F2C6-38A9-ABAF-5601AD89FB33> /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices
dyld[27147]: <0E1EF7F9-18B9-3270-8745-56735FFC2644> /usr/lib/liblangid.dylib
dyld[27147]: <3554FFC5-E10B-368B-AD2E-E23A7A61DE2A> /usr/lib/libnetwork.dylib
dyld[27147]: <587EDCBA-0DD4-3F5B-AFA5-56793EFB43CA> /usr/lib/libenergytrace.dylib
dyld[27147]: <80715B45-8203-3BA4-AD63-95340216121A> /usr/lib/system/libkxld.dylib
dyld[27147]: <2D34B5C2-C34A-35A2-81A5-F7D5F3398215> /usr/lib/libpcap.A.dylib
dyld[27147]: <4E86CE35-67B7-3B6D-8006-E6705BEDD809> /usr/lib/libdns_services.dylib
dyld[27147]: <5A8BDDAE-D2A1-3755-8E81-CE24C72722A0> /System/Library/PrivateFrameworks/AppleSystemInfo.framework/Versions/A/AppleSystemInfo
dyld[27147]: <62B063B7-A477-3AA3-B4EE-09D68C64C81B> /System/Library/PrivateFrameworks/IOMobileFramebuffer.framework/Versions/A/IOMobileFramebuffer
dyld[27147]: <626B2762-3514-3C67-B588-7821F473A9A1> /System/Library/Frameworks/IOSurface.framework/Versions/A/IOSurface
dyld[27147]: <C1412D94-CF33-32A1-A519-015B9584BC84> /usr/lib/liblzma.5.dylib
dyld[27147]: <D7B09714-910E-3B1F-AF4E-6DEFB97DE959> /usr/lib/libbz2.1.0.dylib
dyld[27147]: <DBCED5A3-2177-3DC6-873D-1F6D9472EC0F> /usr/lib/libiconv.2.dylib
dyld[27147]: <C26BDB31-5D57-3825-B35A-C3A82957601B> /usr/lib/libcharset.1.dylib
dyld[27147]: <321AD230-E9E8-3090-A692-009537188F63> /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/FSEvents.framework/Versions/A/FSEvents
dyld[27147]: <64A88E5D-A6AC-382B-9891-8686E4D377C3> /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/CarbonCore.framework/Versions/A/CarbonCore
dyld[27147]: <F17653C2-17EE-3395-AE24-AED20AC8FA4F> /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Metadata.framework/Versions/A/Metadata
dyld[27147]: <C4FC18D4-2420-36B0-A714-965DA6409E86> /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/OSServices.framework/Versions/A/OSServices
dyld[27147]: <8036408C-B644-3A0F-91F1-273AB8EC5C49> /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/SearchKit.framework/Versions/A/SearchKit
dyld[27147]: <EB019DC7-1CB9-32EE-B328-91BD7C17409C> /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/AE.framework/Versions/A/AE
dyld[27147]: <B9322264-05DC-3B16-8685-297DA18BCEC7> /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/LaunchServices
dyld[27147]: <37DD3765-3888-37A3-8029-E36AA8191662> /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/DictionaryServices.framework/Versions/A/DictionaryServices
dyld[27147]: <D607A525-2054-3C20-A6AD-B035B5DC7B3A> /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/SharedFileList.framework/Versions/A/SharedFileList
dyld[27147]: <34BF5231-C262-3CFE-9F24-C83C355FB486> /usr/lib/libCheckFix.dylib
dyld[27147]: <CF47D293-0144-37FC-8FD2-3A003BACF2C3> /System/Library/PrivateFrameworks/TCC.framework/Versions/A/TCC
dyld[27147]: <E1B37999-1FDA-380B-A8D5-11AC12399F60> /System/Library/PrivateFrameworks/CoreNLP.framework/Versions/A/CoreNLP
dyld[27147]: <D60A1901-29B7-3B55-A58B-D485051383F7> /System/Library/PrivateFrameworks/MetadataUtilities.framework/Versions/A/MetadataUtilities
dyld[27147]: <4AB7D2B0-9760-312D-A897-2C886F121BFB> /usr/lib/libmecabra.dylib
dyld[27147]: <D9BE0838-CF97-343F-8B18-309B5009068A> /System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate
dyld[27147]: <D5A76B92-8A58-32E9-BD22-AAB360687D4B> /usr/lib/libmecab.dylib
dyld[27147]: <7131C99C-2AB6-34C9-986B-10C7D58F2831> /usr/lib/libCRFSuite.dylib
dyld[27147]: <12572967-1280-3AE7-AC59-44F0E81995C3> /usr/lib/libgermantok.dylib
dyld[27147]: <6A323FB8-47AE-3ED1-ACDC-C9A10C5D7026> /usr/lib/libThaiTokenizer.dylib
dyld[27147]: <69788314-8590-3E56-A338-B70A97E93682> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vImage.framework/Versions/A/vImage
dyld[27147]: <F61511AF-88C1-343E-9F0E-B8DC65707AB4> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/vecLib
dyld[27147]: <5AD96CE7-E97D-3190-BEB3-D838C6165630> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libvMisc.dylib
dyld[27147]: <E69B522B-3900-3946-A755-4F2D34CF1ADB> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libvDSP.dylib
dyld[27147]: <AA8E2E0A-A2AF-357A-B249-2DCA2015C4B5> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
dyld[27147]: <ED366064-919E-3F26-934E-7DB941A8562E> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libLAPACK.dylib
dyld[27147]: <B60ACD1E-7796-3B7E-A434-F0C9BC5FBF35> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libLinearAlgebra.dylib
dyld[27147]: <E36EEFDC-669B-3369-AE42-BA6994458402> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libSparseBLAS.dylib
dyld[27147]: <BF7D3C8E-8BB9-32C3-9EC7-D89F5239A79E> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libQuadrature.dylib
dyld[27147]: <5F4D7CD1-6DCD-360A-8B3A-337ADEDF35F3> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBNNS.dylib
dyld[27147]: <350343A1-177E-3793-9CD9-4914FF8F91AC> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libSparse.dylib
dyld[27147]: <F7AD202F-E0F6-3AE2-BD6E-6B795763F0F6> /System/Library/PrivateFrameworks/CoreAnalytics.framework/Versions/A/CoreAnalytics
dyld[27147]: <77A859C6-4354-354B-9224-03092C70A090> /System/Library/PrivateFrameworks/LinguisticData.framework/Versions/A/LinguisticData
dyld[27147]: <323109F1-CCDA-3743-B339-983BCC9745AD> /System/Library/PrivateFrameworks/Lexicon.framework/Versions/A/Lexicon
dyld[27147]: <92921AC2-7B7B-3548-8095-69BCF74E3D8B> /usr/lib/libChineseTokenizer.dylib
dyld[27147]: <C4DD0E52-9304-32E8-ADE2-A5DA67B7DFA5> /System/Library/PrivateFrameworks/LanguageModeling.framework/Versions/A/LanguageModeling
dyld[27147]: <F7E72ACF-EF6C-3396-87FA-31FF33C889A6> /System/Library/PrivateFrameworks/AppleSauce.framework/Versions/A/AppleSauce
dyld[27147]: <260A59C8-A4C8-3D28-A2F5-57C383467148> /System/Library/PrivateFrameworks/CoreEmoji.framework/Versions/A/CoreEmoji
dyld[27147]: <880BDE60-2403-3650-924A-6079397D8E0A> /usr/lib/libcmph.dylib
dyld[27147]: <EB234FF7-F4AC-3652-9CFA-8D625300C88B> /System/Library/Frameworks/OpenDirectory.framework/Versions/A/Frameworks/CFOpenDirectory.framework/Versions/A/CFOpenDirectory
dyld[27147]: <B8FDB5F6-CADB-3B5B-9D65-93BD045C80C2> /System/Library/Frameworks/OpenDirectory.framework/Versions/A/OpenDirectory
dyld[27147]: <9887D3D0-7D3B-34B0-AC34-01BBDBAA3A00> /System/Library/PrivateFrameworks/APFS.framework/Versions/A/APFS
dyld[27147]: <0BE8FDD4-F312-3C84-B876-81D70FF1DA74> /System/Library/Frameworks/SecurityFoundation.framework/Versions/A/SecurityFoundation
dyld[27147]: <7CD92BCC-7C35-3130-B185-0E064501B557> /usr/lib/libutil.dylib
dyld[27147]: <9B0CBDA4-22B6-33AF-98A1-35E3F252ACD2> /System/Library/PrivateFrameworks/CoreServicesStore.framework/Versions/A/CoreServicesStore
dyld[27147]: <54AC1BD9-A382-3682-BBAB-DCA8D4564E85> /usr/lib/libapp_launch_measurement.dylib
dyld[27147]: <6FB1A824-63F1-3647-B9FC-FEE2A1E5E972> /usr/lib/libxslt.1.dylib
dyld[27147]: <3BFAAD98-AEC6-3967-8DDC-82392AD9DBEB> /System/Library/PrivateFrameworks/BackgroundTaskManagement.framework/Versions/A/BackgroundTaskManagement
dyld[27147]: <943DB52C-BB92-321C-ABED-1FDCF10F04A6> /System/Library/PrivateFrameworks/TrustEvaluationAgent.framework/Versions/A/TrustEvaluationAgent
dyld[27147]: <372E70AB-4638-3142-8573-51030C407FEC> /usr/lib/libsasl2.2.dylib
dyld[27147]: <5673C389-EF04-3261-B1B5-DD807FD3120A> /System/Library/PrivateFrameworks/CoreDaemon.framework/Versions/B/CoreDaemon
dyld[27147]: <CF4CB6CC-35B7-3068-80F9-05D236876B2E> /usr/lib/libresolv.9.dylib
dyld[27147]: <7953C3D4-2EF9-39E6-BEFA-3FBF2FC54AF0> /System/Library/PrivateFrameworks/AppleSRP.framework/Versions/A/AppleSRP
dyld[27147]: <DC3B33A4-148B-3644-B3E2-4FCA4D52A677> /System/Library/Frameworks/ServiceManagement.framework/Versions/A/ServiceManagement
dyld[27147]: <E33CD57D-E25F-3A4B-BBD8-D6968FEAA999> /System/Library/PrivateFrameworks/Heimdal.framework/Versions/A/Heimdal
dyld[27147]: <15BD12AF-CC3D-3A74-B95E-C46A456104CF> /System/Library/Frameworks/GSS.framework/Versions/A/GSS
dyld[27147]: <DCB97E74-5899-3FAF-AFE3-FCE8FC8CAFEB> /System/Library/Frameworks/Kerberos.framework/Versions/A/Libraries/libHeimdalProxy.dylib
dyld[27147]: <5F8B2C75-7552-32E0-A3FD-07A1D890226B> /System/Library/Frameworks/Network.framework/Versions/A/Network
dyld[27147]: <782A05BF-6913-3905-AC7F-5B4D1E4E52C8> /usr/lib/libheimdal-asn1.dylib
dyld[27147]: <23846074-7978-3723-A2BE-00D82E630709> /System/Library/PrivateFrameworks/ProtocolBuffer.framework/Versions/A/ProtocolBuffer
dyld[27147]: <C6724074-22D7-31C9-B686-B7A76AAAB381> /System/Library/PrivateFrameworks/CommonAuth.framework/Versions/A/CommonAuth

I was curious and compared the checksums. Turns out, the official binary and the homebrew one are the same.

@staticfloat
Copy link
Member

What version of macOS are you on?

I see that libraries like libnghttp2 are being picked out of /usr/local/lib, but the way we try and load that file should be dlopen("@rpath/libnghttp2.14.dylib").

The @rpath/ rules should take precedence over searching standard library locations, so that leaves two possibilities:

  • The first is that the RPATH settings are screwed up. This is unlikely, as the binaries are signed, and so if they were modified, they wouldn't run on your system. Can you please run otool -l /path/to/libjulia-internal.dylib, and ensure that there is a section that looks like this:
Load command 17
          cmd LC_RPATH
      cmdsize 32
         path @loader_path/julia/ (offset 12)
Load command 18
          cmd LC_RPATH
      cmdsize 32
         path @loader_path/ (offset 12)
  • The second is that something like DYLD_LIBRARY_PATH is indeed set. Can you please double-check that, from within Julia, running ENV["DYLD_LIBRARY_PATH"] and ENV["DYLD_FALLBACK_LIBRARY_PATH"] both error?

@SamuelBadr
Copy link
Author

I'm on the developer beta of macOS 12 Monterey (21A5534d), which is expected to release publicly in the next couple of weeks.

  • My command 18 doesn't look like yours:
Load command 17
          cmd LC_RPATH
      cmdsize 32
         path @loader_path/ (offset 12)
Load command 18
      cmd LC_FUNCTION_STARTS
  cmdsize 16
  dataoff 2793752
 datasize 9624

(The full output is at https://gist.github.com/Samuel3008/42ba40f37973350ad10e89695c69ad61 in case it helps)

  • I just checked and ENV["DYLD_LIBRARY_PATH"] and ENV["DYLD_FALLBACK_LIBRARY_PATH"] do indeed error.

@staticfloat
Copy link
Member

The command numbers aren't as important as the fact that we have both @loader_path/ and @loader_path/julia in there, which we do.

I'm out of ideas; it appears that macOS is just loading things in a different order, which would really surprise me, but is possible I suppose. Once macOS 12 final is released, I'll see if I can reproduce.

@SamuelBadr
Copy link
Author

Oh that's a bummer. Regardless, thank you so much for your time and help!

@kellertuer
Copy link

Is there a fix available somehow? I ran into the same problem on the most recent version of Monterey just now.

staticfloat added a commit to JuliaLang/julia that referenced this issue Oct 20, 2021
Many thanks to Randy Rucker from Apple for pointing out that we were
technically relying on undefined behavior in `dyld` for loading things
via `jl_dlopen("@loader_path/@rpath/libfoo.dylib")`, due to the
composition of `dlopen("@rpath/libfoo.dylib")` in our Julia code, and
the `DL_LOAD_PATH` prepending we do in `jl_load_dynamic_library()`.

This PR uses a slightly modified version of a patch emailed to me by
Randy, and should solve
JuliaLang/Downloads.jl#149 on macOS Monterey
where the undefined behavior in `dyld` has changed.
@staticfloat
Copy link
Member

I reached out to Apple and they provided some guidance on what could be the issue: can anyone affected by this please try either building this branch, or running this binary (x86_64 only), and seeing if it resolves the issue?

@SamuelBadr
Copy link
Author

Ok so for registered packages it solves the problem and works perfectly! But I just noticed unregistered packages don't work (and I get the same error on 1.6.3 so nothing to do with your patch but maybe with the bug?).

(@v1.8) pkg> add https://github.com/JuliaLang/Example.jl
ERROR: GitError(Code:ENOTFOUND, Class:Repository, could not find repository from '/Users/samuel/.julia/clones/15037770950778237535')
Stacktrace:
  [1] macro expansion
    @ ~/Downloads/julia-27cb0c43e1/share/julia/stdlib/v1.8/LibGit2/src/error.jl:110 [inlined]
  [2] LibGit2.GitRepo(path::String)
    @ LibGit2 ~/Downloads/julia-27cb0c43e1/share/julia/stdlib/v1.8/LibGit2/src/repository.jl:11
  [3] #ensure_clone#2
    @ ~/Downloads/julia-27cb0c43e1/share/julia/stdlib/v1.8/Pkg/src/GitTools.jl:72 [inlined]
  [4] handle_repo_add!(ctx::Pkg.Types.Context, pkg::Pkg.Types.PackageSpec)
    @ Pkg.Types ~/Downloads/julia-27cb0c43e1/share/julia/stdlib/v1.8/Pkg/src/Types.jl:709
  [5] handle_repos_add!(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec})
    @ Pkg.Types ~/Downloads/julia-27cb0c43e1/share/julia/stdlib/v1.8/Pkg/src/Types.jl:777
  [6] add(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; preserve::Pkg.Types.PreserveLevel, platform::Base.BinaryPlatforms.Platform, kwargs::Base.Pairs{Symbol, Base.TTY, Tuple{Symbol}, NamedTuple{(:io,), Tuple{Base.TTY}}})
    @ Pkg.API ~/Downloads/julia-27cb0c43e1/share/julia/stdlib/v1.8/Pkg/src/API.jl:254
  [7] add(pkgs::Vector{Pkg.Types.PackageSpec}; io::Base.TTY, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Pkg.API ~/Downloads/julia-27cb0c43e1/share/julia/stdlib/v1.8/Pkg/src/API.jl:155
  [8] add(pkgs::Vector{Pkg.Types.PackageSpec})
    @ Pkg.API ~/Downloads/julia-27cb0c43e1/share/julia/stdlib/v1.8/Pkg/src/API.jl:145
  [9] do_cmd!(command::Pkg.REPLMode.Command, repl::REPL.LineEditREPL)
    @ Pkg.REPLMode ~/Downloads/julia-27cb0c43e1/share/julia/stdlib/v1.8/Pkg/src/REPLMode/REPLMode.jl:408
 [10] do_cmd(repl::REPL.LineEditREPL, input::String; do_rethrow::Bool)
    @ Pkg.REPLMode ~/Downloads/julia-27cb0c43e1/share/julia/stdlib/v1.8/Pkg/src/REPLMode/REPLMode.jl:386
 [11] do_cmd
    @ ~/Downloads/julia-27cb0c43e1/share/julia/stdlib/v1.8/Pkg/src/REPLMode/REPLMode.jl:377 [inlined]
 [12] (::Pkg.REPLMode.var"#24#27"{REPL.LineEditREPL, REPL.LineEdit.Prompt})(s::REPL.LineEdit.MIState, buf::IOBuffer, ok::Bool)
    @ Pkg.REPLMode ~/Downloads/julia-27cb0c43e1/share/julia/stdlib/v1.8/Pkg/src/REPLMode/REPLMode.jl:550
 [13] #invokelatest#2
    @ ./essentials.jl:718 [inlined]
 [14] invokelatest
    @ ./essentials.jl:716 [inlined]
 [15] run_interface(terminal::REPL.Terminals.TextTerminal, m::REPL.LineEdit.ModalInterface, s::REPL.LineEdit.MIState)
    @ REPL.LineEdit ~/Downloads/julia-27cb0c43e1/share/julia/stdlib/v1.8/REPL/src/LineEdit.jl:2509
 [16] run_frontend(repl::REPL.LineEditREPL, backend::REPL.REPLBackendRef)
    @ REPL ~/Downloads/julia-27cb0c43e1/share/julia/stdlib/v1.8/REPL/src/REPL.jl:1245
 [17] (::REPL.var"#49#54"{REPL.LineEditREPL, REPL.REPLBackendRef})()
    @ REPL ./task.jl:411

@kellertuer
Copy link

Yes that binary works again and can download packages. Thanks for the fix – can we have that as a patch for 1.6.x? That would be awesome.

staticfloat added a commit to JuliaLang/julia that referenced this issue Oct 21, 2021
* Disable `DL_LOAD_PATH` prepending for `@`-paths on Darwin

Many thanks to Randy Rucker from Apple for pointing out that we were
technically relying on undefined behavior in `dyld` for loading things
via `jl_dlopen("@loader_path/@rpath/libfoo.dylib")`, due to the
composition of `dlopen("@rpath/libfoo.dylib")` in our Julia code, and
the `DL_LOAD_PATH` prepending we do in `jl_load_dynamic_library()`.

This PR uses a slightly modified version of a patch emailed to me by
Randy, and should solve
JuliaLang/Downloads.jl#149 on macOS Monterey
where the undefined behavior in `dyld` has changed.

* Apply suggestions from code review

Co-authored-by: Jameson Nash <vtjnash@gmail.com>

* Use `[]` instead of `*`

Co-authored-by: Jameson Nash <vtjnash@gmail.com>
@staticfloat
Copy link
Member

@samuel3008 I think your issue may be unrelated. Can you please open a new issue on the Pkg.jl repository with all your system information? Can you also try performing those steps with a new depot (e.g. start Julia with a JULIA_DEPOT_PATH=/tmp/temp_depot or similar) to force it to set everything up from scratch and see if this is related to bad state in your default depot?

Thanks for the fix – can we have that as a patch for 1.6.x? That would be awesome.

Yes, the PR (which was just merged, so master builds should start to have the fix built-in from here on out) is tagged to be backported to both v1.6.X and v1.7.X.

@SamuelBadr
Copy link
Author

Thank you for your help!

PS: I opened the Pkg.jl issue (JuliaLang/Pkg.jl#2788 in case anybody is trying to find it).

KristofferC pushed a commit to JuliaLang/julia that referenced this issue Oct 22, 2021
* Disable `DL_LOAD_PATH` prepending for `@`-paths on Darwin

Many thanks to Randy Rucker from Apple for pointing out that we were
technically relying on undefined behavior in `dyld` for loading things
via `jl_dlopen("@loader_path/@rpath/libfoo.dylib")`, due to the
composition of `dlopen("@rpath/libfoo.dylib")` in our Julia code, and
the `DL_LOAD_PATH` prepending we do in `jl_load_dynamic_library()`.

This PR uses a slightly modified version of a patch emailed to me by
Randy, and should solve
JuliaLang/Downloads.jl#149 on macOS Monterey
where the undefined behavior in `dyld` has changed.

* Apply suggestions from code review

Co-authored-by: Jameson Nash <vtjnash@gmail.com>

* Use `[]` instead of `*`

Co-authored-by: Jameson Nash <vtjnash@gmail.com>
(cherry picked from commit 76c2431)
KristofferC pushed a commit to JuliaLang/julia that referenced this issue Oct 29, 2021
* Disable `DL_LOAD_PATH` prepending for `@`-paths on Darwin

Many thanks to Randy Rucker from Apple for pointing out that we were
technically relying on undefined behavior in `dyld` for loading things
via `jl_dlopen("@loader_path/@rpath/libfoo.dylib")`, due to the
composition of `dlopen("@rpath/libfoo.dylib")` in our Julia code, and
the `DL_LOAD_PATH` prepending we do in `jl_load_dynamic_library()`.

This PR uses a slightly modified version of a patch emailed to me by
Randy, and should solve
JuliaLang/Downloads.jl#149 on macOS Monterey
where the undefined behavior in `dyld` has changed.

* Apply suggestions from code review

Co-authored-by: Jameson Nash <vtjnash@gmail.com>

* Use `[]` instead of `*`

Co-authored-by: Jameson Nash <vtjnash@gmail.com>
(cherry picked from commit 76c2431)
KristofferC pushed a commit to JuliaLang/julia that referenced this issue Oct 29, 2021
* Disable `DL_LOAD_PATH` prepending for `@`-paths on Darwin

Many thanks to Randy Rucker from Apple for pointing out that we were
technically relying on undefined behavior in `dyld` for loading things
via `jl_dlopen("@loader_path/@rpath/libfoo.dylib")`, due to the
composition of `dlopen("@rpath/libfoo.dylib")` in our Julia code, and
the `DL_LOAD_PATH` prepending we do in `jl_load_dynamic_library()`.

This PR uses a slightly modified version of a patch emailed to me by
Randy, and should solve
JuliaLang/Downloads.jl#149 on macOS Monterey
where the undefined behavior in `dyld` has changed.

* Apply suggestions from code review

Co-authored-by: Jameson Nash <vtjnash@gmail.com>

* Use `[]` instead of `*`

Co-authored-by: Jameson Nash <vtjnash@gmail.com>
(cherry picked from commit 76c2431)
@Liu-Wenzheng
Copy link

I reached out to Apple and they provided some guidance on what could be the issue: can anyone affected by this please try either building this branch, or running this binary (x86_64 only), and seeing if it resolves the issue?

The binary is a .tar.gz file. After I unzipped it, which file should I run? bin/Julia or some files else?
Screen Shot 2021-11-10 at 5 19 27 PM

KristofferC pushed a commit to JuliaLang/julia that referenced this issue Nov 11, 2021
* Disable `DL_LOAD_PATH` prepending for `@`-paths on Darwin

Many thanks to Randy Rucker from Apple for pointing out that we were
technically relying on undefined behavior in `dyld` for loading things
via `jl_dlopen("@loader_path/@rpath/libfoo.dylib")`, due to the
composition of `dlopen("@rpath/libfoo.dylib")` in our Julia code, and
the `DL_LOAD_PATH` prepending we do in `jl_load_dynamic_library()`.

This PR uses a slightly modified version of a patch emailed to me by
Randy, and should solve
JuliaLang/Downloads.jl#149 on macOS Monterey
where the undefined behavior in `dyld` has changed.

* Apply suggestions from code review

Co-authored-by: Jameson Nash <vtjnash@gmail.com>

* Use `[]` instead of `*`

Co-authored-by: Jameson Nash <vtjnash@gmail.com>
(cherry picked from commit 76c2431)
@staticfloat
Copy link
Member

@Liu-Wenzheng no need to use that build anymore, just download the latest nightly version of Julia from this webpage.

@FelixBenning
Copy link

FelixBenning commented Nov 18, 2021

==> Pouring julia--1.6.3_3.monterey.bottle.tar.gz
🍺  /usr/local/Cellar/julia/1.6.3_3: 1,745 files, 224.9MB
felix@stochastik3 ~ % julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.3 (2021-09-23)
 _/ |\__'_|_|_|\__'_|  |  Built by Homebrew (v1.6.3_3)
|__/                   |

(@v1.6) pkg> add Pluto
  Installing known registries into `~/.julia`
┌ Error: curl_easy_setopt: 48
└ @ Downloads.Curl /private/tmp/julia-20211024-77365-8svegd/julia-1.6.3/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_setopt: 48
└ @ Downloads.Curl /private/tmp/julia-20211024-77365-8svegd/julia-1.6.3/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_setopt: 48
└ @ Downloads.Curl /private/tmp/julia-20211024-77365-8svegd/julia-1.6.3/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_setopt: 48
└ @ Downloads.Curl /private/tmp/julia-20211024-77365-8svegd/julia-1.6.3/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_setopt: 48

EDIT: ohhh nightly version... nvm

@zfengg
Copy link

zfengg commented Nov 23, 2021

==> Pouring julia--1.6.3_3.monterey.bottle.tar.gz
🍺  /usr/local/Cellar/julia/1.6.3_3: 1,745 files, 224.9MB
felix@stochastik3 ~ % julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.3 (2021-09-23)
 _/ |\__'_|_|_|\__'_|  |  Built by Homebrew (v1.6.3_3)
|__/                   |

(@v1.6) pkg> add Pluto
  Installing known registries into `~/.julia`
┌ Error: curl_easy_setopt: 48
└ @ Downloads.Curl /private/tmp/julia-20211024-77365-8svegd/julia-1.6.3/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_setopt: 48
└ @ Downloads.Curl /private/tmp/julia-20211024-77365-8svegd/julia-1.6.3/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_setopt: 48
└ @ Downloads.Curl /private/tmp/julia-20211024-77365-8svegd/julia-1.6.3/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_setopt: 48
└ @ Downloads.Curl /private/tmp/julia-20211024-77365-8svegd/julia-1.6.3/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_setopt: 48

EDIT: ohhh nightly version... nvm

You may try julia-1.7-rc3 from https://julialang.org/downloads/. It works for me.

@kellertuer
Copy link

Works also again in the just published Julia 1.6.4 (4 days ago).

@StefanKarpinski
Copy link
Member

Great to hear! Thanks for reporting back.

@tbeason
Copy link

tbeason commented Feb 3, 2022

I'd like to add a new data point here. I'm not sure it is fully fixed. I get this error after building from 1.7.1 from source.

┌ Error: curl_easy_setopt: 48
└ @ Downloads.Curl /dev/shm/tbeason/julia-1.7.1/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/utils.jl:36

Note that the path it gives is where I built julia but not where it is installed. I did add a prefix=/blah/blah/ in Make.user and julia is installed in the correct location.

Now, if I activate any folder that I am in to create a new environment, I can add packages without getting such error. So I think in my case this must be related to a path error during the build process?

LilithHafner pushed a commit to LilithHafner/julia that referenced this issue Feb 22, 2022
…42721)

* Disable `DL_LOAD_PATH` prepending for `@`-paths on Darwin

Many thanks to Randy Rucker from Apple for pointing out that we were
technically relying on undefined behavior in `dyld` for loading things
via `jl_dlopen("@loader_path/@rpath/libfoo.dylib")`, due to the
composition of `dlopen("@rpath/libfoo.dylib")` in our Julia code, and
the `DL_LOAD_PATH` prepending we do in `jl_load_dynamic_library()`.

This PR uses a slightly modified version of a patch emailed to me by
Randy, and should solve
JuliaLang/Downloads.jl#149 on macOS Monterey
where the undefined behavior in `dyld` has changed.

* Apply suggestions from code review

Co-authored-by: Jameson Nash <vtjnash@gmail.com>

* Use `[]` instead of `*`

Co-authored-by: Jameson Nash <vtjnash@gmail.com>
@dpo
Copy link

dpo commented Mar 7, 2022

I'm using Julia 1.7.1 downloaded from julialang.org on macOS Monterey and I'm getting this error again.

LilithHafner pushed a commit to LilithHafner/julia that referenced this issue Mar 8, 2022
…42721)

* Disable `DL_LOAD_PATH` prepending for `@`-paths on Darwin

Many thanks to Randy Rucker from Apple for pointing out that we were
technically relying on undefined behavior in `dyld` for loading things
via `jl_dlopen("@loader_path/@rpath/libfoo.dylib")`, due to the
composition of `dlopen("@rpath/libfoo.dylib")` in our Julia code, and
the `DL_LOAD_PATH` prepending we do in `jl_load_dynamic_library()`.

This PR uses a slightly modified version of a patch emailed to me by
Randy, and should solve
JuliaLang/Downloads.jl#149 on macOS Monterey
where the undefined behavior in `dyld` has changed.

* Apply suggestions from code review

Co-authored-by: Jameson Nash <vtjnash@gmail.com>

* Use `[]` instead of `*`

Co-authored-by: Jameson Nash <vtjnash@gmail.com>
@StefanKarpinski
Copy link
Member

Can you try the debugging steps requested above? Such as DYLD_PRINT_LIBRARIES=YES julia -e0?

@nrkumar93
Copy link

Make it use the libcurl packaged with Julia. This works for me

export LD_LIBRARY_PATH="/opt/julia-1.6.7/lib/julia:$LD_LIBRARY_PATH"

staticfloat added a commit to JuliaLang/julia that referenced this issue Dec 23, 2022
* Disable `DL_LOAD_PATH` prepending for `@`-paths on Darwin

Many thanks to Randy Rucker from Apple for pointing out that we were
technically relying on undefined behavior in `dyld` for loading things
via `jl_dlopen("@loader_path/@rpath/libfoo.dylib")`, due to the
composition of `dlopen("@rpath/libfoo.dylib")` in our Julia code, and
the `DL_LOAD_PATH` prepending we do in `jl_load_dynamic_library()`.

This PR uses a slightly modified version of a patch emailed to me by
Randy, and should solve
JuliaLang/Downloads.jl#149 on macOS Monterey
where the undefined behavior in `dyld` has changed.

* Apply suggestions from code review

Co-authored-by: Jameson Nash <vtjnash@gmail.com>

* Use `[]` instead of `*`

Co-authored-by: Jameson Nash <vtjnash@gmail.com>
(cherry picked from commit 76c2431)
@arunoruto
Copy link

I am having the same issue on my Fedora 37 instance running julia 1.8

 Error: curl_easy_setopt: 48
└ @ Downloads.Curl /builddir/build/BUILD/julia-1.8.0/build/usr/share/julia/stdlib/v1.8/Downloads/src/Curl/utils.jl:29

@StefanKarpinski
Copy link
Member

Can you answer some of the above debugging steps? Eg DYLD_PRINT_LIBRARIES=YES julia -e0 2>&1 | grep curl and how did you install Julia? Are you using official binaries?

@Eemilp
Copy link

Eemilp commented Feb 8, 2023

I'm runing Julia 1.8.0 on Fedora 37 installed from Fedora repositories with the same error. DYLD_PRINT_LIBRARIES=YES julia -e0 2>&1 | grep curl produces no output at all.

Update: Installing Julia through Jualiaup fixes this so the Fedora packages may be just outdated.

@arunoruto
Copy link

Can you answer some of the above debugging steps? Eg DYLD_PRINT_LIBRARIES=YES julia -e0 2>&1 | grep curl and how did you install Julia? Are you using official binaries?

Similar to @Eemilp I also have no output on my Fedora 37 instance. I installed it from the default repositories dnf install julia.

@StefanKarpinski
Copy link
Member

StefanKarpinski commented Feb 9, 2023

Oh, we don't support distro Julia builds. They're usually broken because they do things like try to use whatever versions of libraries that happen to have, in this case they're using a version of libcurl that seems not to be compatible with Julia. You can complain to Fedora and ask them to fix it, but it's recommended that you use juliaup to install official Julia binaries instead: https://github.com/JuliaLang/juliaup.

@giordano
Copy link
Contributor

giordano commented Feb 9, 2023

For what is worth, DYLD_PRINT_LIBRARIES is a macOS-specific environment variable, it is expected that it doesn't have any effect for you on Linux.

CC @nalimilan for the Fedora package.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests