Error: curl_multi_add_handle: 8 #196
Comments
a-cakir
changed the title
That's because this error is done as a logging statement instead of throwing an exception. I don't know why, maybe intervention isn't needed or something? According to https://curl.se/libcurl/c/curl_multi_add_handle.html
Was this in an |
|
Yes, it was in |
|
I don't know if this is the same issue, but just now (good timing!) I ctrl-c'd while downloading something and got ^CUnhandled Task ERROR: InterruptException:
Stacktrace:
[1] curl_multi_socket_action
@ ~/.asdf/installs/julia/1.7.3/share/julia/stdlib/v1.7/Downloads/src/Curl/Curl.jl:48 [inlined]
[2] curl_multi_socket_action
@ ~/.asdf/installs/julia/1.7.3/share/julia/stdlib/v1.7/Downloads/src/Curl/Curl.jl:56 [inlined]
[3] macro expansion
@ ~/.asdf/installs/julia/1.7.3/share/julia/stdlib/v1.7/Downloads/src/Curl/utils.jl:28 [inlined]
[4] (::Downloads.Curl.var"#41#47"{UInt8, Int32, FileWatching.FDWatcher, Downloads.Curl.Multi})()
@ Downloads.Curl ~/.asdf/installs/julia/1.7.3/share/julia/stdlib/v1.7/Downloads/src/Curl/Multi.jl:176
[5] lock(f::Downloads.Curl.var"#41#47"{UInt8, Int32, FileWatching.FDWatcher, Downloads.Curl.Multi}, l::ReentrantLock)
@ Base ./lock.jl:190
[6] macro expansion
@ ~/.asdf/installs/julia/1.7.3/share/julia/stdlib/v1.7/Downloads/src/Curl/Multi.jl:174 [inlined]
[7] (::Downloads.Curl.var"#40#46"{Int32, FileWatching.FDWatcher, Downloads.Curl.Multi})()
@ Downloads.Curl ./task.jl:429
┌ Error: curl_multi_socket_action: 8
└ @ Downloads.Curl /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/utils.jl:29
^CThen when I tried to download again, I got more ┌ Error: curl_multi_add_handle: 8
└ @ Downloads.Curl /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/utils.jl:29Then I cleared the Downloader, with julia> using AWS
julia> AWS.AWS_DOWNLOADER[] = nothingand it seemed to fix it. Did you also ctrl-c something before getting the error messages @a-cakir? |
|
I did also ctrl-c but I cannot fully recall if I started seeing this error only after the ctrl-c unfortunately. |
|
The same error again and it only started appearing after Ctrl-C. And your suggestion above seems to fix it. |
|
Ok, good! I am guessing the issue then is corruption of downloader state on interrupt. Seems like Downloads.jl is missing some handling of this. |
|
@vtjnash, would it be possible to rig up our interrupt handling to blow the download object away? Or is it possible for us to handle interrupts more gracefully some other way? |
|
There is no safe or reliable way to recover from this that I know |
|
C'mon, yes, there's nothing fully reliable that can be done, but is there something less than 100% reliable that can be done? Most software manages to not completely shit the bed when someone does ctrl-C. |
|
you probably need to wrap every |
|
Ok, so all the cfunctions should instead return with an error code indicating failure? I can try that. |
|
FWIW I see this on 1.8 too |
|
Ok, I finally got around to trying this and putting try/catch blocks in cfunctions seems to interact badly with libuv's getaddrinfo thread stuff. With this patch: diff --git a/src/Curl/Easy.jl b/src/Curl/Easy.jl
index 64fd159..59025ab 100644
--- a/src/Curl/Easy.jl
+++ b/src/Curl/Easy.jl
@@ -315,11 +315,15 @@ function header_callback(
count :: Csize_t,
easy_p :: Ptr{Cvoid},
)::Csize_t
- easy = unsafe_pointer_to_objref(easy_p)::Easy
- n = size * count
- hdr = unsafe_string(data, n)
- push!(easy.res_hdrs, hdr)
- return n
+ try
+ easy = unsafe_pointer_to_objref(easy_p)::Easy
+ n = size * count
+ hdr = unsafe_string(data, n)
+ push!(easy.res_hdrs, hdr)
+ return n
+ catch
+ return -1
+ end
end
# feed data to read_callbackI get this error while trying to do multiple downloads from different tasks: @testset "concurrent requests" begin
mine = Downloader()
for downloader in (nothing, mine)
have_lsof = Sys.which("lsof") !== nothing
count_tcp() = Base.count(x->contains("TCP",x), split(read(`lsof -p $(getpid())`, String), '\n'))
if have_lsof
n_tcp = count_tcp()
end
delay = 2
count = 100
url = "$server/delay/$delay"
t = @elapsed @sync for id = 1:count
@async begin
json = download_json("$url?id=$id", downloader = downloader)
@test get(json["args"], "id", nothing) == ["$id"]
end
end
@test t < 0.9*count*delay
if have_lsof
@test n_tcp == count_tcp()
end
end
endError: |
|
With #205 I don't get that error anymore, so I can try comprehensively adding try/catch blocks to all the callbacks. |
StefanKarpinski
added a commit
that referenced
this issue
Oct 1, 2022
This may help #196. The issue there may be that callback functions raise Julia errors sometimes (they shouldn't but stuff happens, ya know?). This wraps each one in a try/catch block and logs an "unkonwn error" message if that happens and returns from the callback with an appropriate return code to indicate error. The libcurl docs aren't super clear on these and this is hard to test so hopefully I got it right, but we'll see. Should be better.
|
Any feedback here? I don't know if anyone has gotten to try a nightly Julia out but this might be fixed there. |
|
I'm going to close this for now. Please comment if you encounter this on nightly and we can reopen. |
|
I just encountered this again on 1.10-beta2. Doing some PkgEval development, I hit CTRL-C to interrupt an operation I had started. I had to mash CTRL-C a couple of times to actually get my REPL back, as can be seen in the following output: Interestingly, when I wanted to perform the call again afterwards, some of the delayed interrupts (presumably resulting from me mashing CTRL-C) got delivered without me actually pressing CTRL-C: i.e. despite the |
|
I'm really not sure that we can expect this package to defend against all possible ctrl-c mashing that can be done. Jameson has already told us that it's basically impossible to reliably recover from and we're doing what we can. |
|
Fair enough. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I ran into the
Error: curl_multi_add_handle: 8error, though sporadically. The interesting part is that the error did not stop the code execution, which I dont know enough about the details of the Downloads.jl module to know if that makes sense or not.The julia version that I am using is 1.7.3
The error I got is below (Repeated countless of times)
@ericphanson
The text was updated successfully, but these errors were encountered: