Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Julia 1.3 threaded write InexactError #144

Closed
scottstanie opened this issue Aug 6, 2019 · 3 comments
Closed

Julia 1.3 threaded write InexactError #144

scottstanie opened this issue Aug 6, 2019 · 3 comments

Comments

@scottstanie
Copy link

I can't seem to save different files while in a threaded for loop:

julia> using JLD2

julia> function f1()
       function innerwrite(layer, i)
           jldopen("testwrite$i.jld2", "w") do f
               write(f, "stack", layer)
               println("writing testwrite$i on thread $(Threads.threadid())")
           end
       end
       rr = rand(800, 800, 5);
       Threads.@threads for i = 1:size(rr, 3)
           layer = rr[:, :, i]
           innerwrite(layer, i)
       end
       end
f1 (generic function with 1 method)

julia> f1()
writing testwrite4 on thread 3
writing testwrite5 on thread 4
writing testwrite1 on thread 1
writing testwrite2 on thread 1
ERROR: InexactError: trunc(UInt32, -1320009088)
Stacktrace:
 [1] wait(::Task) at ./task.jl:221
 [2] macro expansion at ./threadingconstructs.jl:75 [inlined]
 [3] f1() at ./REPL[2]:9
 [4] top-level scope at REPL[3]:1

julia> versioninfo()
Julia Version 1.3.0-alpha.0
Commit 6c11e7c2c4 (2019-07-23 01:46 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.6.0)
  CPU: Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)
Environment:
  JULIA_NUM_THREADS = 4

Strangely, it also seems to be having the same bug (possibly a Julia 1.3 thread bug) that allows the code to run after, but only on 1 thread.
When I test the function again:

julia> f1()
writing testwrite1 on thread 1
writing testwrite2 on thread 1
writing testwrite3 on thread 1
writing testwrite4 on thread 1
writing testwrite5 on thread 1

I opened this also in JLD JuliaIO/JLD.jl#261 and Julia JuliaLang/julia#32802 as I'm not quite sure where the problem is here.

@JeffBezanson
Copy link
Contributor

t looks like this package is not thread-safe. It has a few pieces of global state. In this case I think the problem is

# simultaneously. This strategy is not thread-safe.
:

# We sometimes need to compute checksums. We do this by first calling begin_checksum when
# starting to handle whatever needs checksumming, and calling end_checksum afterwards. Note
# that we never compute nested checksums, but we may compute multiple checksums
# simultaneously. This strategy is not thread-safe.

const CHECKSUM_POS = Int64[]
const NCHECKSUM = Ref{Int}(0)

@JonasIsensee
Copy link
Collaborator

AFAICT there are two places that are not threadsafe.
the above plus a global list of currently open files.

The open file list could easily be made safe with a lock
but the checksum list..
In my view this shouldn't live in the main scope.
Maybe it could be moved into an additional field of theJLDFile

@JonasIsensee
Copy link
Collaborator

Opened as a general feature request in #200

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants