Parallel compilation is buggy

[lukaszcz]

I can deterministically reproduce a parallelism error on my Linux laptop:

1. Check out devnet (commit 15c2cf78796321f6d0b9039e705747754f8e8dea) in anoma-apps.
2. Run juvix clean --global && juvix dependencies update && juvix typecheck
   The result:

juvix: /home/heliax/Documents/anoma-apps/.juvix-build/0.6.8/home/heliax/Documents/anoma-apps/.juvix-build/0.6.8/deps/f6bbef2610b14aa5b27ede28104218d3b0a1c6ba9a42fafa37cea79ab3890f40/.juvix-build/0.6.8/stdlib/Stdlib/Cairo/Ec.jvo: withBinaryFile: resource busy (file is locked)

I get this error every time, but for a different file each time. The error doesn't appear with -N 1.

[lukaszcz]

This part in ParallelTemplate.compile looks fishy to me, but I don't fully understand how it's supposed to work:

  runReader varCompilationState
    . runReader nodesIx
    . runReader args
    . runReader compileQ
    . runReader deps
    . crashOnError
    $ do
      replicateConcurrently_ _compileArgsNumWorkers $
        lookForWork @nodeId @node @compileProof

Why are the runReader outside of replicateConcurrently? Won't it cause lookForWork to not be fully evaluated (it returns some data structure which represents delayed computation that needs the "reader" values provided to get evaluated, and these values are provided outside the replicateConcurrently, so possibly outside a mutex the code should be evaluated in?)

[lukaszcz]

Well, the simple fix of moving the runReader inside replicateConcurrently doesn't seem to help, but the issue may be more subtle. I'm not sure if what we execute under atomically is really evaluated atomically in the way we expect and not just some half-evaluated data structure is returned which is then non-atomically evaluated further when we do runReader. But I might just not understand the code well enough.

[lukaszcz]

Ah, no, but it should work because runConcurrent is somewhere at the top.