Can't run multiple forge script in parallel

[adhusson]

Component

Forge

Have you ensured that all of these are up to date?

• Foundry
• Foundryup

What version of Foundry are you on?

forge 0.2.0 (7e9e6a5 2022-08-22T00:05:42.088216Z)

What command(s) is the bug in?

forge script

Operating System

macOS (Apple Silicon)

Describe the bug

When running tests in parallel against multiple instances of anvil, I start multiple forge scripts at once, and get Could not open compiler cache on some of them.

It seems reasonable to allow multiple runs in parallel, so maybe forge instances could 1) wait when the cache is locked, and 2) not lock until they know they have to write, that way they could all read a fresh cache in parallel.

[joshieDo]

Even if that would be fixed, I'm not sure it would be okay to run them in parallel if you're using them with --broadcast on the same chain. The way I see it, for this to work:

• Either 1) or 2).
• If --broadcast is passed, lock the chain: eg. broadcast/1337/.lock and only let go, when every transaction has been sent or the user forces it with a dialog

what is your usecase?

[adhusson]

Yup, I'm not running broadcast on the same chain! Talking to multiple anvil instances at the same time (running tests in parallel).

edit: to expand on your comment: shouldn't it be both 1) and 2)? If just 1), then you have unnecessary waiting. If just 2), then you have failures whenever a lock is on. Also, about the .lock file, this would definitely be helpful for multiple senders going to the same chain (and should probably be implemented?) but is I think not related to my issue.

[mds1]

@adhusson Is this still an issue? I think I was just able to get two scripts running (seemingly) in parallel, as I did not get the Could not open compiler cache error. Closing this optimistically, we can reopen if needed though

[lnist]

@mds1 : I've just done foundryup, but I still have the issue. Starting the following in 7 parallel processes gives the Could not open compiler cache error:

forge script     --rpc-url http://127.0.0.1:8545     --froms 0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266     --private-key 0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80     --broadcast -vvv     --root /my/foundry/code          MyFoundryScript

The port number of course differs in each process.

Foundry up:

foundryup: installed - forge 0.2.0 (388c3c0 2023-04-12T00:10:28.965283288Z)
foundryup: installed - cast 0.2.0 (388c3c0 2023-04-12T00:10:28.965283288Z)
foundryup: installed - anvil 0.1.0 (388c3c0 2023-04-12T00:10:59.673375678Z)
foundryup: installed - chisel 0.1.1 (388c3c0 2023-04-12T00:10:59.667448854Z)

[lnist]

Note: After playing around to make a smaller example it turns out that forge script is not enough to make it fail - but prefixing it with anvil_setCode cheatcode is, so here is a repro with the forge init setup:

BYTECODE=$(forge inspect Counter deployedBytecode)

anvil --host 127.0.0.1 --port 8543 &
anvil --host 127.0.0.1 --port 8544 &
anvil --host 127.0.0.1 --port 8545 &
anvil --host 127.0.0.1 --port 8546 &
anvil --host 127.0.0.1 --port 8547 &
anvil --host 127.0.0.1 --port 8548 &
anvil --host 127.0.0.1 --port 8549 &

# Wait a bit

cast rpc anvil_setCode  --rpc-url http://127.0.0.1:8543 0xdecaf00000000000000000000000000000000000 $BYTECODE && forge script --rpc-url http://127.0.0.1:8543 --froms 0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266 --private-key 0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80 --broadcast -vvv CounterScript &
cast rpc anvil_setCode  --rpc-url http://127.0.0.1:8544 0xdecaf00000000000000000000000000000000000 $BYTECODE && forge script --rpc-url http://127.0.0.1:8544 --froms 0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266 --private-key 0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80 --broadcast -vvv CounterScript &
cast rpc anvil_setCode  --rpc-url http://127.0.0.1:8545 0xdecaf00000000000000000000000000000000000 $BYTECODE && forge script --rpc-url http://127.0.0.1:8545 --froms 0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266 --private-key 0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80 --broadcast -vvv CounterScript &
cast rpc anvil_setCode  --rpc-url http://127.0.0.1:8546 0xdecaf00000000000000000000000000000000000 $BYTECODE && forge script --rpc-url http://127.0.0.1:8546 --froms 0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266 --private-key 0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80 --broadcast -vvv CounterScript &
cast rpc anvil_setCode  --rpc-url http://127.0.0.1:8547 0xdecaf00000000000000000000000000000000000 $BYTECODE && forge script --rpc-url http://127.0.0.1:8547 --froms 0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266 --private-key 0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80 --broadcast -vvv CounterScript &
cast rpc anvil_setCode  --rpc-url http://127.0.0.1:8548 0xdecaf00000000000000000000000000000000000 $BYTECODE && forge script --rpc-url http://127.0.0.1:8548 --froms 0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266 --private-key 0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80 --broadcast -vvv CounterScript &
cast rpc anvil_setCode  --rpc-url http://127.0.0.1:8549 0xdecaf00000000000000000000000000000000000 $BYTECODE && forge script --rpc-url http://127.0.0.1:8549 --froms 0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266 --private-key 0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80 --broadcast -vvv CounterScript &

[mds1]

Thanks! I was able to reproduce this with the above steps

[klkvr]

Can't reproduce this anymore. Scripts aren't relying on cache as heavily now, so this likely has been fixed.