evm t8n is extremely slow

[winsvega]

I can't batch execute tests on besu's evm cause it takes too long time. been running the whole day not even half the state tests finished.
is it because besu java is slow on deployment for each individual test?
in rpc mode was fast.

[shemnon]

Java has a huge startup penalty, to JIT all the java classes. To address this I am working on getting GraalVM builds of the evmtool - #5192

[winsvega]

ethereumjs implemented t8n as a service. they start it once and then listener accept http requests and writes response with t8n format to the filesystem

[shemnon]

That can be done too. Can you point me to their interface requirements or what retesteth will need interface wise?

[winsvega]

this is how I start the daemon

cd $ETHEREUMJS_PATH/packages/vm
npx ts-node test/retesteth/transition-cluster.ts &> /dev/null &

stop

killall node

and run a t8n request:

curl -X POST -d "${1} ${2} ${3} ${4} ${5} ${6} ${7} ${8} ${9} ${10} ${11} ${12} ${13} ${14} ${15} ${16} ${17} ${18} ${19} ${20}" --silent http://localhost:3000/

Basically all the t8n flags go as $1 $2 args to the daemon.
This is what they gave me. btw. I think pyspecs can also work with this.

[shemnon]

so are there only 2 args or are they quoted or newlined?

[winsvega]

it will look like this if to open ${} bash args

curl -X POST -d "--state.fork ArrowGlacier --state.reward 0 --state.chainid 1 --input.alloc /dev/shm/e9bab9d9-8dc8-4812-91bd-4919b0f67552/alloc.json --input.txs /dev/shm/e9bab9d9-8dc8-4812-91bd-4919b0f67552/txs.rlp --input.env /dev/shm/e9bab9d9-8dc8-4812-91bd-4919b0f67552/env.json --output.basedir /dev/shm/e9bab9d9-8dc8-4812-91bd-4919b0f67552 --output.result out.json --output.alloc outAlloc.json" --silent http://localhost:3000/

[shemnon]

So all inputs and outputs will be via files, and there is no streaming output from the HTTP, it's just CLI args one test per line?

[winsvega]

in this case yes.
but since its a script. you can reply via curl and .sh script can write the files. so it can be a remote host too

[non-fungible-nelson]

Hey @shemnon - any ideas on what to do with this ticket? or tracking progress in a PR elsewhere?

[winsvega]

When you have time would be nice to have smth like ethereumjs do. Starting t8n as a daemon accepting requests without class reload.
Then I will be able to run all the tests we have in reasonable time.

[winsvega]

I see besu t8n fails some tests for sure.
info: (ValidBlocks/bcExample/optionsTest_London, fork: London, block: 3) reported different stateRoot

but I can't check it all without the speed/daemon optimization.