Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update to spec tests v1.3.0 alpha.2 #4917

Merged
merged 3 commits into from
Dec 18, 2022
Merged

Update to spec tests v1.3.0 alpha.2 #4917

merged 3 commits into from
Dec 18, 2022

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Dec 17, 2022

@twoeths twoeths mentioned this pull request Dec 17, 2022
Merged
@twoeths twoeths force-pushed the tuyen/spec_tests_v1.3.0-alpha.2 branch from 78ec671 to 1a56694 Compare December 18, 2022 02:37
@github-actions
Copy link
Contributor

github-actions bot commented Dec 18, 2022 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 2696a5e Previous: d960f3e Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.1638 ms/op 967.01 us/op 1.20
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 66.982 us/op 60.055 us/op 1.12
BLS verify - blst-native 1.8642 ms/op 2.1779 ms/op 0.86
BLS verifyMultipleSignatures 3 - blst-native 3.8083 ms/op 4.4916 ms/op 0.85
BLS verifyMultipleSignatures 8 - blst-native 8.1929 ms/op 9.7005 ms/op 0.84
BLS verifyMultipleSignatures 32 - blst-native 29.681 ms/op 35.259 ms/op 0.84
BLS aggregatePubkeys 32 - blst-native 39.817 us/op 46.397 us/op 0.86
BLS aggregatePubkeys 128 - blst-native 155.76 us/op 182.04 us/op 0.86
getAttestationsForBlock 81.932 ms/op 78.047 ms/op 1.05
isKnown best case - 1 super set check 426.00 ns/op 437.00 ns/op 0.97
isKnown normal case - 2 super set checks 418.00 ns/op 432.00 ns/op 0.97
isKnown worse case - 16 super set checks 415.00 ns/op 431.00 ns/op 0.96
CheckpointStateCache - add get delete 8.0290 us/op 8.4080 us/op 0.95
validate gossip signedAggregateAndProof - struct 4.2698 ms/op 5.0236 ms/op 0.85
validate gossip attestation - struct 2.0222 ms/op 2.3598 ms/op 0.86
pickEth1Vote - no votes 2.0819 ms/op 2.1680 ms/op 0.96
pickEth1Vote - max votes 15.565 ms/op 15.655 ms/op 0.99
pickEth1Vote - Eth1Data hashTreeRoot value x2048 11.500 ms/op 13.005 ms/op 0.88
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 19.996 ms/op 21.175 ms/op 0.94
pickEth1Vote - Eth1Data fastSerialize value x2048 1.1928 ms/op 1.0913 ms/op 1.09
pickEth1Vote - Eth1Data fastSerialize tree x2048 9.2853 ms/op 9.4792 ms/op 0.98
bytes32 toHexString 1.0180 us/op 853.00 ns/op 1.19
bytes32 Buffer.toString(hex) 576.00 ns/op 652.00 ns/op 0.88
bytes32 Buffer.toString(hex) from Uint8Array 895.00 ns/op 991.00 ns/op 0.90
bytes32 Buffer.toString(hex) + 0x 572.00 ns/op 671.00 ns/op 0.85
Object access 1 prop 0.25800 ns/op 0.26400 ns/op 0.98
Map access 1 prop 0.25600 ns/op 0.26000 ns/op 0.98
Object get x1000 16.330 ns/op 11.026 ns/op 1.48
Map get x1000 0.98000 ns/op 0.92300 ns/op 1.06
Object set x1000 106.70 ns/op 70.230 ns/op 1.52
Map set x1000 63.575 ns/op 41.909 ns/op 1.52
Return object 10000 times 0.36290 ns/op 0.43470 ns/op 0.83
Throw Error 10000 times 6.7585 us/op 6.5522 us/op 1.03
fastMsgIdFn sha256 / 200 bytes 4.2710 us/op 4.9400 us/op 0.86
fastMsgIdFn h32 xxhash / 200 bytes 465.00 ns/op 460.00 ns/op 1.01
fastMsgIdFn h64 xxhash / 200 bytes 604.00 ns/op 732.00 ns/op 0.83
fastMsgIdFn sha256 / 1000 bytes 13.627 us/op 15.662 us/op 0.87
fastMsgIdFn h32 xxhash / 1000 bytes 611.00 ns/op 633.00 ns/op 0.97
fastMsgIdFn h64 xxhash / 1000 bytes 696.00 ns/op 818.00 ns/op 0.85
fastMsgIdFn sha256 / 10000 bytes 117.65 us/op 136.54 us/op 0.86
fastMsgIdFn h32 xxhash / 10000 bytes 2.3330 us/op 2.5530 us/op 0.91
fastMsgIdFn h64 xxhash / 10000 bytes 1.6620 us/op 1.7870 us/op 0.93
enrSubnets - fastDeserialize 64 bits 2.2240 us/op 2.1150 us/op 1.05
enrSubnets - ssz BitVector 64 bits 789.00 ns/op 796.00 ns/op 0.99
enrSubnets - fastDeserialize 4 bits 279.00 ns/op 288.00 ns/op 0.97
enrSubnets - ssz BitVector 4 bits 824.00 ns/op 812.00 ns/op 1.01
prioritizePeers score -10:0 att 32-0.1 sync 2-0 93.669 us/op 76.808 us/op 1.22
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 134.70 us/op 113.11 us/op 1.19
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 251.37 us/op 203.86 us/op 1.23
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 406.47 us/op 320.36 us/op 1.27
prioritizePeers score 0:0 att 64-1 sync 4-1 511.62 us/op 397.45 us/op 1.29
array of 16000 items push then shift 2.8000 us/op 51.615 us/op 0.05
LinkedList of 16000 items push then shift 16.981 ns/op 12.269 ns/op 1.38
array of 16000 items push then pop 210.14 ns/op 186.86 ns/op 1.12
LinkedList of 16000 items push then pop 16.468 ns/op 11.891 ns/op 1.38
array of 24000 items push then shift 3.9597 us/op 77.389 us/op 0.05
LinkedList of 24000 items push then shift 18.947 ns/op 12.443 ns/op 1.52
array of 24000 items push then pop 209.44 ns/op 194.61 ns/op 1.08
LinkedList of 24000 items push then pop 17.572 ns/op 11.968 ns/op 1.47
intersect bitArray bitLen 8 29.293 ns/op 21.652 ns/op 1.35
intersect array and set length 8 151.97 ns/op 122.74 ns/op 1.24
intersect bitArray bitLen 128 108.90 ns/op 70.870 ns/op 1.54
intersect array and set length 128 2.1466 us/op 1.6511 us/op 1.30
Buffer.concat 32 items 5.2110 us/op 4.4720 us/op 1.17
Uint8Array.set 32 items 4.0850 us/op 3.6510 us/op 1.12
pass gossip attestations to forkchoice per slot 6.1814 ms/op 3.7081 ms/op 1.67
computeDeltas 6.3853 ms/op 4.5829 ms/op 1.39
computeProposerBoostScoreFromBalances 2.8438 ms/op 2.4981 ms/op 1.14
altair processAttestation - 250000 vs - 7PWei normalcase 3.6582 ms/op 3.4534 ms/op 1.06
altair processAttestation - 250000 vs - 7PWei worstcase 5.7223 ms/op 5.4066 ms/op 1.06
altair processAttestation - setStatus - 1/6 committees join 210.05 us/op 178.24 us/op 1.18
altair processAttestation - setStatus - 1/3 committees join 400.91 us/op 355.07 us/op 1.13
altair processAttestation - setStatus - 1/2 committees join 562.05 us/op 513.49 us/op 1.09
altair processAttestation - setStatus - 2/3 committees join 720.80 us/op 681.85 us/op 1.06
altair processAttestation - setStatus - 4/5 committees join 994.58 us/op 940.98 us/op 1.06
altair processAttestation - setStatus - 100% committees join 1.1825 ms/op 1.1269 ms/op 1.05
altair processBlock - 250000 vs - 7PWei normalcase 25.502 ms/op 23.516 ms/op 1.08
altair processBlock - 250000 vs - 7PWei normalcase hashState 37.865 ms/op 37.722 ms/op 1.00
altair processBlock - 250000 vs - 7PWei worstcase 76.202 ms/op 74.048 ms/op 1.03
altair processBlock - 250000 vs - 7PWei worstcase hashState 104.14 ms/op 111.36 ms/op 0.94
phase0 processBlock - 250000 vs - 7PWei normalcase 3.3087 ms/op 3.2842 ms/op 1.01
phase0 processBlock - 250000 vs - 7PWei worstcase 44.471 ms/op 50.792 ms/op 0.88
altair processEth1Data - 250000 vs - 7PWei normalcase 736.86 us/op 672.63 us/op 1.10
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 9.2500 us/op 7.7710 us/op 1.19
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 28.142 us/op 22.048 us/op 1.28
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 12.067 us/op 10.177 us/op 1.19
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 9.4910 us/op 7.4650 us/op 1.27
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 101.20 us/op 87.467 us/op 1.16
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 1.0060 ms/op 864.56 us/op 1.16
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 1.3538 ms/op 1.1514 ms/op 1.18
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 1.3383 ms/op 1.1085 ms/op 1.21
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 4.2590 ms/op 4.0174 ms/op 1.06
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 2.9197 ms/op 2.3659 ms/op 1.23
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 6.8279 ms/op 7.0369 ms/op 0.97
Tree 40 250000 create 631.30 ms/op 587.94 ms/op 1.07
Tree 40 250000 get(125000) 284.74 ns/op 265.20 ns/op 1.07
Tree 40 250000 set(125000) 2.0206 us/op 1.9382 us/op 1.04
Tree 40 250000 toArray() 30.692 ms/op 26.127 ms/op 1.17
Tree 40 250000 iterate all - toArray() + loop 30.562 ms/op 26.809 ms/op 1.14
Tree 40 250000 iterate all - get(i) 105.79 ms/op 106.78 ms/op 0.99
MutableVector 250000 create 15.166 ms/op 13.054 ms/op 1.16
MutableVector 250000 get(125000) 13.063 ns/op 10.713 ns/op 1.22
MutableVector 250000 set(125000) 568.78 ns/op 506.85 ns/op 1.12
MutableVector 250000 toArray() 6.5524 ms/op 5.7672 ms/op 1.14
MutableVector 250000 iterate all - toArray() + loop 7.2373 ms/op 6.3763 ms/op 1.14
MutableVector 250000 iterate all - get(i) 3.4939 ms/op 2.6414 ms/op 1.32
Array 250000 create 5.8288 ms/op 5.3270 ms/op 1.09
Array 250000 clone - spread 2.1074 ms/op 2.0369 ms/op 1.03
Array 250000 get(125000) 0.93000 ns/op 0.99300 ns/op 0.94
Array 250000 set(125000) 0.93200 ns/op 0.96100 ns/op 0.97
Array 250000 iterate all - loop 167.81 us/op 152.32 us/op 1.10
effectiveBalanceIncrements clone Uint8Array 300000 59.066 us/op 36.231 us/op 1.63
effectiveBalanceIncrements clone MutableVector 300000 584.00 ns/op 565.00 ns/op 1.03
effectiveBalanceIncrements rw all Uint8Array 300000 252.91 us/op 244.41 us/op 1.03
effectiveBalanceIncrements rw all MutableVector 300000 159.15 ms/op 139.12 ms/op 1.14
phase0 afterProcessEpoch - 250000 vs - 7PWei 183.93 ms/op 186.43 ms/op 0.99
phase0 beforeProcessEpoch - 250000 vs - 7PWei 69.606 ms/op 70.103 ms/op 0.99
altair processEpoch - mainnet_e81889 486.68 ms/op 539.56 ms/op 0.90
mainnet_e81889 - altair beforeProcessEpoch 131.27 ms/op 77.000 ms/op 1.70
mainnet_e81889 - altair processJustificationAndFinalization 19.272 us/op 15.058 us/op 1.28
mainnet_e81889 - altair processInactivityUpdates 10.831 ms/op 8.8066 ms/op 1.23
mainnet_e81889 - altair processRewardsAndPenalties 88.833 ms/op 124.04 ms/op 0.72
mainnet_e81889 - altair processRegistryUpdates 2.7930 us/op 2.1430 us/op 1.30
mainnet_e81889 - altair processSlashings 797.00 ns/op 473.00 ns/op 1.68
mainnet_e81889 - altair processEth1DataReset 804.00 ns/op 571.00 ns/op 1.41
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.3372 ms/op 2.0979 ms/op 1.11
mainnet_e81889 - altair processSlashingsReset 6.4400 us/op 3.6860 us/op 1.75
mainnet_e81889 - altair processRandaoMixesReset 5.4240 us/op 4.0470 us/op 1.34
mainnet_e81889 - altair processHistoricalRootsUpdate 837.00 ns/op 606.00 ns/op 1.38
mainnet_e81889 - altair processParticipationFlagUpdates 2.5510 us/op 2.0150 us/op 1.27
mainnet_e81889 - altair processSyncCommitteeUpdates 808.00 ns/op 529.00 ns/op 1.53
mainnet_e81889 - altair afterProcessEpoch 193.73 ms/op 195.53 ms/op 0.99
phase0 processEpoch - mainnet_e58758 509.10 ms/op 557.52 ms/op 0.91
mainnet_e58758 - phase0 beforeProcessEpoch 241.70 ms/op 148.21 ms/op 1.63
mainnet_e58758 - phase0 processJustificationAndFinalization 19.221 us/op 15.802 us/op 1.22
mainnet_e58758 - phase0 processRewardsAndPenalties 110.32 ms/op 104.24 ms/op 1.06
mainnet_e58758 - phase0 processRegistryUpdates 9.8120 us/op 7.6640 us/op 1.28
mainnet_e58758 - phase0 processSlashings 691.00 ns/op 563.00 ns/op 1.23
mainnet_e58758 - phase0 processEth1DataReset 710.00 ns/op 546.00 ns/op 1.30
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.9130 ms/op 1.6331 ms/op 1.17
mainnet_e58758 - phase0 processSlashingsReset 4.3280 us/op 3.3840 us/op 1.28
mainnet_e58758 - phase0 processRandaoMixesReset 5.1430 us/op 4.0480 us/op 1.27
mainnet_e58758 - phase0 processHistoricalRootsUpdate 770.00 ns/op 611.00 ns/op 1.26
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.3060 us/op 3.7120 us/op 1.16
mainnet_e58758 - phase0 afterProcessEpoch 159.00 ms/op 161.51 ms/op 0.98
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.7912 ms/op 1.9331 ms/op 1.44
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 3.2649 ms/op 2.1756 ms/op 1.50
altair processInactivityUpdates - 250000 normalcase 37.211 ms/op 41.505 ms/op 0.90
altair processInactivityUpdates - 250000 worstcase 44.347 ms/op 33.230 ms/op 1.33
phase0 processRegistryUpdates - 250000 normalcase 8.1500 us/op 6.2940 us/op 1.29
phase0 processRegistryUpdates - 250000 badcase_full_deposits 422.47 us/op 388.05 us/op 1.09
phase0 processRegistryUpdates - 250000 worstcase 0.5 208.09 ms/op 196.50 ms/op 1.06
altair processRewardsAndPenalties - 250000 normalcase 119.25 ms/op 102.36 ms/op 1.17
altair processRewardsAndPenalties - 250000 worstcase 82.857 ms/op 71.669 ms/op 1.16
phase0 getAttestationDeltas - 250000 normalcase 12.193 ms/op 11.619 ms/op 1.05
phase0 getAttestationDeltas - 250000 worstcase 12.322 ms/op 11.861 ms/op 1.04
phase0 processSlashings - 250000 worstcase 5.6832 ms/op 5.1229 ms/op 1.11
altair processSyncCommitteeUpdates - 250000 260.87 ms/op 286.43 ms/op 0.91
BeaconState.hashTreeRoot - No change 438.00 ns/op 455.00 ns/op 0.96
BeaconState.hashTreeRoot - 1 full validator 61.984 us/op 70.243 us/op 0.88
BeaconState.hashTreeRoot - 32 full validator 615.44 us/op 728.13 us/op 0.85
BeaconState.hashTreeRoot - 512 full validator 7.4686 ms/op 8.4309 ms/op 0.89
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 76.276 us/op 83.828 us/op 0.91
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.1388 ms/op 1.2441 ms/op 0.92
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 14.946 ms/op 16.440 ms/op 0.91
BeaconState.hashTreeRoot - 1 balances 58.746 us/op 68.356 us/op 0.86
BeaconState.hashTreeRoot - 32 balances 570.55 us/op 626.15 us/op 0.91
BeaconState.hashTreeRoot - 512 balances 5.6809 ms/op 6.1474 ms/op 0.92
BeaconState.hashTreeRoot - 250000 balances 92.936 ms/op 100.10 ms/op 0.93
aggregationBits - 2048 els - zipIndexesInBitList 26.519 us/op 25.816 us/op 1.03
regular array get 100000 times 67.289 us/op 60.514 us/op 1.11
wrappedArray get 100000 times 67.307 us/op 60.525 us/op 1.11
arrayWithProxy get 100000 times 28.073 ms/op 26.601 ms/op 1.06
ssz.Root.equals 1.0740 us/op 923.00 ns/op 1.16
byteArrayEquals 1.0440 us/op 925.00 ns/op 1.13
shuffle list - 16384 els 11.231 ms/op 11.472 ms/op 0.98
shuffle list - 250000 els 166.02 ms/op 175.52 ms/op 0.95
processSlot - 1 slots 12.131 us/op 12.609 us/op 0.96
processSlot - 32 slots 1.7020 ms/op 1.9174 ms/op 0.89
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 336.30 us/op 316.53 us/op 1.06
getCommitteeAssignments - req 1 vs - 250000 vc 5.5992 ms/op 5.4614 ms/op 1.03
getCommitteeAssignments - req 100 vs - 250000 vc 7.5838 ms/op 7.9833 ms/op 0.95
getCommitteeAssignments - req 1000 vs - 250000 vc 8.1340 ms/op 8.4859 ms/op 0.96
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 9.5500 ns/op 7.3000 ns/op 1.31
state getBlockRootAtSlot - 250000 vs - 7PWei 1.1162 us/op 1.0608 us/op 1.05
computeProposers - vc 250000 15.618 ms/op 17.128 ms/op 0.91
computeEpochShuffling - vc 250000 170.94 ms/op 169.98 ms/op 1.01
getNextSyncCommittee - vc 250000 257.98 ms/op 282.24 ms/op 0.91

by benchmarkbot/action

@twoeths twoeths marked this pull request as ready for review December 18, 2022 04:01
@twoeths twoeths requested a review from a team as a code owner December 18, 2022 04:01
@wemeetagain wemeetagain merged commit 73e36a2 into unstable Dec 18, 2022
@wemeetagain wemeetagain deleted the tuyen/spec_tests_v1.3.0-alpha.2 branch December 18, 2022 04:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@twoeths @wemeetagain