Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix deserializeEnrSubnets #3956

Merged
merged 1 commit into from
Apr 28, 2022
Merged

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Apr 27, 2022

Motivation

Lodestar node is not able to connect to an attestation subnet peer because deserializeEnrSubnets always return an empty array

Description

Fix deserializeEnrSubnets

closes #3940

@twoeths twoeths requested a review from a team as a code owner April 27, 2022 09:54
@codecov
Copy link

codecov bot commented Apr 27, 2022

Codecov Report

Merging #3956 (a176e69) into master (97f6012) will decrease coverage by 0.06%.
The diff coverage is n/a.

❗ Current head a176e69 differs from pull request most recent head a0a3c7f. Consider uploading reports for the commit a0a3c7f to get more accurate results

@@            Coverage Diff             @@
##           master    #3956      +/-   ##
==========================================
- Coverage   36.33%   36.26%   -0.07%     
==========================================
  Files         324      324              
  Lines        9063     9099      +36     
  Branches     1455     1465      +10     
==========================================
+ Hits         3293     3300       +7     
- Misses       5597     5626      +29     
  Partials      173      173              

@github-actions
Copy link
Contributor

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: a176e69 Previous: 3a38e65 Ratio
BeaconState.hashTreeRoot - No change 624.00 ns/op 515.00 ns/op 1.21
BeaconState.hashTreeRoot - 1 full validator 61.486 us/op 64.641 us/op 0.95
BeaconState.hashTreeRoot - 32 full validator 625.04 us/op 642.38 us/op 0.97
BeaconState.hashTreeRoot - 512 full validator 7.1449 ms/op 7.1703 ms/op 1.00
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 83.073 us/op 82.347 us/op 1.01
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.1304 ms/op 1.1472 ms/op 0.99
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 15.779 ms/op 15.372 ms/op 1.03
BeaconState.hashTreeRoot - 1 balances 64.580 us/op 64.926 us/op 0.99
BeaconState.hashTreeRoot - 32 balances 586.53 us/op 556.33 us/op 1.05
BeaconState.hashTreeRoot - 512 balances 5.4193 ms/op 5.5525 ms/op 0.98
BeaconState.hashTreeRoot - 250000 balances 123.83 ms/op 111.89 ms/op 1.11
processSlot - 1 slots 10.741 us/op 11.465 us/op 0.94
processSlot - 32 slots 1.6936 ms/op 1.8398 ms/op 0.92
getCommitteeAssignments - req 1 vs - 250000 vc 5.9436 ms/op 6.2221 ms/op 0.96
getCommitteeAssignments - req 100 vs - 250000 vc 8.1033 ms/op 8.6839 ms/op 0.93
getCommitteeAssignments - req 1000 vs - 250000 vc 8.4338 ms/op 9.4079 ms/op 0.90
computeProposers - vc 250000 18.029 ms/op 19.095 ms/op 0.94
computeEpochShuffling - vc 250000 171.85 ms/op 189.48 ms/op 0.91
getNextSyncCommittee - vc 250000 290.48 ms/op 313.56 ms/op 0.93
altair processAttestation - 250000 vs - 7PWei normalcase 3.7839 ms/op 4.2494 ms/op 0.89
altair processAttestation - 250000 vs - 7PWei worstcase 5.3420 ms/op 6.1814 ms/op 0.86
altair processAttestation - setStatus - 1/6 committees join 209.01 us/op 228.00 us/op 0.92
altair processAttestation - setStatus - 1/3 committees join 369.43 us/op 458.36 us/op 0.81
altair processAttestation - setStatus - 1/2 committees join 597.62 us/op 623.29 us/op 0.96
altair processAttestation - setStatus - 2/3 committees join 796.53 us/op 787.83 us/op 1.01
altair processAttestation - setStatus - 4/5 committees join 989.68 us/op 1.0654 ms/op 0.93
altair processAttestation - setStatus - 100% committees join 1.2681 ms/op 1.3029 ms/op 0.97
altair processBlock - 250000 vs - 7PWei normalcase 30.526 ms/op 26.529 ms/op 1.15
altair processBlock - 250000 vs - 7PWei normalcase hashState 36.735 ms/op 39.785 ms/op 0.92
altair processBlock - 250000 vs - 7PWei worstcase 76.952 ms/op 84.719 ms/op 0.91
altair processBlock - 250000 vs - 7PWei worstcase hashState 112.87 ms/op 126.77 ms/op 0.89
altair processEth1Data - 250000 vs - 7PWei normalcase 728.25 us/op 858.19 us/op 0.85
altair processEpoch - mainnet_e81889 604.31 ms/op 635.32 ms/op 0.95
mainnet_e81889 - altair beforeProcessEpoch 146.78 ms/op 150.08 ms/op 0.98
mainnet_e81889 - altair processJustificationAndFinalization 53.091 us/op 39.558 us/op 1.34
mainnet_e81889 - altair processInactivityUpdates 11.397 ms/op 12.486 ms/op 0.91
mainnet_e81889 - altair processRewardsAndPenalties 91.635 ms/op 150.71 ms/op 0.61
mainnet_e81889 - altair processRegistryUpdates 8.8210 us/op 11.038 us/op 0.80
mainnet_e81889 - altair processSlashings 3.9450 us/op 4.0510 us/op 0.97
mainnet_e81889 - altair processEth1DataReset 4.3530 us/op 4.3170 us/op 1.01
mainnet_e81889 - altair processEffectiveBalanceUpdates 7.2587 ms/op 7.6369 ms/op 0.95
mainnet_e81889 - altair processSlashingsReset 13.057 us/op 13.343 us/op 0.98
mainnet_e81889 - altair processRandaoMixesReset 15.424 us/op 17.758 us/op 0.87
mainnet_e81889 - altair processHistoricalRootsUpdate 5.6380 us/op 5.7100 us/op 0.99
mainnet_e81889 - altair processParticipationFlagUpdates 13.746 us/op 7.6780 us/op 1.79
mainnet_e81889 - altair processSyncCommitteeUpdates 2.5260 us/op 2.2730 us/op 1.11
mainnet_e81889 - altair afterProcessEpoch 221.09 ms/op 218.12 ms/op 1.01
altair processInactivityUpdates - 250000 normalcase 44.306 ms/op 43.817 ms/op 1.01
altair processInactivityUpdates - 250000 worstcase 31.948 ms/op 31.448 ms/op 1.02
altair processRewardsAndPenalties - 250000 normalcase 126.13 ms/op 110.72 ms/op 1.14
altair processRewardsAndPenalties - 250000 worstcase 112.69 ms/op 86.822 ms/op 1.30
altair processSyncCommitteeUpdates - 250000 282.37 ms/op 336.67 ms/op 0.84
Tree 40 250000 create 600.44 ms/op 748.69 ms/op 0.80
Tree 40 250000 get(125000) 270.61 ns/op 341.95 ns/op 0.79
Tree 40 250000 set(125000) 1.9686 us/op 2.5387 us/op 0.78
Tree 40 250000 toArray() 30.615 ms/op 37.644 ms/op 0.81
Tree 40 250000 iterate all - toArray() + loop 30.086 ms/op 38.830 ms/op 0.77
Tree 40 250000 iterate all - get(i) 113.14 ms/op 134.52 ms/op 0.84
MutableVector 250000 create 14.945 ms/op 19.960 ms/op 0.75
MutableVector 250000 get(125000) 14.074 ns/op 15.617 ns/op 0.90
MutableVector 250000 set(125000) 544.13 ns/op 600.88 ns/op 0.91
MutableVector 250000 toArray() 5.6952 ms/op 7.3151 ms/op 0.78
MutableVector 250000 iterate all - toArray() + loop 6.3081 ms/op 7.7414 ms/op 0.81
MutableVector 250000 iterate all - get(i) 3.5661 ms/op 4.0741 ms/op 0.88
Array 250000 create 6.3303 ms/op 6.1366 ms/op 1.03
Array 250000 clone - spread 2.7241 ms/op 2.9044 ms/op 0.94
Array 250000 get(125000) 1.2640 ns/op 1.3380 ns/op 0.94
Array 250000 set(125000) 1.2060 ns/op 1.3880 ns/op 0.87
Array 250000 iterate all - loop 188.04 us/op 198.97 us/op 0.95
effectiveBalanceIncrements clone Uint8Array 300000 83.368 us/op 83.673 us/op 1.00
effectiveBalanceIncrements clone MutableVector 300000 878.00 ns/op 610.00 ns/op 1.44
effectiveBalanceIncrements rw all Uint8Array 300000 312.56 us/op 356.24 us/op 0.88
effectiveBalanceIncrements rw all MutableVector 300000 176.89 ms/op 167.80 ms/op 1.05
aggregationBits - 2048 els - zipIndexesInBitList 27.163 us/op 28.094 us/op 0.97
regular array get 100000 times 69.428 us/op 79.469 us/op 0.87
wrappedArray get 100000 times 74.870 us/op 78.643 us/op 0.95
arrayWithProxy get 100000 times 39.143 ms/op 41.988 ms/op 0.93
ssz.Root.equals 562.00 ns/op 523.00 ns/op 1.07
byteArrayEquals 524.00 ns/op 487.00 ns/op 1.08
phase0 processBlock - 250000 vs - 7PWei normalcase 4.4296 ms/op 3.8850 ms/op 1.14
phase0 processBlock - 250000 vs - 7PWei worstcase 57.206 ms/op 53.747 ms/op 1.06
phase0 afterProcessEpoch - 250000 vs - 7PWei 210.50 ms/op 205.45 ms/op 1.02
phase0 beforeProcessEpoch - 250000 vs - 7PWei 80.039 ms/op 80.800 ms/op 0.99
phase0 processEpoch - mainnet_e58758 559.98 ms/op 646.59 ms/op 0.87
mainnet_e58758 - phase0 beforeProcessEpoch 231.34 ms/op 252.13 ms/op 0.92
mainnet_e58758 - phase0 processJustificationAndFinalization 38.699 us/op 35.294 us/op 1.10
mainnet_e58758 - phase0 processRewardsAndPenalties 78.761 ms/op 121.53 ms/op 0.65
mainnet_e58758 - phase0 processRegistryUpdates 18.087 us/op 17.686 us/op 1.02
mainnet_e58758 - phase0 processSlashings 3.5690 us/op 3.0670 us/op 1.16
mainnet_e58758 - phase0 processEth1DataReset 3.4650 us/op 2.8030 us/op 1.24
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 6.2419 ms/op 6.8900 ms/op 0.91
mainnet_e58758 - phase0 processSlashingsReset 12.080 us/op 9.4000 us/op 1.29
mainnet_e58758 - phase0 processRandaoMixesReset 13.943 us/op 11.766 us/op 1.19
mainnet_e58758 - phase0 processHistoricalRootsUpdate 4.2280 us/op 3.5840 us/op 1.18
mainnet_e58758 - phase0 processParticipationRecordUpdates 10.005 us/op 10.193 us/op 0.98
mainnet_e58758 - phase0 afterProcessEpoch 177.10 ms/op 181.99 ms/op 0.97
phase0 processEffectiveBalanceUpdates - 250000 normalcase 7.3110 ms/op 7.8437 ms/op 0.93
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 7.8276 ms/op 8.1090 ms/op 0.97
phase0 processRegistryUpdates - 250000 normalcase 17.464 us/op 14.359 us/op 1.22
phase0 processRegistryUpdates - 250000 badcase_full_deposits 509.83 us/op 500.82 us/op 1.02
phase0 processRegistryUpdates - 250000 worstcase 0.5 256.36 ms/op 252.53 ms/op 1.02
phase0 getAttestationDeltas - 250000 normalcase 16.504 ms/op 17.803 ms/op 0.93
phase0 getAttestationDeltas - 250000 worstcase 16.873 ms/op 17.496 ms/op 0.96
phase0 processSlashings - 250000 worstcase 6.6508 ms/op 7.6820 ms/op 0.87
shuffle list - 16384 els 13.022 ms/op 13.007 ms/op 1.00
shuffle list - 250000 els 183.53 ms/op 184.64 ms/op 0.99
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 406.83 us/op 449.71 us/op 0.90
pass gossip attestations to forkchoice per slot 3.9368 ms/op 4.0336 ms/op 0.98
computeDeltas 3.5391 ms/op 3.6886 ms/op 0.96
computeProposerBoostScoreFromBalances 573.17 us/op 603.22 us/op 0.95
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.8822 ms/op 2.3653 ms/op 0.80
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 83.567 us/op 86.534 us/op 0.97
BLS verify - blst-native 2.1233 ms/op 2.2940 ms/op 0.93
BLS verifyMultipleSignatures 3 - blst-native 4.3900 ms/op 4.6949 ms/op 0.94
BLS verifyMultipleSignatures 8 - blst-native 9.3009 ms/op 10.033 ms/op 0.93
BLS verifyMultipleSignatures 32 - blst-native 34.163 ms/op 35.613 ms/op 0.96
BLS aggregatePubkeys 32 - blst-native 44.949 us/op 47.308 us/op 0.95
BLS aggregatePubkeys 128 - blst-native 175.59 us/op 184.31 us/op 0.95
getAttestationsForBlock 66.693 ms/op 72.422 ms/op 0.92
CheckpointStateCache - add get delete 11.309 us/op 13.046 us/op 0.87
validate gossip signedAggregateAndProof - struct 4.7459 ms/op 5.2690 ms/op 0.90
validate gossip attestation - struct 2.3228 ms/op 2.4699 ms/op 0.94
pickEth1Vote - no votes 2.4444 ms/op 2.5542 ms/op 0.96
pickEth1Vote - max votes 19.429 ms/op 22.082 ms/op 0.88
pickEth1Vote - Eth1Data hashTreeRoot value x2048 11.801 ms/op 12.912 ms/op 0.91
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 21.532 ms/op 22.336 ms/op 0.96
pickEth1Vote - Eth1Data fastSerialize value x2048 2.0858 ms/op 1.9584 ms/op 1.07
pickEth1Vote - Eth1Data fastSerialize tree x2048 12.749 ms/op 15.939 ms/op 0.80
bytes32 toHexString 1.0500 us/op 1.1000 us/op 0.95
bytes32 Buffer.toString(hex) 766.00 ns/op 830.00 ns/op 0.92
bytes32 Buffer.toString(hex) from Uint8Array 1.0240 us/op 1.1440 us/op 0.90
bytes32 Buffer.toString(hex) + 0x 790.00 ns/op 843.00 ns/op 0.94
Object access 1 prop 0.38100 ns/op 0.41000 ns/op 0.93
Map access 1 prop 0.33700 ns/op 0.34800 ns/op 0.97
Object get x1000 20.934 ns/op 20.989 ns/op 1.00
Map get x1000 1.1840 ns/op 1.3240 ns/op 0.89
Object set x1000 120.88 ns/op 128.11 ns/op 0.94
Map set x1000 76.286 ns/op 79.968 ns/op 0.95
Return object 10000 times 0.46060 ns/op 0.45310 ns/op 1.02
Throw Error 10000 times 7.2560 us/op 7.3177 us/op 0.99
enrSubnets - fastDeserialize 64 bits 2.9670 us/op 1.0540 us/op 2.81
enrSubnets - ssz BitVector 64 bits 859.00 ns/op 940.00 ns/op 0.91
enrSubnets - fastDeserialize 4 bits 488.00 ns/op 479.00 ns/op 1.02
enrSubnets - ssz BitVector 4 bits 812.00 ns/op 920.00 ns/op 0.88
RateTracker 1000000 limit, 1 obj count per request 205.35 ns/op 225.97 ns/op 0.91
RateTracker 1000000 limit, 2 obj count per request 148.73 ns/op 164.86 ns/op 0.90
RateTracker 1000000 limit, 4 obj count per request 127.98 ns/op 135.64 ns/op 0.94
RateTracker 1000000 limit, 8 obj count per request 109.97 ns/op 123.30 ns/op 0.89
RateTracker with prune 4.0420 us/op 4.6410 us/op 0.87
array of 16000 items push then shift 3.6177 us/op 3.7832 us/op 0.96
LinkedList of 16000 items push then shift 18.258 ns/op 20.103 ns/op 0.91
array of 16000 items push then pop 222.95 ns/op 240.62 ns/op 0.93
LinkedList of 16000 items push then pop 17.167 ns/op 17.878 ns/op 0.96
array of 24000 items push then shift 5.1155 us/op 5.3478 us/op 0.96
LinkedList of 24000 items push then shift 20.033 ns/op 22.135 ns/op 0.91
array of 24000 items push then pop 192.16 ns/op 222.45 ns/op 0.86
LinkedList of 24000 items push then pop 17.984 ns/op 19.022 ns/op 0.95

by benchmarkbot/action

@wemeetagain wemeetagain merged commit 2b9fe2a into master Apr 28, 2022
@wemeetagain wemeetagain deleted the tuyen/fix-deserializeEnrSubnets branch April 28, 2022 09:33
const bytesBuf = Buffer.from(bytes, "hex");
it(`Deserialize attnet ${bytes}`, () => {
expect(
deserializeEnrSubnets(bytesBuf, ATTESTATION_SUBNET_COUNT).slice(0, ATTESTATION_SUBNET_COUNT)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seeing this has already been merged, this is more of a question...is the slice needed in the test? I removed it, that is:

expect(deserializeEnrSubnets(bytesBuf, ATTESTATION_SUBNET_COUNT)).to.deep.equal(bools)

and the tests still passed alright.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yaah it's not really needed, it's only needed for sync committee, the test above I guess. However the test overall still serves our purpose.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

PeerManager failed to connect to subnet peers
3 participants