-
-
Notifications
You must be signed in to change notification settings - Fork 290
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement support for validator next-epoch proposer duties #3782
Changes from 12 commits
9a0a792
5a1426a
bcdfbb6
4846918
18cde25
9bee893
5a632b1
0a1aba3
ac83ab8
fafca52
7d08ada
f0a9c3f
520c749
d54e124
02a722a
b7e526a
72fd6ea
12aa60d
b6b1b8c
c75d926
3c252a7
162a66b
8b58292
99e973f
be65566
ab35439
1ec1ccb
736cef7
9a51aa4
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -10,7 +10,7 @@ import { | |
readonlyValues, | ||
TreeBacked, | ||
} from "@chainsafe/ssz"; | ||
import {allForks, altair, Number64, ParticipationFlags} from "@chainsafe/lodestar-types"; | ||
import {allForks, altair, Number64, ParticipationFlags, ValidatorIndex} from "@chainsafe/lodestar-types"; | ||
import {createIBeaconConfig, IBeaconConfig, IChainForkConfig} from "@chainsafe/lodestar-config"; | ||
import {Tree} from "@chainsafe/persistent-merkle-tree"; | ||
import {MutableVector} from "@chainsafe/persistent-ts"; | ||
|
@@ -20,6 +20,7 @@ import {CachedEpochParticipation, CachedEpochParticipationProxyHandler} from "./ | |
import {ForkName} from "@chainsafe/lodestar-params"; | ||
import {CachedInactivityScoreList, CachedInactivityScoreListProxyHandler} from "./cachedInactivityScoreList"; | ||
import {newFilledArray} from "../util/array"; | ||
import {computeProposers} from "../util"; | ||
|
||
/** | ||
* `BeaconState` with various caches | ||
|
@@ -273,6 +274,22 @@ export class BeaconStateContext<T extends allForks.BeaconState> { | |
) as CachedBeaconState<T>; | ||
} | ||
|
||
getState(): allForks.BeaconState { | ||
return this.type.createTreeBacked(this.tree); | ||
} | ||
|
||
getNextEpochBeaconProposer(): ValidatorIndex[] { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is dangerous because it uses a state, which mutates every slot to mutate data in the epoch context, which is shared between multiple states. |
||
const nextShuffling = this.epochCtx.nextShuffling; | ||
let nextProposers = this.epochCtx.nextEpochProposers.get(nextShuffling.epoch); | ||
if (!nextProposers) { | ||
nextProposers = computeProposers(this.getState(), nextShuffling, this.epochCtx.effectiveBalanceIncrements); | ||
// Only keep proposers for one epoch in the future, so clear before setting new value | ||
this.epochCtx.nextEpochProposers.clear(); | ||
this.epochCtx.nextEpochProposers.set(nextShuffling.epoch, nextProposers); | ||
} | ||
return nextProposers; | ||
} | ||
|
||
/** | ||
* Toggle all `MutableVector` caches to use `TransientVector` | ||
*/ | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -103,6 +103,17 @@ export class EpochContext { | |
* 32 x Number | ||
*/ | ||
proposers: ValidatorIndex[]; | ||
|
||
/** | ||
* Map of Indexes of the block proposers keyed by the next epoch. | ||
* | ||
* We allow requesting proposal duties only one epoch in the future | ||
* Note: There is a small probability that returned validators differs | ||
* than what is returned when the epoch is reached. | ||
* | ||
* 32 x Number | ||
*/ | ||
nextEpochProposers: Map<Epoch, ValidatorIndex[]> = new Map<Epoch, ValidatorIndex[]>(); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Cache exactly 1 or 0 next proposers for the next epoch of this epochCtx. We should research the safety of +2, +3 epoch duties before generalizing. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Yeah. That is what is being done. Are you hinting at something else? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If there's exactly 1 possible value for that why do a map? nextEpochProposers: ValidatorIndex[] | null; is more intuitive |
||
/** | ||
* Shuffling of validator indexes. Immutable through the epoch, then it's replaced entirely. | ||
* Note: Per spec definition, shuffling will always be defined. They are never called before loadState() | ||
|
@@ -474,6 +485,13 @@ export class EpochContext { | |
`Requesting beacon proposer for different epoch current shuffling: ${epoch} != ${this.currentShuffling.epoch}` | ||
); | ||
} | ||
|
||
for (const cachedEpoch of this.nextEpochProposers.keys()) { | ||
// Do not keep past cached future proposal duties. | ||
if (cachedEpoch <= epoch) { | ||
this.nextEpochProposers.delete(cachedEpoch); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why is this necessary? Once you have computed nextEpochProposers entry it can be left there |
||
} | ||
} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Now computation of proposers is deferred AND cached when computed There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @dapplion this won't cache though, and on every request to get next proposers, the You can confirm this by running the application, and trigger the end point multiple times, the if branch will always run. I noticed this while working on the implementation and I think the reason why this is the case is how the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
If this is true, then it's a big issue! Can you open a dedicated issue for it? It should be investigated latter
This code runs on the main thread and according to benchmarks it takes ~100ms. That's still a significant amount of time to block the main thread and repeated computations must be avoided if possible There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. And please link back to this context after opening a new dedicated issue. :) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
return this.proposers[slot % SLOTS_PER_EPOCH]; | ||
} | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this necessary?
this
should already be treeBacked