feat(miner): add miner deadline diffing logic. #4178
Conversation
frrist
commented
Oct 5, 2020
frrist
commented
- logic gathering all removed, faulted, recovered, and recovering sectorIDs.
| } | ||
| dlDiff := make(DeadlinesDiff, numDl) | ||
| if err := pre.ForEachDeadline(func(idx uint64, preDl Deadline) error { | ||
| curDl, err := cur.LoadDeadline(idx) |
There was a problem hiding this comment.
This is, unfortunately, going to be a bit slow as it has to keep loading the deadline object.
It's probably better to just load all deadlines into two arrays, then compare them.
There was a problem hiding this comment.
Really low priority (only if it becomes an issue).
| partDiff := make(DeadlineDiff) | ||
| if err := pre.ForEachPartition(func(idx uint64, prePart Partition) error { | ||
| // try loading current partition at this index | ||
| curPart, err := cur.LoadPartition(idx) |
There was a problem hiding this comment.
Ditto my comment above about loading into arrays, also really low priority.
|
|
||
| // all previous partitions have been walked. | ||
| // all partitions in cur and not in prev are new... can they be faulty already? | ||
| // TODO is this correct? |
There was a problem hiding this comment.
I believe you're right that there can't be any faults, but I'm not 100% sure. We should be allocating new partitions in ConfirmSectorProofsValid, which should happen during cron (after all fault messages have been processed).
| return nil, err | ||
| } | ||
|
|
||
| // all current good sectors |
There was a problem hiding this comment.
This code is correct. However, note: "active" means actively contributing towards power. In actors v2, this won't include new but not-yet-posted sectors.
But that doesn't matter for this code recovered sectors will always be "posted" (i.e., we'll submit a post to recover them).
| if err != nil { | ||
| return nil, err | ||
| } | ||
| dlDiff := make(DeadlinesDiff, numDl) |
There was a problem hiding this comment.
do we care about cases where a new deadline has been added in cur that was not in pre, or vice-versa?
Is there a reason to make dlDiff an array of all deadlines (with nil entries for those that have not changed) rather than a map of index->diff? if we expect sparse deadline changes, that might be a more natural representation.
There was a problem hiding this comment.
do we care about cases where a new deadline has been added in cur that was not in pre, or vice-versa?
My understanding (is somewhat incomplete here, but I think) since Deadlines are a fixed size it will never be the case that there exists a deadline in cur and not pre, or vice-versa, meaning we will walk each deadline on calls to ForEachDeadline. I suppose there could be an edge case here if we update the miner actor with a larger deadlines array and this code is called at the upgrade boundary -- I will leave a comment ackonwledging this as I am unsure how to address it...
I could be the case (perhaps this is what was meant) that there exists an empty deadline (a deadline with no partitions) in cur or pre, but I think this addressed by walking both cur and pre in the DiffDeadline method.
Is there a reason to make dlDiff an array of all deadlines (with nil entries for those that have not changed) rather than a map of index->diff? if we expect sparse deadline changes, that might be a more natural representation.
I don't have a strong reason for it, and agree a map would be better choice -- will address.
There was a problem hiding this comment.
the default case is that fixed-size list (all non-populated entries) are the same 'empty deadline', e.g. https://decentralized.app/?state=WyJiYWZ5MmJ6YWNlZHpqejJqZW94aHRoY3lyNDRxd3lmc2kyeHBmcnpsM2dmZDJjNDZoN3lqcG9lNDJ6bnQzNiIsW1swLDAsMCwwLDAsMCwwLCIiLDAse30sWzAseyI2Ijp7fX1dLHt9LHt9LHt9LHt9LHt9LHt9LHt9LHt9XV0sMCwwXQ==#storageMinerActor.Head.Deadlines.Due
- logic gathering all removed, faulted, recovered, and recovering sectors for a miner.
frrist
force-pushed
the
frrist/miner-diff-deadlines
branch
from
45cad54
to
d4cbd4f
Compare
