VOM virtual object and vref weak key GC

[FUDCo]

No description provided.

[warner]

Good work! Some minor changes to make, as well as bringing it up-to-date with current trunk.

[warner]

The deadSet is managed both "up" and "down": we add things to deadSet when the finalizer runs, and we actively remove them (deadSet.delete) when creating a new Presence in convertValToSlot.

possiblyDeadSet is currently only managed "up": we add vrefs to it when the VOM sees one of the three pillars being removed, but we don't remove things from it when a pillar is reestablished (e.g. convertValToSlot, or VOM's setRefCount).

We don't necessarily need to do it now, but I'm thinking that, in general, deadSet should be for tracking things whose Object references (Representatives, for VOs) have been removed, and all the other processing for VOs should be done in the VOM.

The change would be to merge possiblyDeadSet and deadSet, actively .delete() vrefs when re-instantiating their Objects (actually we already do this, for both Presences and Representatives), and remove the now-redundant if (!getValForSlot(vref)) check in processDeadSet.

However, this probably interacts with what happens when the other two pillars are dropped. On one hand, if the only pillar keeping a VO alive is virtualized data, and userspace explicltly deletes it, that should allow the syscall.retireExport to happen immediately, rather than waiting for end-of-crank or bringOutYourDead. On the other hand, the probe to see whether the Presence pillar is up or down must query the WeakRef, and those results are not safe to use until bringOutYourDead (because organic GC could cause variance). So maybe the safest thing to do is to only allow VOs to be deleted during processDeadSet, even if it wasn't a Representative being deleted that enabled it, which means sticking with the pattern you established in this PR.

[mhofman]

On one hand, if the only pillar keeping a VO alive is virtualized data, and userspace explicltly deletes it, that should allow the syscall.retireExport to happen immediately, rather than waiting for end-of-crank or bringOutYourDead. On the other hand, the probe to see whether the Presence pillar is up or down must query the WeakRef, and those results are not safe to use until bringOutYourDead (because organic GC could cause variance). So maybe the safest thing to do is to only allow VOs to be deleted during processDeadSet, even if it wasn't a Representative being deleted that enabled it, which means sticking with the pattern you established in this PR.

I'm probably out of my depth here, but why can't the check for the Presence pillar rely on a flag "there is a known Presence, it may be dead, but since we haven't reaped it yet, we don't know", without checking for the WeakRef directly.
So there's be 3 states for presences:

• Live presence (or unreachable presence that hasn't been actively collected by the engine yet)
• Dead presence, but not yet reaped. The finalizer ran, and added it to the dead set
• Dead presence, reaped. BOYD was called.

If userspace deletes virtualized data, it would only immediately retire the export if the presence was reaped. If it wasn't reaped, the export retire would happen when BOYD is called.

[warner]

I've reviewed the tree that results from (this PR, plus #3732, plus #3968), and it looks good. It'll need rebasing, of course. Please use a merge commit when you're done, to preserve the stack of changes we've made (in this case, I think squashing would discard too much information).

[FUDCo]

Closes #3547