Periodic full garbage collection of resolved promises

[FUDCo]

Paraphrased from the discussion in #1049:

Something more clever:

This would be the part where we add a new syscall and/or dispatch method, and have liveslots accumulate the slots to drop at the end of the crank. This would be maybe a 98% case: it covers more situations, but we still need improved GC to release everything that's no longer reachable.

@warner is keen on making the drops be explicit. His thought is that which slots to drop is a policy decision, made based upon various heuristics about utility/performance-savings and soundness, and that explicit syscalls would make the invariants easier to maintain (if the kernel only drops entries in response to a syscall.dropSlots, the kernel and vats tables should always be aligned, even if the vat makes weird choices as to when to exercise its slot-dropping power). It'd be a bit weird if/because both sides get to drop things: the vat would call syscall.dropSlots after syscall.fulfill* (and the kernel should modify the c-lists to match), but on the receiving side maybe the kernel would direct the vat to delete entries after/during dispatch.notifyFulfill*. Maybe we pick one side or the other, and both sending and receiving result in e.g. the vat calling syscall.dropSlots. This will probably be easier to think about once we know how GC is going to work, where we don't want strong references point in both directions all the time.

Dean felt it was better to use implicit/inferred drops for the easy cases of resolved promises, and explicit calls for the more complicated cases we figure out later. He was concerned about the performance costs of having more syscalls, as well as what I think of as "debuggor ergonomics": how hard is it for the human (the "debuggor", you know, like "operator") performing the task of debugging (using a "debugger", the assistant software) to focus on the data they need. More syscalls means more noise in the logs, more things to ignore, more single-step iterations to get through to the important parts, etc.

I (@FUDCo) am inclined towards a simple mark-and-sweep garbage collector on the kernel side to deal with what are from the kernel's perspective unreferenced cycles, and simply have vats re-export promises in the rare cases that is needed.

This ticket is now specifically about @FUDCo 's last point: something in the kernel to deal with collecting cycles that include resolved promises.

[warner]

After closing #1124, this remaining ticket is about resolved promise cycles, mainly in the kernel. Two entries in the kernel promise table might both be resolved, and their resolution data might point to each other, and the reference counts on each are sufficient to protect them from collection by the "refcount just went to zero" logic.

Two ideas we've considered:

• on some long-duration timer, conduct a full mark-and-sweep of the whole promise table, to find the unreferenced ones and remove them
• have an external source (with more time/CPU on its hands) submit a "proposed cycle removal" instruction, which names a set of kernel promise IDs. The kernel performs a (revertable) crank in which it deletes the outbound references from their resolution data, then checks the refcounts on all promises mentioned in the cycle. If all are now zero, the removal is allowed to proceed (deleting now-unreferenced leaves that weren't part of the cycle but were kept alive by it), and everything is committed. If not, the changes are reverted.
  • on the chain, this would need to be submitted as a transaction, with some fee (to protect against invalid claims), and a reward (to incentivize people doing this work)

This doesn't cover cross-chain cycles, for which we'll need some science. It also doesn't handle cycles that cross vats within a single kernel, and I have even less idea how to close those (since they'll involve real Promise objects within each vat, and I don't think we get a lot of visibility into how they are used/not-used by the JS side).

[mhofman]

Wouldn't promise redirection solve promise cycles?