record worker bundleIDs in vat metadata, use during reload #7208
Comments
warner
changed the title
Merged
|
Does this include #7012? |
warner
added a commit
that referenced
this issue
Mar 30, 2023
When launching an xsnap worker, we must load two special bundles into the worker's empty JS engine: xsnap-lockdown (which turns it into a SES environment), and swingset-xsnap-supervisor (which loads liveslots and the command handlers). We get these bundles from packages of the same names. Previously, we would load the current versions of these bundles each time the controller/kernel started, and use them for all vats started or upgraded during that incarnation of the controller (i.e. until the process exited). This did not maintain stable behavior for a vat which e.g. does not record a heap snapshot before its initial controller incarnation finishes: if the xsnap-lockdown package were updated between controller incarnations, the second incarnation would launch the worker with a different bundle than the first, potentially causing divergence between the vat's original behavior (as recorded in its transcript) and the replay. If there were any local-manager liveslots -based vats, these would suffer the same problem on every kernel reboot, since only xsnap-hosted vats can use heap snapshots. With this commit, the lockdown and supervisor bundles are stored in the swing-store `bundleStore`, and their IDs are recorded in the metadata for each vat. We check the bundle-providing packages each time we start a new vat (or upgrade an existing one), so the new worker will use the latest bundles. But by recording the IDs separately for each vat, we ensure that every launch of that vat worker will the same bundles. This stability will help us maintain the deterministic behavior of a vat despite changes to the packages depended upon by any given version of the kernel. Unit tests and replay tools can use `options.overrideBundles=[..]` to replace the usual bundles with alternates: either no bundles, or custom lockdown/supervisor bundles. closes #7208
warner
added a commit
that referenced
this issue
Mar 30, 2023
When launching an xsnap worker, we must load two special bundles into the worker's empty JS engine: xsnap-lockdown (which turns it into a SES environment), and swingset-xsnap-supervisor (which loads liveslots and the command handlers). We get these bundles from packages of the same names. Previously, we would load the current versions of these bundles each time the controller/kernel started, and use them for all vats started or upgraded during that incarnation of the controller (i.e. until the process exited). This did not maintain stable behavior for a vat which e.g. does not record a heap snapshot before its initial controller incarnation finishes: if the xsnap-lockdown package were updated between controller incarnations, the second incarnation would launch the worker with a different bundle than the first, potentially causing divergence between the vat's original behavior (as recorded in its transcript) and the replay. If there were any local-manager liveslots -based vats, these would suffer the same problem on every kernel reboot, since only xsnap-hosted vats can use heap snapshots. With this commit, the lockdown and supervisor bundles are stored in the swing-store `bundleStore`, and their IDs are recorded in the metadata for each vat. We check the bundle-providing packages each time we start a new vat (or upgrade an existing one), so the new worker will use the latest bundles. But by recording the IDs separately for each vat, we ensure that every launch of that vat worker will the same bundles. This stability will help us maintain the deterministic behavior of a vat despite changes to the packages depended upon by any given version of the kernel. Unit tests and replay tools can use `options.overrideBundles=[..]` to replace the usual bundles with alternates: either no bundles, or custom lockdown/supervisor bundles. closes #7208
warner
added a commit
that referenced
this issue
Mar 30, 2023
When launching an xsnap worker, we must load two special bundles into the worker's empty JS engine: xsnap-lockdown (which turns it into a SES environment), and swingset-xsnap-supervisor (which loads liveslots and the command handlers). We get these bundles from packages of the same names. Previously, we would load the current versions of these bundles each time the controller/kernel started, and use them for all vats started or upgraded during that incarnation of the controller (i.e. until the process exited). This did not maintain stable behavior for a vat which e.g. does not record a heap snapshot before its initial controller incarnation finishes: if the xsnap-lockdown package were updated between controller incarnations, the second incarnation would launch the worker with a different bundle than the first, potentially causing divergence between the vat's original behavior (as recorded in its transcript) and the replay. If there were any local-manager liveslots -based vats, these would suffer the same problem on every kernel reboot, since only xsnap-hosted vats can use heap snapshots. With this commit, the lockdown and supervisor bundles are stored in the swing-store `bundleStore`, and their IDs are recorded in the metadata for each vat. We check the bundle-providing packages each time we start a new vat (or upgrade an existing one), so the new worker will use the latest bundles. But by recording the IDs separately for each vat, we ensure that every launch of that vat worker will the same bundles. This stability will help us maintain the deterministic behavior of a vat despite changes to the packages depended upon by any given version of the kernel. Unit tests and replay tools can use `options.overrideBundles=[..]` to replace the usual bundles with alternates: either no bundles, or custom lockdown/supervisor bundles. closes #7208
Nope. It records and retains the xsnap-lockdown and swingset-xsnap-supervisor bundles (whatever is present in the corresponding dependencies under We'll have to refrain from changing the During that window, we'll probably implement some sort of metadata flag to say "use xsnap-1.x", and we'll figure out some sort of aliased dependency so that "1.x" can coexist with a subsequent "2.x". We won't have that metadata in the bulldozer-created vats, but it wants to live in |
warner
added a commit
that referenced
this issue
Mar 30, 2023
When launching an xsnap worker, we must load two special bundles into the worker's empty JS engine: xsnap-lockdown (which turns it into a SES environment), and swingset-xsnap-supervisor (which loads liveslots and the command handlers). We get these bundles from packages of the same names. Previously, we would load the current versions of these bundles each time the controller/kernel started, and use them for all vats started or upgraded during that incarnation of the controller (i.e. until the process exited). This did not maintain stable behavior for a vat which e.g. does not record a heap snapshot before its initial controller incarnation finishes: if the xsnap-lockdown package were updated between controller incarnations, the second incarnation would launch the worker with a different bundle than the first, potentially causing divergence between the vat's original behavior (as recorded in its transcript) and the replay. If there were any local-manager liveslots -based vats, these would suffer the same problem on every kernel reboot, since only xsnap-hosted vats can use heap snapshots. With this commit, the lockdown and supervisor bundles are stored in the swing-store `bundleStore`, and their IDs are recorded in the metadata for each vat. We check the bundle-providing packages each time we start a new vat (or upgrade an existing one), so the new worker will use the latest bundles. But by recording the IDs separately for each vat, we ensure that every launch of that vat worker will the same bundles. This stability will help us maintain the deterministic behavior of a vat despite changes to the packages depended upon by any given version of the kernel. Unit tests and replay tools can use `options.overrideBundles=[..]` to replace the usual bundles with alternates: either no bundles, or custom lockdown/supervisor bundles. closes #7208
warner
added a commit
that referenced
this issue
Apr 4, 2023
When launching an xsnap worker, we must load two special bundles into the worker's empty JS engine: xsnap-lockdown (which turns it into a SES environment), and swingset-xsnap-supervisor (which loads liveslots and the command handlers). We get these bundles from packages of the same names. Previously, we would load the current versions of these bundles each time the controller/kernel started, and use them for all vats started or upgraded during that incarnation of the controller (i.e. until the process exited). This did not maintain stable behavior for a vat which e.g. does not record a heap snapshot before its initial controller incarnation finishes: if the xsnap-lockdown package were updated between controller incarnations, the second incarnation would launch the worker with a different bundle than the first, potentially causing divergence between the vat's original behavior (as recorded in its transcript) and the replay. If there were any local-manager liveslots -based vats, these would suffer the same problem on every kernel reboot, since only xsnap-hosted vats can use heap snapshots. With this commit, the lockdown and supervisor bundles are stored in the swing-store `bundleStore`, and their IDs are recorded in the metadata for each vat. We check the bundle-providing packages each time we start a new vat (or upgrade an existing one), so the new worker will use the latest bundles. But by recording the IDs separately for each vat, we ensure that every launch of that vat worker will the same bundles. This stability will help us maintain the deterministic behavior of a vat despite changes to the packages depended upon by any given version of the kernel. Unit tests and replay tools can use `options.overrideBundles=[..]` to replace the usual bundles with alternates: either no bundles, or custom lockdown/supervisor bundles. closes #7208
warner
added a commit
that referenced
this issue
Apr 5, 2023
When launching an xsnap worker, we must load two special bundles into the worker's empty JS engine: xsnap-lockdown (which turns it into a SES environment), and swingset-xsnap-supervisor (which loads liveslots and the command handlers). We get these bundles from packages of the same names. Previously, we would load the current versions of these bundles each time the controller/kernel started, and use them for all vats started or upgraded during that incarnation of the controller (i.e. until the process exited). This did not maintain stable behavior for a vat which e.g. does not record a heap snapshot before its initial controller incarnation finishes: if the xsnap-lockdown package were updated between controller incarnations, the second incarnation would launch the worker with a different bundle than the first, potentially causing divergence between the vat's original behavior (as recorded in its transcript) and the replay. If there were any local-manager liveslots -based vats, these would suffer the same problem on every kernel reboot, since only xsnap-hosted vats can use heap snapshots. With this commit, the lockdown and supervisor bundles are stored in the swing-store `bundleStore`, and their IDs are recorded in the metadata for each vat. We check the bundle-providing packages each time we start a new vat (or upgrade an existing one), so the new worker will use the latest bundles. But by recording the IDs separately for each vat, we ensure that every launch of that vat worker will the same bundles. This stability will help us maintain the deterministic behavior of a vat despite changes to the packages depended upon by any given version of the kernel. Unit tests and replay tools can use `options.overrideBundles=[..]` to replace the usual bundles with alternates: either no bundles, or custom lockdown/supervisor bundles. closes #7208
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Here's the new plan, to replace the #6596 of making a new package for the stable "worker-v1" behavior.
swingset-xsnap-supervisor(extract swingset-xsnap-supervisor out to a separate package #7063), but abandonswingset-worker-xsnap-v1(create new swingset-worker-xsnap-v1 package #7101)bundleStoreto acceptnestedEvaluatebundles too (with an ID that's probably the hash of the JSON encoding of the whole bundle object) (make bundleStore accept nestedEvaluate bundles too #7190)bundleStoreunder their hash IDs, and add a pair ofkvStorekeys to remind the kernel of what the "current lockdown/supervisor bundle ID" areoptions)bundleStoreand feed them to the new worker, instead of using fixed copiesOriginally posted by @warner in #6596 (comment)
The text was updated successfully, but these errors were encountered: