Integrate SharedArrayBuffers #2260
Comments
|
Another problem is that we haven't clearly defined "potential" process boundaries. You can |
|
The addenda for DOM are in a slight state of disrepair, sadly. The autoritative description is supposed to be here: http://tc39.github.io/ecmascript_sharedmem/dom_shmem.html but it is not linked to from anywhere; I'll fix that immediately and make sure it's quite visible. The formatting also seems completely broken; I'll fix that too. We removed transfering because that seemed to be the consensus among the two of you (Domenic and Anne) and that has now propagated into documentation, examples, blog posts, test suites, and implementations, so I sure hope nobody's having cold feet about that :-) I can only echo Anne on process boundaries, this area feels pretty squishy right now. In Firefox I'm working on making it impossible to share memory except between a dedicated worker and its parent, sibling, or descendant worker, and between any of those workers and the parent document. This is motivated by process boundaries (we'll assume dedicated workers are in-process.) Ideally we'd allow for using message channels in addition to direct postMessage, and to allow for sharing between a sharedworker and the dedicated workers it owns, but I don't know what's possible for the first cut. |
Sorry, I was being imprecise; I meant cloning :). I'll edit the OP. The process boundaries thing is pretty interesting; sounds like I can help try to nail that down. I think the spec has some of this machinery in place since every worker is associated with a "responsible browsing context" (i.e. a tab or iframe). Is there cross-vendor consensus on allowing sharing between the following? (Based on your list):
Notably this includes "uncle" workers, not just "sibling" workers, but probably you meant to include those? Also it does not allow sharing between a Window and its iframes. Is that OK? We should be able to get MessageChannel and any other structured-clone using things for free from the spec. Shall we leave SharedWorkers out of the first cut, or should we take a crack at those too? Would love to hear from other vendors here on this process boundary question, so anything you can do to get them to comment would be helpful. |
|
Yes, I meant the closure of dedicated workers and browsing context, so uncle workers and all sorts of cousins n times removed.
It's a little open in my mind (because this is not my usual field of operations) how we handle the situation of an iframe within a window, when those have compatible security domains. The interesting case is if the iframe (say) cannot be prevented from sharing a SharedArrayBuffer it creates with its window, eg by storing it in a global variable in the window, or similar. What is the processing model here? Can the window and the iframe have different event loops, or are they defined to share one? I assume the latter, if they can each update each others' globals, but clarification is welcome. And anyway, if that kind of sharing behavior is possible, how can we reasonably prevent it, should we wish to? The issue with MessageChannel has been that when a message is sent containing a SAB, the other end of the channel may not be connected to anything and it's only on receipt that we can check permissions. IIUC. I'm happy to leave SharedWorker out of it for now, for simplicity. Cc'ing some other vendors: @binji, @pizlonator, @akroshg |
That's a great point. I don't think we can realistically prevent it without some intrusive and weird caller-realm checks or similar. So I think we want to allow sharing within a unit of related similar-origin browsing contexts, which is precisely the set of windows that could reach each other synchronously. And I guess then we take the transitive closure of this with the reachable workers, too. So the current proposal is:
|
This would work if we have defined serialization and deserialization algorithms, right? And defined what would happen if deserialization failed? (As we probably need to do anyway, since deserialization can fail simply due to alloc, for which normal ArrayBuffer objects already need to throw.) |
|
@annevk: yes. |
This seems correct and is probably sufficient. "Sharing an event loop" does not actually force same-process since this is probably just a requirement that at most one loop runs at a time, but if these documents can share data by storing them in each others' globals then same-process is much more plausible. (Proxies on the shared data are still possible to enable not-shared-process I guess.) I'll ask our multiprocess people what they think about this. |
|
@domenic, I asked our multiprocess people about the UORSOBC, here's what I heard back:
It sounds like we have a place to stand, now we just need the details :) |
This rewrites most of the cloneable and transferable object infrastructure to better reflect the reality that structured cloning requires separate serialization and deserialization steps, instead of a single operation that creates a new object in the target Realm. This is most evident in the case of MessagePorts, as noted in #2277. It also allows us to avoid awkward double-cloning with an intermediate "user-agent defined Realm", as seen in e.g. history.state or IndexedB; instead we can simply store the serialized form and later deserialize. Concretely, this: * Replaces the concept of cloneable objects with serializable objects. For platform objects, instead of defining a [[Clone]]() internal method, serializable platform objects are annotated with the new [Serializable] IDL attribute, and include serialization and deserialization steps in their definition. * Updates the concept of transferable objects. For platform objects, instead of defining a [[Transfer]]() internal method, transferable platform objects are annotated with the new [Transferable] IDL attribute, and include transfer and transfer-receiving steps. Additionally, the [[Detached]] internal slot for such objects is now managed more automatically. * Removes the StructuredClone() abstract operation in favor of separate StructuredSerialize() and StructuredDeserialize() abstract operations. In practice we found that performing a structured clone alone is never necessary in specs. It is always either coupled with a transfer list, for which StructuredCloneWithTransfer() can be used, or it is best expressed as separate serialization and deserialization steps. * Removes IsTransferable() and Transfer() abstract operations. When defined more properly, these became less useful by themselves, so they were inlined into the rest of the machinery. * Introduces StructuredSerialzieWithTransfer() and StructuredDeserializeWithTransfer(), which can be used by other specifications which need to define their own postMessage()-style algorithm but for which StructuredCloneWithTransfer() is not sufficient. Closes #785. Closes #935. Closes #2277. Closes #1162. Sets the stage for #936 and #2260/#2361.
Instead of each worker being owned by a document, have it instead be owned by one or more of its parents (more only in the case of SharedWorkerGlobalScope). This makes it possible to define JavaScript agent clusters (#2260) and also helps with allowing service workers to have nested workers (#411). This is marked editorial as it should have no normative impact. This also removes a step that should have been removed as part of 4e2b006 9418bd.
Instead of each worker being owned by a document, have it instead be owned by one or more of its parents (more only in the case of SharedWorkerGlobalScope). This makes it possible to define JavaScript agent clusters (#2260) and also helps with allowing service workers to have nested workers (#411). This is marked editorial as it should have no normative impact. This also removes a step that should have been removed as part of 4e2b006.
This puts the infrastructure in place to define SharedArrayBuffer in detail. See also #2260.
This puts the infrastructure in place to define SharedArrayBuffer in detail. See also #2260.
Instead of each worker being owned by a document, have it instead be owned by one or more of its parents (more only in the case of SharedWorkerGlobalScope). This makes it possible to define JavaScript agent clusters (#2260) and also helps with allowing service workers to have nested workers (#411). This is marked editorial as it should have no normative impact. This also removes a step that should have been removed as part of 4e2b006.
This puts the infrastructure in place to define SharedArrayBuffer in detail. See also #2260.
Define the infrastructure for SharedArrayBuffer. This also clarifies along which boundaries a browser implementation can use processes and threads. shaky foundation. Because of that, similar-origin window agents are the best place to store state that would formerly go on unit of related similar-origin browsing contexts. tc39/ecma262#882 is follow-up to define agents in more detail; in particular make their implicit realms slot explicit. w3c/css-houdini-drafts#224 is follow-up to define worklet ownership better which is needed to define how they relate to agent (sub)clusters. Fixes part of #2260. Fixes w3c/ServiceWorker#1115. Fixes most of w3c/css-houdini-drafts#380 (no tests and no nice grouping of multiple realms in a single agent as that is not needed).
Fixes #2260 (by being the last in a set of fixes). Closes tc39/proposal-ecmascript-sharedmem#144 and closes tc39/proposal-ecmascript-sharedmem#65.
Define the infrastructure for SharedArrayBuffer. This also clarifies along which boundaries a browser implementation can use processes and threads. shaky foundation. Because of that, similar-origin window agents are the best place to store state that would formerly go on unit of related similar-origin browsing contexts. tc39/ecma262#882 is follow-up to define agents in more detail; in particular make their implicit realms slot explicit. w3c/css-houdini-drafts#224 is follow-up to define worklet ownership better which is needed to define how they relate to agent (sub)clusters. Fixes part of #2260. Fixes w3c/ServiceWorker#1115. Fixes most of w3c/css-houdini-drafts#380 (no tests and no nice grouping of multiple realms in a single agent as that is not needed).
The messageerror event is used when deserialization fails. E.g., when an ArrayBuffer object cannot be allocated. This also removes StructuredCloneWithTransfer as deserializing errors now need to be handled on their own. Tests: web-platform-tests/wpt#5567. Service workers follow-up: w3c/ServiceWorker#1116. Fixes part of #2260 and fixes #935.
The messageerror event is used when deserialization fails. E.g., when an ArrayBuffer object cannot be allocated. This also removes StructuredCloneWithTransfer as deserializing errors now need to be handled on their own. Tests: web-platform-tests/wpt#5567. Service workers follow-up: w3c/ServiceWorker#1116. Fixes part of #2260 and fixes #935.
Define the infrastructure for SharedArrayBuffer. This also clarifies along which boundaries a browser implementation can use processes and threads. Tests: web-platform-tests/wpt#5569. Follow-up to define similar-origin window agents upon a less shaky foundation is #2528. Because of that, similar-origin window agents are the best place to store state that would formerly go on unit of related similar-origin browsing contexts. tc39/ecma262#882 is follow-up to define agents in more detail; in particular make their implicit realms slot explicit. w3c/css-houdini-drafts#224 is follow-up to define worklet ownership better which is needed to define how they relate to agent (sub)clusters. Fixes part of #2260. Fixes #851. Fixes w3c/ServiceWorker#1115. Fixes most of w3c/css-houdini-drafts#380 (no tests and no nice grouping of multiple realms in a single agent as that is not needed).
Additionally, define StructuredSerializeForStorage when for serializable objects need to be stored more persistently. Tests: web-platform-tests/wpt#5003. Fixes #2260 (by being the last in a set of fixes). Closes tc39/proposal-ecmascript-sharedmem#144 and closes tc39/proposal-ecmascript-sharedmem#65.
Define the infrastructure for SharedArrayBuffer. This also clarifies along which boundaries a browser implementation can use processes and threads. Tests: web-platform-tests/wpt#5569. Follow-up to define similar-origin window agents upon a less shaky foundation is #2528. Because of that, similar-origin window agents are the best place to store state that would formerly go on unit of related similar-origin browsing contexts. Follow-up for better agent shutdown notifications: #2581. tc39/ecma262#882 is follow-up to define agents in more detail; in particular make their implicit realms slot explicit. w3c/css-houdini-drafts#224 is follow-up to define worklet ownership better which is needed to define how they relate to agent (sub)clusters. Fixes part of #2260. Fixes #851. Fixes w3c/ServiceWorker#1115. Fixes most of w3c/css-houdini-drafts#380 (no tests and no nice grouping of multiple realms in a single agent as that is not needed).
Additionally, define StructuredSerializeForStorage when for serializable objects need to be stored more persistently. Tests: web-platform-tests/wpt#5003. Fixes #2260 (by being the last in a set of fixes). Closes tc39/proposal-ecmascript-sharedmem#144 and closes tc39/proposal-ecmascript-sharedmem#65.
Instead of each worker being owned by a document, have it instead be owned by one or more of its parents (more only in the case of SharedWorkerGlobalScope). This makes it possible to define JavaScript agent clusters (whatwg#2260) and also helps with allowing service workers to have nested workers (whatwg#411). This is marked editorial as it should have no normative impact. This also removes a step that should have been removed as part of whatwg@4e2b006.
The messageerror event is used when deserialization fails. E.g., when an ArrayBuffer object cannot be allocated. This also removes StructuredCloneWithTransfer as deserializing errors now need to be handled on their own. Tests: web-platform-tests/wpt#5567. Service workers follow-up: w3c/ServiceWorker#1116. Fixes part of whatwg#2260 and fixes whatwg#935.
Define the infrastructure for SharedArrayBuffer. This also clarifies along which boundaries a browser implementation can use processes and threads. Tests: web-platform-tests/wpt#5569. Follow-up to define similar-origin window agents upon a less shaky foundation is whatwg#2528. Because of that, similar-origin window agents are the best place to store state that would formerly go on unit of related similar-origin browsing contexts. Follow-up for better agent shutdown notifications: whatwg#2581. tc39/ecma262#882 is follow-up to define agents in more detail; in particular make their implicit realms slot explicit. w3c/css-houdini-drafts#224 is follow-up to define worklet ownership better which is needed to define how they relate to agent (sub)clusters. Fixes part of whatwg#2260. Fixes whatwg#851. Fixes w3c/ServiceWorker#1115. Fixes most of w3c/css-houdini-drafts#380 (no tests and no nice grouping of multiple realms in a single agent as that is not needed).
Additionally, define StructuredSerializeForStorage when for serializable objects need to be stored more persistently. Tests: web-platform-tests/wpt#5003. StructuredSerializeForStorage follow-up: * whatwg/notifications#99 * w3c/IndexedDB#197 Fixes whatwg#2260 (by being the last in a set of fixes). Closes tc39/proposal-ecmascript-sharedmem#144 and closes tc39/proposal-ecmascript-sharedmem#65.
Instead of each worker being owned by a document, have it instead be owned by one or more of its parents (more only in the case of SharedWorkerGlobalScope). This makes it possible to define JavaScript agent clusters (whatwg#2260) and also helps with allowing service workers to have nested workers (whatwg#411). This is marked editorial as it should have no normative impact. This also removes a step that should have been removed as part of whatwg@4e2b006.
The messageerror event is used when deserialization fails. E.g., when an ArrayBuffer object cannot be allocated. This also removes StructuredCloneWithTransfer as deserializing errors now need to be handled on their own. Tests: web-platform-tests/wpt#5567. Service workers follow-up: w3c/ServiceWorker#1116. Fixes part of whatwg#2260 and fixes whatwg#935.
Define the infrastructure for SharedArrayBuffer. This also clarifies along which boundaries a browser implementation can use processes and threads. Tests: web-platform-tests/wpt#5569. Follow-up to define similar-origin window agents upon a less shaky foundation is whatwg#2528. Because of that, similar-origin window agents are the best place to store state that would formerly go on unit of related similar-origin browsing contexts. Follow-up for better agent shutdown notifications: whatwg#2581. tc39/ecma262#882 is follow-up to define agents in more detail; in particular make their implicit realms slot explicit. w3c/css-houdini-drafts#224 is follow-up to define worklet ownership better which is needed to define how they relate to agent (sub)clusters. Fixes part of whatwg#2260. Fixes whatwg#851. Fixes w3c/ServiceWorker#1115. Fixes most of w3c/css-houdini-drafts#380 (no tests and no nice grouping of multiple realms in a single agent as that is not needed).
Additionally, define StructuredSerializeForStorage when for serializable objects need to be stored more persistently. Tests: web-platform-tests/wpt#5003. StructuredSerializeForStorage follow-up: * whatwg/notifications#99 * w3c/IndexedDB#197 Fixes whatwg#2260 (by being the last in a set of fixes). Closes tc39/proposal-ecmascript-sharedmem#144 and closes tc39/proposal-ecmascript-sharedmem#65.
This rewrites most of the cloneable and transferable object infrastructure to better reflect the reality that structured cloning requires separate serialization and deserialization steps, instead of a single operation that creates a new object in the target Realm. This is most evident in the case of MessagePorts, as noted in whatwg#2277. It also allows us to avoid awkward double-cloning with an intermediate "user-agent defined Realm", as seen in e.g. history.state or IndexedB; instead we can simply store the serialized form and later deserialize. Concretely, this: * Replaces the concept of cloneable objects with serializable objects. For platform objects, instead of defining a [[Clone]]() internal method, serializable platform objects are annotated with the new [Serializable] IDL attribute, and include serialization and deserialization steps in their definition. * Updates the concept of transferable objects. For platform objects, instead of defining a [[Transfer]]() internal method, transferable platform objects are annotated with the new [Transferable] IDL attribute, and include transfer and transfer-receiving steps. Additionally, the [[Detached]] internal slot for such objects is now managed more automatically. * Removes the StructuredClone() abstract operation in favor of separate StructuredSerialize() and StructuredDeserialize() abstract operations. In practice we found that performing a structured clone alone is never necessary in specs. It is always either coupled with a transfer list, for which StructuredCloneWithTransfer() can be used, or it is best expressed as separate serialization and deserialization steps. * Removes IsTransferable() and Transfer() abstract operations. When defined more properly, these became less useful by themselves, so they were inlined into the rest of the machinery. * Introduces StructuredSerialzieWithTransfer() and StructuredDeserializeWithTransfer(), which can be used by other specifications which need to define their own postMessage()-style algorithm but for which StructuredCloneWithTransfer() is not sufficient. Closes whatwg#785. Closes whatwg#935. Closes whatwg#2277. Closes whatwg#1162. Sets the stage for whatwg#936 and whatwg#2260/whatwg#2361.
Instead of each worker being owned by a document, have it instead be owned by one or more of its parents (more only in the case of SharedWorkerGlobalScope). This makes it possible to define JavaScript agent clusters (whatwg#2260) and also helps with allowing service workers to have nested workers (whatwg#411). This is marked editorial as it should have no normative impact. This also removes a step that should have been removed as part of whatwg@4e2b006.
The messageerror event is used when deserialization fails. E.g., when an ArrayBuffer object cannot be allocated. This also removes StructuredCloneWithTransfer as deserializing errors now need to be handled on their own. Tests: web-platform-tests/wpt#5567. Service workers follow-up: w3c/ServiceWorker#1116. Fixes part of whatwg#2260 and fixes whatwg#935.
Define the infrastructure for SharedArrayBuffer. This also clarifies along which boundaries a browser implementation can use processes and threads. Tests: web-platform-tests/wpt#5569. Follow-up to define similar-origin window agents upon a less shaky foundation is whatwg#2528. Because of that, similar-origin window agents are the best place to store state that would formerly go on unit of related similar-origin browsing contexts. Follow-up for better agent shutdown notifications: whatwg#2581. tc39/ecma262#882 is follow-up to define agents in more detail; in particular make their implicit realms slot explicit. w3c/css-houdini-drafts#224 is follow-up to define worklet ownership better which is needed to define how they relate to agent (sub)clusters. Fixes part of whatwg#2260. Fixes whatwg#851. Fixes w3c/ServiceWorker#1115. Fixes most of w3c/css-houdini-drafts#380 (no tests and no nice grouping of multiple realms in a single agent as that is not needed).
Additionally, define StructuredSerializeForStorage when for serializable objects need to be stored more persistently. Tests: web-platform-tests/wpt#5003. StructuredSerializeForStorage follow-up: * whatwg/notifications#99 * w3c/IndexedDB#197 Fixes whatwg#2260 (by being the last in a set of fixes). Closes tc39/proposal-ecmascript-sharedmem#144 and closes tc39/proposal-ecmascript-sharedmem#65.
/cc @lars-t-hansen @bradnelson
The spec currently does not describe
transferringcloning SharedArrayBuffers. I believe some relevant content might be in https://tc39.github.io/ecmascript_sharedmem/shmem.html#WebBrowserEmbedding and need upstreaming, although that doesn't seem comprehensive. I also recall discussions previously about whether or not it should be in the transfer list, but I can't find them right now. (I think the answer was it shouldn't, because we don't detach it.)I think this mainly involves modifying StructuredClone. I'm happy to work on this but I want to be sure I found all the background material written so far, and I don't think I have :).
The text was updated successfully, but these errors were encountered: