feat(orchestration): stopgap integration w asyncFlow #9521

erights · 2024-06-17T23:58:26Z

closes: #XXXX
refs: #XXXX

Description

Security Considerations

Scaling Considerations

Documentation Considerations

Testing Considerations

Upgrade Considerations

cloudflare-workers-and-pages · 2024-06-18T00:01:49Z

Deploying agoric-sdk with Cloudflare Pages

Latest commit:	`68908e3`
Status:	✅ Deploy successful!
Preview URL:	https://ac400b5f.agoric-sdk.pages.dev
Branch Preview URL:	https://markm-stopgap-orch-use-async.agoric-sdk.pages.dev

View logs

mhofman · 2024-06-18T20:40:01Z

packages/orchestration/src/facade.js

+      const guestFunc = async (...args) => fn(orc, ctx, ...args);
+      const hostFunc = asyncFlow(zone, durableName, guestFunc);
+      return hostFunc;


Here is what I had in my draft:

Suggested change

const guestFunc = async (...args) => fn(orc, ctx, ...args);

const hostFunc = asyncFlow(zone, durableName, guestFunc);

return hostFunc;

const subzone = zone.subZone(durableName);

// TODO: create durable exo wrappers for ctx props

const ctxKit = {}; // makeDurableCtxKit(ctx);

const asyncFlowGuestFunc = async (flowSupportKit, ...args) => {

// TODO: Add support for recreating a `this` exo class context

const exoContext = undefined;

// TODO: Get orc from `flowSupportKit` once it's made durable

const orcArg = orc;

// TODO: build ctx from a durable skeleton using `flowSupportKit`

const ctxArg = ctx;

return apply(fn, exoContext, [orcArg, ctxArg, ...args]);

}

const asyncFlowHostFn = asyncFlow(subzone, 'asyncFlow', asyncFlowGuestFunc);

return harden({

[durableName](...args) {

// TODO: Build the needed durable objects

const orcKit = {}; // internalMakeOrchestratorKit();

// TODO: plumb the necessary ingredients to reconstruct a `this` exo context inside the guest

return asyncFlowHostFn({ orc: orcKit.orchestrator, ctxKit }, ...args);

},

}[durableName]);

See #9566 , where this work continues.

@erights

refs: #9281 ## Description AsyncFlow requires that everything passing the membrane is durable. This makes the facade objects durable to conform. Doing so for `localChainFacade` is deferred so we can get this into master sooner, to aid @erights 's #9521 . ### Security Considerations none ### Scaling Considerations Exo for each chain and each account ### Documentation Considerations none ### Testing Considerations Existing coverage ### Upgrade Considerations none, not yet deployed

erights · 2024-06-23T23:14:20Z

Closing in favor of #9566

closes: #XXXX refs: #9449 #9521 #9304 #9281 ## Description Changed async-flow to support endowments. Changed `orchestrate` to use `asyncFlow` with endowments. Changed `sendAnywhere` example orchestration contract to be more compatible with this new `orchestrate`. The CI errors are all in the `orchestration` package. After some earlier iteration where orchestration failures indicated async-flow bugs, which I fixed, the remaining errors seem plausibly to be integration bugs on the orchestration side revealed by using this improved `orchestrate` function. If so, that satisfies the purpose of this PR -- to enable integration testing to reveal such errors. However, this leaves open the question of how to bring this PR to fruition despite these CI errors. In that iteration, the majority of errors were due to host-side promises, which we expected. To proceed with integration testing, I temporarily turned that case into a warning, by wrapping the host-side promise with a host-side vow. This stopgap measure is obviously fragile under upgrade. It would cause may upgrades to fail However, I have not investigated these CI errors enough to be at all confident that none of them are due to bugs in async-flow. For any of those, they should be fixed in this PR. ### Security Considerations nothing new ### Scaling Considerations none, given that total endowments are low cardinality. All these endowments are prepare-time per-function. There should not be any cardinality limit on the activations making use of these endowments. But like all other async-flow scaling issues, that remains to be tested. ### Documentation Considerations The endowment rules and taxonomy is interesting, and should be documented. ### Testing Considerations We get CI errors only from the `orchestration` package. Some of these may be the integration bugs we wanted this exercise to reveal. However, others may be async-flow bugs, which should have been caught by async-flow unit tests. The warning stopgap I mentioned above [appears in CI](https://github.com/Agoric/agoric-sdk/actions/runs/9637015639/job/26575694851?pr=9566#step:12:648) as, for example ``` Warning for now: vow expected, not promise Promise { <pending> } (Error#1) Error#1: where warning happened at makeGuestForHostVow (.../async-flow/src/replay-membrane.js:329:9) at eval (.../async-flow/src/convert.js:119:10) at innerConvert (.../async-flow/src/convert.js:63:8) at convertRecur (.../async-flow/src/convert.js:30:8) at convert (.../async-flow/src/convert.js:76:1) at performCall (.../async-flow/src/replay-membrane.js:137:1) at guestCallsHost (.../async-flow/src/replay-membrane.js:195:9) at In "getChain" method of (Orchestrator orchestrator) [as getChain] (.../async-flow/src/replay-membrane.js:282:8) at eval (.../orchestration/src/examples/unbondExample.contract.js:60:23) at eval (.../async-flow/src/async-flow.js:222:1) at Object.restart (.../async-flow/src/async-flow.js:222:30) at makeAsyncFlowKit (.../async-flow/src/async-flow.js:430:6) at asyncFlow_hostFlow (.../async-flow/src/async-flow.js:448:13) at orcFn (.../orchestration/src/facade.js:124:15) at eval (.../pass-style/src/make-far.js:224:31) ``` The relevant lines are ``` at In "getChain" method of (Orchestrator orchestrator) [as getChain] (.../async-flow/src/replay-membrane.js:282:8) at eval (.../orchestration/src/examples/unbondExample.contract.js:60:23) ``` where the first line indicates what method or method guard provided the inappropriate promise ```js getChain: M.callWhen(M.string()).returns(ChainInfoShape), ``` and the second line indicates where the guest code called it ```js const omni = await orch.getChain('omniflixhub'); ``` ### Upgrade Considerations The orchestration code in question cannot be truly upgrade safe until we see no more of these "vow expected, not promise" warnings. Even then, we should expect that async-flow as of this PR is ready for lots of testing, but not yet ready to run on the main chain with durable state expected to survive real upgrades.

erights requested review from turadg, mhofman and dtribble June 17, 2024 23:58

turadg force-pushed the 9281-provideAsyncTools branch from 05cd7ee to f096604 Compare June 18, 2024 01:11

feat(orchestration): stopgap integration w asyncFlow

68908e3

erights force-pushed the markm-stopgap-orch-use-async-flow branch from 6db8022 to 68908e3 Compare June 18, 2024 15:45

Base automatically changed from 9281-provideAsyncTools to master June 18, 2024 17:19

mhofman reviewed Jun 18, 2024

View reviewed changes

turadg mentioned this pull request Jun 18, 2024

refactor: exofy Orchestrator, RemoteChainFacade #9529

Merged

erights added the asyncFlow related to membrane-based replay and upgrade of async functions label Jun 20, 2024

erights closed this Jun 23, 2024

erights mentioned this pull request Jun 23, 2024

feat(async-flow): endowments #9566

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(orchestration): stopgap integration w asyncFlow #9521

feat(orchestration): stopgap integration w asyncFlow #9521

erights commented Jun 17, 2024

cloudflare-workers-and-pages bot commented Jun 18, 2024 •

edited

Loading

mhofman Jun 18, 2024 •

edited

Loading

erights Jun 23, 2024

erights commented Jun 23, 2024

-      const guestFunc = async (...args) => fn(orc, ctx, ...args);
-      const hostFunc = asyncFlow(zone, durableName, guestFunc);
-      return hostFunc;
+      const subzone = zone.subZone(durableName);
+      // TODO: create durable exo wrappers for ctx props
+      const ctxKit = {}; // makeDurableCtxKit(ctx);
+      const asyncFlowGuestFunc = async (flowSupportKit, ...args) => {
+        // TODO: Add support for recreating a `this` exo class context
+        const exoContext = undefined;
+        // TODO: Get orc from `flowSupportKit` once it's made durable
+        const orcArg = orc;
+        // TODO: build ctx from a durable skeleton using `flowSupportKit`
+        const ctxArg = ctx;
+        return apply(fn, exoContext, [orcArg, ctxArg, ...args]);
+      }
+      const asyncFlowHostFn = asyncFlow(subzone, 'asyncFlow', asyncFlowGuestFunc);
+      return harden({
+        [durableName](...args) {
+          // TODO: Build the needed durable objects
+          const orcKit = {}; // internalMakeOrchestratorKit();
+          // TODO: plumb the necessary ingredients to reconstruct a `this` exo context inside the guest
+          return asyncFlowHostFn({ orc: orcKit.orchestrator, ctxKit }, ...args);
+        },
+      }[durableName]);

feat(orchestration): stopgap integration w asyncFlow #9521

feat(orchestration): stopgap integration w asyncFlow #9521

Conversation

erights commented Jun 17, 2024

Description

Security Considerations

Scaling Considerations

Documentation Considerations

Testing Considerations

Upgrade Considerations

cloudflare-workers-and-pages bot commented Jun 18, 2024 • edited Loading

Deploying agoric-sdk with Cloudflare Pages

mhofman Jun 18, 2024 • edited Loading

Choose a reason for hiding this comment

erights Jun 23, 2024

Choose a reason for hiding this comment

erights commented Jun 23, 2024

cloudflare-workers-and-pages bot commented Jun 18, 2024 •

edited

Loading

mhofman Jun 18, 2024 •

edited

Loading