.Net: Feature flow planner

[yan-li]

Motivation and Context

This PR is trying to help making it easier to build complex Copilot which is capable of commanding, Q&A, collaboration, suggestion, etc, in reliable ways.

A new FlowPlanner is introduced, which leverages the available skills, execute the plan in multi-step, multi-pass way, including interaction with users to get clarification on the goal, input for skills/functions, and authorization for actions when needed.

More details in #2164.

Description

Major changes

• FlowModel
• ChatSkill, which is to make the implementation of multi-pass skill easier with built in chat history accessor
• FlowExecutor
• ReAct engine, which is similar as the StepwisePlanner implementation but more isolated and enriched with chat integration
• UnitTests
• IntegrationTests
• Examples

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
• All unit tests pass, and I have added new tests where possible
• I didn't break anyone 😄

[lemillermicrosoft]

Thanks for the PR, will take some time to go through it. I'd like to understand these pieces better (xml docs will help) and whether it would make sense to include as part of Plan or SKFunction objects.

[yan-li]

Thanks Lee. Just completed the docs for the data models.
I did explore Plan in the beginning. however there are some gaps so I am proposing a different IPlan implementation. // haven't really implemented the marker interface yet.

Here are the quoted comments from Flow.cs.

/// Principles:
/// 1. The model should be decoupled from execution status
/// 2. The model is mutable to allow dynamic changes
/// 3. The model doesn't enforce any execution order as long as the dependencies are satisfied.

Besides these, there are some other differences

1. for each step in the flow, there could be a set of skills to use to achieve the step goal.
2. each step should support multi-in, multi-out
3. execution context for each step should be highly isolated to avoid noise and waste of tokens

[lemillermicrosoft]

I think Kernel is more appropriate here.

[yan-li]

The idea of having kernel builder is that for each step, we could use an isolated kernel.
Otherwise when we present functions in ReAct steps, unwanted skills would show up there and mislead the reasoning.

[lemillermicrosoft]

Okay note for myself to take a second look at how the skills are managed or determined to be unwanted from the global skills collection.

[lemillermicrosoft]

Not sure about global naming -- is that appropriate?

Also, why not use the skills available in the kernel? This seems more like a custom loader that should not be planner specific logic.

[yan-li]

yes this is related to the Flow load process which loads the skills into kernel.
as mentioned in above comment, for each step, we initialize a new kernel and load skills.

And there are two ways to instantiate the skills.

1. via reflection
2. from this global skill collection

[lemillermicrosoft]

So is the idea that while result.Result is not empty, that message is passed to the user, and user message is fed back into the flow?

[yan-li]

Yes. Here is a high level example to utilize the planner is having a web api which accepts user input and return response.

[Route("complete/{userInput}")]
public async Task<string[]> Complete(string userInput)
{
  Flow flow = this.GetActiveFlowFromStorage(userContext);
  if (flow == null)
  {
    flow = this.SelectNextFlow(userInput)
  }
  
  var result = await planner.ExecuteFlowAsync(flow, userInput).ConfigureAwait(false);
  if (result.isComplete)
  {
     // mark this as complete in storage
  }

  return JsonSerializer.Deserialize<string[]>(result.Result); 
}

[lemillermicrosoft]

note for self: stepasync/invokeasync and/or enabling functions in a plan to have control over execution state.

[lemillermicrosoft]

The idea here is that the result of ExecuteFlow is a message to the user and when that is empty, the Answer is the message to the user, is that right?

Would be nice to tighten up the pattern there, to make it more clear if execution needs to continue or not. I will try and take a pass at making this follow the Plan object model. I'm also still leaning on the idea that the get information from use is a function the kernel has a handle to.

[yan-li]

here are the actual code we use to determine if the flow has completed and show proceed.

if (flow.Provides.All(p => result.Variables.ContainsKey(p)))
{
    // flow has completed, update state in DB
}