Skip to content

Commit

Permalink
.Net: Improved example with telemetry in filters for streaming scenar…
Browse files Browse the repository at this point in the history
…io (#9775)

### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

Improved an example with telemetry in filters for streaming scenario to
return streaming chunks from filter to operation's origin and logging
the result at the same time.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
  • Loading branch information
dmytrostruk authored Nov 21, 2024
1 parent ea2ba47 commit 6cc6822
Showing 1 changed file with 38 additions and 19 deletions.
57 changes: 38 additions & 19 deletions dotnet/samples/Concepts/Filtering/TelemetryWithFilters.cs
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,17 @@ public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, F

logger.LogInformation("Function {FunctionName} succeeded.", context.Function.Name);

await this.LogFunctionResultAsync(context);
if (context.IsStreaming)
{
// Overriding the result in a streaming scenario enables the filter to stream chunks
// back to the operation's origin without interrupting the data flow.
var enumerable = context.Result.GetValue<IAsyncEnumerable<StreamingChatMessageContent>>();
context.Result = new FunctionResult(context.Result, ProcessFunctionResultStreamingAsync(enumerable!));
}
else
{
ProcessFunctionResult(context.Result);
}
}
catch (Exception exception)
{
Expand All @@ -167,34 +177,43 @@ public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, F
}
}

private async Task LogFunctionResultAsync(FunctionInvocationContext context)
private void ProcessFunctionResult(FunctionResult functionResult)
{
string? result = functionResult.GetValue<string>();
object? usage = functionResult.Metadata?["Usage"];

if (!string.IsNullOrWhiteSpace(result))
{
logger.LogTrace("Function result: {Result}", result);
}

if (logger.IsEnabled(LogLevel.Information) && usage is not null)
{
logger.LogInformation("Usage: {Usage}", JsonSerializer.Serialize(usage));
}
}

private async IAsyncEnumerable<StreamingChatMessageContent> ProcessFunctionResultStreamingAsync(IAsyncEnumerable<StreamingChatMessageContent> data)
{
string? result = null;
object? usage = null;

if (context.IsStreaming)
var stringBuilder = new StringBuilder();

await foreach (var item in data)
{
var stringBuilder = new StringBuilder();
yield return item;

await foreach (var item in context.Result.GetValue<IAsyncEnumerable<StreamingChatMessageContent>>()!)
if (item.Content is not null)
{
if (item.Content is not null)
{
stringBuilder.Append(item.Content);
}

usage = item.Metadata?["Usage"];
stringBuilder.Append(item.Content);
}

result = stringBuilder.ToString();
}
else
{
result = context.Result.GetValue<string>();
usage = context.Result.Metadata?["Usage"];
usage = item.Metadata?["Usage"];
}

if (result is not null)
var result = stringBuilder.ToString();

if (!string.IsNullOrWhiteSpace(result))
{
logger.LogTrace("Function result: {Result}", result);
}
Expand Down

0 comments on commit 6cc6822

Please sign in to comment.