Skip to content

Commit 80f598c

Browse files
dmytrostrukdmytrostruk
authored
.Net: Marked Kernel events as deprecated in favor of filters (microsoft#4714)
### Description 1. Added `Obsolete` attribute to Kernel events. 2. Added ADR with information about Kernel filters. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄
1 parent 0871037 commit 80f598c

22 files changed

+380
-248
lines changed

‎docs/decisions/0031-kernel-filters.md

+150
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
---
2+
# These are optional elements. Feel free to remove any of them.
3+
status: accepted
4+
contact: dmytrostruk
5+
date: 2023-01-23
6+
deciders: sergeymenshykh, markwallace, rbarreto, stephentoub, dmytrostruk
7+
---
8+
9+
# Kernel Filters
10+
11+
## Context and Problem Statement
12+
13+
Current way of intercepting some event during function execution works as expected using Kernel Events and event handlers. Example:
14+
15+
```csharp
16+
ILogger logger = loggerFactory.CreateLogger("MyLogger");
17+
18+
var kernel = Kernel.CreateBuilder()
19+
.AddOpenAIChatCompletion(
20+
modelId: TestConfiguration.OpenAI.ChatModelId,
21+
apiKey: TestConfiguration.OpenAI.ApiKey)
22+
.Build();
23+
24+
void MyInvokingHandler(object? sender, FunctionInvokingEventArgs e)
25+
{
26+
logger.LogInformation("Invoking: {FunctionName}", e.Function.Name)
27+
}
28+
29+
void MyInvokedHandler(object? sender, FunctionInvokedEventArgs e)
30+
{
31+
if (e.Result.Metadata is not null && e.Result.Metadata.ContainsKey("Usage"))
32+
{
33+
logger.LogInformation("Token usage: {TokenUsage}", e.Result.Metadata?["Usage"]?.AsJson());
34+
}
35+
}
36+
37+
kernel.FunctionInvoking += MyInvokingHandler;
38+
kernel.FunctionInvoked += MyInvokedHandler;
39+
40+
var result = await kernel.InvokePromptAsync("How many days until Christmas? Explain your thinking.")
41+
```
42+
43+
There are a couple of problems with this approach:
44+
45+
1. Event handlers does not support dependency injection. It's hard to get access to specific service, which is registered in application, unless the handler is defined in the same scope where specific service is available. This approach provides some limitations in what place in solution the handler could be defined. (e.g. If developer wants to use `ILoggerFactory` in handler, the handler should be defined in place where `ILoggerFactory` instance is available).
46+
2. It's not clear in what specific period of application runtime the handler should be attached to kernel. Also, it's not clear if developer needs to detach it at some point.
47+
3. Mechanism of events and event handlers in .NET may not be familiar to .NET developers who didn't work with events previously.
48+
49+
<!-- This is an optional element. Feel free to remove. -->
50+
51+
## Decision Drivers
52+
53+
1. Dependency injection for handlers should be supported to easily access registered services within application.
54+
2. There should not be any limitations where handlers are defined within solution, whether it's Startup.cs or separate file.
55+
3. There should be clear way of registering and removing handlers at specific point of application runtime.
56+
4. The mechanism of receiving and processing events in Kernel should be easy and common in .NET ecosystem.
57+
5. New approach should support the same functionality that is available in Kernel Events - cancel function execution, change kernel arguments, change rendered prompt before sending it to AI etc.
58+
59+
## Decision Outcome
60+
61+
Introduce Kernel Filters - the approach of receiving the events in Kernel in similar way as action filters in ASP.NET.
62+
63+
Two new abstractions will be used across Semantic Kernel and developers will have to implement these abstractions in a way that will cover their needs.
64+
65+
For function-related events: `IFunctionFilter`
66+
67+
```csharp
68+
public interface IFunctionFilter
69+
{
70+
void OnFunctionInvoking(FunctionInvokingContext context);
71+
72+
void OnFunctionInvoked(FunctionInvokedContext context);
73+
}
74+
```
75+
76+
For prompt-related events: `IPromptFilter`
77+
78+
```csharp
79+
public interface IPromptFilter
80+
{
81+
void OnPromptRendering(PromptRenderingContext context);
82+
83+
void OnPromptRendered(PromptRenderedContext context);
84+
}
85+
```
86+
87+
New approach will allow developers to define filters in separate classes and easily inject required services to process kernel event correctly:
88+
89+
MyFunctionFilter.cs - filter with the same logic as event handler presented above:
90+
91+
```csharp
92+
public sealed class MyFunctionFilter : IFunctionFilter
93+
{
94+
private readonly ILogger _logger;
95+
96+
public MyFunctionFilter(ILoggerFactory loggerFactory)
97+
{
98+
this._logger = loggerFactory.CreateLogger("MyLogger");
99+
}
100+
101+
public void OnFunctionInvoking(FunctionInvokingContext context)
102+
{
103+
this._logger.LogInformation("Invoking {FunctionName}", context.Function.Name);
104+
}
105+
106+
public void OnFunctionInvoked(FunctionInvokedContext context)
107+
{
108+
var metadata = context.Result.Metadata;
109+
110+
if (metadata is not null && metadata.ContainsKey("Usage"))
111+
{
112+
this._logger.LogInformation("Token usage: {TokenUsage}", metadata["Usage"]?.AsJson());
113+
}
114+
}
115+
}
116+
```
117+
118+
As soon as new filter is defined, it's easy to configure it to be used in Kernel using dependency injection (pre-construction) or add filter after Kernel initialization (post-construction):
119+
120+
```csharp
121+
IKernelBuilder kernelBuilder = Kernel.CreateBuilder();
122+
kernelBuilder.AddOpenAIChatCompletion(
123+
modelId: TestConfiguration.OpenAI.ChatModelId,
124+
apiKey: TestConfiguration.OpenAI.ApiKey);
125+
126+
// Adding filter with DI (pre-construction)
127+
kernelBuilder.Services.AddSingleton<IFunctionFilter, MyFunctionFilter>();
128+
129+
Kernel kernel = kernelBuilder.Build();
130+
131+
// Adding filter after Kernel initialization (post-construction)
132+
// kernel.FunctionFilters.Add(new MyAwesomeFilter());
133+
134+
var result = await kernel.InvokePromptAsync("How many days until Christmas? Explain your thinking.");
135+
```
136+
137+
It's also possible to configure multiple filters which will be triggered in order of registration:
138+
139+
```csharp
140+
kernelBuilder.Services.AddSingleton<IFunctionFilter, Filter1>();
141+
kernelBuilder.Services.AddSingleton<IFunctionFilter, Filter2>();
142+
kernelBuilder.Services.AddSingleton<IFunctionFilter, Filter3>();
143+
```
144+
145+
And it's possible to change the order of filter execution in runtime or remove specific filter if needed:
146+
147+
```csharp
148+
kernel.FunctionFilters.Insert(0, new InitialFilter());
149+
kernel.FunctionFilters.RemoveAt(1);
150+
```

‎dotnet/docs/EXPERIMENTS.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ You can use the following diagnostic IDs to ignore warnings or errors for a part
1515
- SKEXP0001: Embedding services
1616
- SKEXP0002: Image services
1717
- SKEXP0003: Memory connectors
18-
- SKEXP0004: Kernel Events and Filters
18+
- SKEXP0004: Kernel Filters
1919

2020
## OpenAI and Azure OpenAI services
2121

‎dotnet/samples/KernelSyntaxExamples/Example57_KernelHooks.cs

+2
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,8 @@
1010

1111
namespace Examples;
1212

13+
#pragma warning disable CS0618 // Events are deprecated
14+
1315
public class Example57_KernelHooks : BaseTest
1416
{
1517
/// <summary>

‎dotnet/samples/KernelSyntaxExamples/Example76_Filters.cs

-103
This file was deleted.

‎dotnet/samples/KernelSyntaxExamples/Getting_Started/Step6_Responsible_AI.cs

+43-24
Original file line numberDiff line numberDiff line change
@@ -2,54 +2,73 @@
22

33
using System.Threading.Tasks;
44
using Examples;
5+
using Microsoft.Extensions.DependencyInjection;
56
using Microsoft.SemanticKernel;
67
using Xunit;
78
using Xunit.Abstractions;
89

910
namespace GettingStarted;
1011

11-
// This example shows how to use rendering event hooks to ensure that prompts are rendered in a responsible manner.
1212
public class Step6_Responsible_AI : BaseTest
1313
{
1414
/// <summary>
15-
/// Show how to use rendering event hooks to ensure that prompts are rendered in a responsible manner.
15+
/// Show how to use prompt filters to ensure that prompts are rendered in a responsible manner.
1616
/// </summary>
1717
[Fact]
1818
public async Task RunAsync()
1919
{
2020
// Create a kernel with OpenAI chat completion
21-
Kernel kernel = Kernel.CreateBuilder()
21+
var builder = Kernel.CreateBuilder()
2222
.AddOpenAIChatCompletion(
2323
modelId: TestConfiguration.OpenAI.ChatModelId,
24-
apiKey: TestConfiguration.OpenAI.ApiKey)
25-
.Build();
24+
apiKey: TestConfiguration.OpenAI.ApiKey);
2625

27-
// Handler which is called before a prompt is rendered
28-
void MyRenderingHandler(object? sender, PromptRenderingEventArgs e)
29-
{
30-
if (e.Arguments.ContainsName("card_number"))
31-
{
32-
e.Arguments["card_number"] = "**** **** **** ****";
33-
}
34-
}
35-
36-
// Handler which is called after a prompt is rendered
37-
void MyRenderedHandler(object? sender, PromptRenderedEventArgs e)
38-
{
39-
e.RenderedPrompt += " NO SEXISM, RACISM OR OTHER BIAS/BIGOTRY";
26+
builder.Services.AddSingleton<ITestOutputHelper>(this.Output);
4027

41-
WriteLine(e.RenderedPrompt);
42-
}
28+
// Add prompt filter to the kernel
29+
builder.Services.AddSingleton<IPromptFilter, PromptFilter>();
4330

44-
// Add the handlers to the kernel
45-
kernel.PromptRendering += MyRenderingHandler;
46-
kernel.PromptRendered += MyRenderedHandler;
31+
var kernel = builder.Build();
4732

4833
KernelArguments arguments = new() { { "card_number", "4444 3333 2222 1111" } };
49-
WriteLine(await kernel.InvokePromptAsync("Tell me some useful information about this credit card number {{$card_number}}?", arguments));
34+
35+
var result = await kernel.InvokePromptAsync("Tell me some useful information about this credit card number {{$card_number}}?", arguments);
36+
37+
WriteLine(result);
5038
}
5139

5240
public Step6_Responsible_AI(ITestOutputHelper output) : base(output)
5341
{
5442
}
43+
44+
private sealed class PromptFilter : IPromptFilter
45+
{
46+
private readonly ITestOutputHelper _output;
47+
48+
public PromptFilter(ITestOutputHelper output)
49+
{
50+
this._output = output;
51+
}
52+
53+
/// <summary>
54+
/// Method which is called before a prompt is rendered.
55+
/// </summary>
56+
public void OnPromptRendered(PromptRenderedContext context)
57+
{
58+
context.RenderedPrompt += " NO SEXISM, RACISM OR OTHER BIAS/BIGOTRY";
59+
60+
this._output.WriteLine(context.RenderedPrompt);
61+
}
62+
63+
/// <summary>
64+
/// Method which is called after a prompt is rendered.
65+
/// </summary>
66+
public void OnPromptRendering(PromptRenderingContext context)
67+
{
68+
if (context.Arguments.ContainsName("card_number"))
69+
{
70+
context.Arguments["card_number"] = "**** **** **** ****";
71+
}
72+
}
73+
}
5574
}

0 commit comments

Comments
 (0)