-
Notifications
You must be signed in to change notification settings - Fork 3.5k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
.Net: Process mermaid flowchart code generation, image generation on …
…flowchart and sample usage. (#9705) I propose adding functionality to render workflows/processes as visual diagrams, using Markdown (Mermaid syntax) and generating images. This feature would greatly aid in documenting and communicating process flows within the framework. ### Motivation and Context Why: Improved visualization and understanding of the workflow, reduce cognitive load and also it makes it cooler. So we can visualize the Process Workflow in Step 01 (Step01_Processes) like this  Source and example (once merged): https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/GettingStartedWithProcesses/Step01/Step01_Processes.cs And the "Fried Fish Preparation Process" looks like this:  And like this if we set the nesting to 1 (no nesting), so it does not go into a subprocess (a step which is a KernelProcess):  Source and example: https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/GettingStartedWithProcesses/Step03/Step03a_FoodPreparation.cs Problem solved: Eases understanding complex workflows, helps to validate your workflow and to improve it. Makes understanding FASTER. Scenario: All scenarios Related issue: #9514 **Existing issue**: Usually a Step (node) has edges to other Steps which is OK. But, in case of a Step which is a Process (ie, composed of other steps), This Step can have an Edge to an Inner Step, which makes no sense. After a great discussion with the team, @alliscode and @esttenorio (thanks a ton for the great interaction and discussion) this is done at the moment to represent Entry Steps, where the workflow is starting on the Process Step. This made it impossible to draw this properly and is ignored, but upon discovery it was rendered like this for reference:  And if we debug the Edges of the Step which is a Process, we see it has two, the second one properly formed is the exit Edge but the first one on the Edge Key it does not have a properly formed EventId but a simple name:  This comes from the entry point defined next (Thanks @esttenorio for finding it):  Here is there the root process steps "links" with the internal steps of the nested process. Which works, but we end up with a Edge from an "event" (FriedFishProcess.ProcessEvents.PrepareFriedFish) to an Internal Step. The biggest problem here is that the Event is parentless, not associated to a Step. So, we cannot "paint" the graph to depict where does it come from. The technical solution is to attach the Edge to the target Step, but it has no parent. Proposal: Make all Events with a Single Parent. One Event, One Parent, point. And associate the events to that parent through an Edge. so we know where it starts and where it ends(goes). This way will enable proper tooling and accurate rendering - and the inherent graph will not try to "trick us". (I will create an issue for this) **Solution:** At the moment we are identifying and ignoring those Edges. ### Description Added ProcessVisualizationExtensions.cs in Process.Core to render the markdown at the core, next to the ProcessBuilder, as an extension method to it. Updated Sample 01 to showcase example, also added the MermaidRenderer.cs to render an image of this mermaid code by using PuppeteerSharp, which had to be added to the GettingStartedWithProcesses. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 --------- Co-authored-by: Chris <[email protected]>
- Loading branch information
Showing
6 changed files
with
309 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
123 changes: 123 additions & 0 deletions
123
dotnet/samples/GettingStartedWithProcesses/Utilities/MermaidRenderer.cs
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,123 @@ | ||
| // Copyright (c) Microsoft. All rights reserved. | ||
|
|
||
| using System.Reflection; | ||
| using PuppeteerSharp; | ||
|
|
||
| namespace Utilities; | ||
|
|
||
| /// <summary> | ||
| /// Renders Mermaid diagrams to images using Puppeteer-Sharp. | ||
| /// </summary> | ||
| public static class MermaidRenderer | ||
| { | ||
| /// <summary> | ||
| /// Generates a Mermaid diagram image from the provided Mermaid code. | ||
| /// </summary> | ||
| /// <param name="mermaidCode"></param> | ||
| /// <param name="filenameOrPath"></param> | ||
| /// <returns></returns> | ||
| /// <exception cref="InvalidOperationException"></exception> | ||
| public static async Task<string> GenerateMermaidImageAsync(string mermaidCode, string filenameOrPath) | ||
| { | ||
| // Ensure the filename has the correct .png extension | ||
| if (!filenameOrPath.EndsWith(".png", StringComparison.OrdinalIgnoreCase)) | ||
| { | ||
| throw new ArgumentException("The filename must have a .png extension.", nameof(filenameOrPath)); | ||
| } | ||
|
|
||
| string outputFilePath; | ||
|
|
||
| // Check if the user provided an absolute path | ||
| if (Path.IsPathRooted(filenameOrPath)) | ||
| { | ||
| // Use the provided absolute path | ||
| outputFilePath = filenameOrPath; | ||
|
|
||
| // Ensure the directory exists | ||
| string directoryPath = Path.GetDirectoryName(outputFilePath) | ||
| ?? throw new InvalidOperationException("Could not determine the directory path."); | ||
| if (!Directory.Exists(directoryPath)) | ||
| { | ||
| throw new DirectoryNotFoundException($"The directory '{directoryPath}' does not exist."); | ||
| } | ||
| } | ||
| else | ||
| { | ||
| // Use the assembly's directory for relative paths | ||
| string? assemblyPath = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location); | ||
| if (assemblyPath == null) | ||
| { | ||
| throw new InvalidOperationException("Could not determine the assembly path."); | ||
| } | ||
|
|
||
| string outputPath = Path.Combine(assemblyPath, "output"); | ||
| Directory.CreateDirectory(outputPath); // Ensure output directory exists | ||
| outputFilePath = Path.Combine(outputPath, filenameOrPath); | ||
| } | ||
|
|
||
| // Download Chromium if it hasn't been installed yet | ||
| BrowserFetcher browserFetcher = new(); | ||
| browserFetcher.Browser = SupportedBrowser.Chrome; | ||
| await browserFetcher.DownloadAsync(); | ||
|
|
||
| // Define the HTML template with Mermaid.js CDN | ||
| string htmlContent = $@" | ||
| <html> | ||
| <head> | ||
| <style> | ||
| body {{ | ||
| display: flex; | ||
| align-items: center; | ||
| justify-content: center; | ||
| margin: 0; | ||
| height: 100vh; | ||
| }} | ||
| </style> | ||
| <script type=""module""> | ||
| import mermaid from 'https://cdn.jsdelivr.net/npm/mermaid@10/dist/mermaid.esm.min.mjs'; | ||
| mermaid.initialize({{ startOnLoad: true }}); | ||
| </script> | ||
| </head> | ||
| <body> | ||
| <div class=""mermaid""> | ||
| {mermaidCode} | ||
| </div> | ||
| </body> | ||
| </html>"; | ||
|
|
||
| // Create a temporary HTML file with the Mermaid code | ||
| string tempHtmlFile = Path.Combine(Path.GetTempPath(), "mermaid_temp.html"); | ||
| try | ||
| { | ||
| await File.WriteAllTextAsync(tempHtmlFile, htmlContent); | ||
|
|
||
| // Launch Puppeteer-Sharp with a headless browser to render the Mermaid diagram | ||
| using (var browser = await Puppeteer.LaunchAsync(new LaunchOptions { Headless = true })) | ||
| using (var page = await browser.NewPageAsync()) | ||
| { | ||
| await page.GoToAsync($"file://{tempHtmlFile}"); | ||
| await page.WaitForSelectorAsync(".mermaid"); // Wait for Mermaid to render | ||
| await page.ScreenshotAsync(outputFilePath, new ScreenshotOptions { FullPage = true }); | ||
| } | ||
| } | ||
| catch (IOException ex) | ||
| { | ||
| throw new IOException("An error occurred while accessing the file.", ex); | ||
| } | ||
| catch (Exception ex) // Catch any other exceptions that might occur | ||
| { | ||
| throw new InvalidOperationException( | ||
| "An unexpected error occurred during the Mermaid diagram rendering.", ex); | ||
| } | ||
| finally | ||
| { | ||
| // Clean up the temporary HTML file | ||
| if (File.Exists(tempHtmlFile)) | ||
| { | ||
| File.Delete(tempHtmlFile); | ||
| } | ||
| } | ||
|
|
||
| return outputFilePath; | ||
| } | ||
| } |
165 changes: 165 additions & 0 deletions
165
dotnet/src/Experimental/Process.Core/Tools/ProcessVisualizationExtensions.cs
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,165 @@ | ||
| // Copyright (c) Microsoft. All rights reserved. | ||
|
|
||
| using System; | ||
| using System.Linq; | ||
| using System.Text; | ||
|
|
||
| namespace Microsoft.SemanticKernel.Process.Tools; | ||
|
|
||
| /// <summary> | ||
| /// Provides extension methods to visualize a process as a Mermaid diagram. | ||
| /// </summary> | ||
| public static class ProcessVisualizationExtensions | ||
| { | ||
| /// <summary> | ||
| /// Generates a Mermaid diagram from a process builder. | ||
| /// </summary> | ||
| /// <param name="processBuilder"></param> | ||
| /// <param name="maxLevel">The maximum indentation level to reach for nested processes, 1 is basically no nesting</param> | ||
| /// <returns></returns> | ||
| public static string ToMermaid(this ProcessBuilder processBuilder, int maxLevel = 2) | ||
| { | ||
| var process = processBuilder.Build(); | ||
| return process.ToMermaid(maxLevel); | ||
| } | ||
|
|
||
| /// <summary> | ||
| /// Generates a Mermaid diagram from a kernel process. | ||
| /// </summary> | ||
| /// <param name="process"></param> | ||
| /// <param name="maxLevel">The maximum indentation level to reach for nested processes, 1 is basically no nesting</param> | ||
| /// <returns></returns> | ||
| public static string ToMermaid(this KernelProcess process, int maxLevel = 2) | ||
| { | ||
| // Check that the maximum level is at least 1 | ||
| if (maxLevel < 1) | ||
| { | ||
| throw new InvalidOperationException("The maximum indentation level must be at least 1."); | ||
| } | ||
|
|
||
| StringBuilder sb = new(); | ||
| sb.AppendLine("flowchart LR"); | ||
|
|
||
| // Generate the Mermaid flowchart content with indentation | ||
| string flowchartContent = RenderProcess(process, 1, isSubProcess: false, maxLevel); | ||
|
|
||
| // Append the formatted content to the main StringBuilder | ||
| sb.Append(flowchartContent); | ||
|
|
||
| return sb.ToString(); | ||
| } | ||
|
|
||
| /// <summary> | ||
| /// Renders a process and its nested processes recursively as a Mermaid flowchart. | ||
| /// </summary> | ||
| /// <param name="process">The process to render.</param> | ||
| /// <param name="level">The indentation level for nested processes.</param> | ||
| /// <param name="isSubProcess">Indicates if the current process is a sub-process.</param> | ||
| /// <param name="maxLevel">The maximum indentation level to reach for nested processes, 1 is basically no nesting</param> | ||
| /// <returns>A string representation of the process in Mermaid syntax.</returns> | ||
| private static string RenderProcess(KernelProcess process, int level, bool isSubProcess, int maxLevel = 2) | ||
| { | ||
| StringBuilder sb = new(); | ||
| string indentation = new(' ', 4 * level); | ||
|
|
||
| // Dictionary to map step IDs to step names | ||
| var stepNames = process.Steps | ||
| .Where(step => step.State.Id != null && step.State.Name != null) | ||
| .ToDictionary( | ||
| step => step.State.Id!, | ||
| step => step.State.Name! | ||
| ); | ||
|
|
||
| // Add Start and End nodes only if this is not a sub-process | ||
| if (!isSubProcess) | ||
| { | ||
| sb.AppendLine($"{indentation}Start[\"Start\"]"); | ||
| sb.AppendLine($"{indentation}End[\"End\"]"); | ||
| } | ||
|
|
||
| // Process each step | ||
| foreach (var step in process.Steps) | ||
| { | ||
| var stepId = step.State.Id; | ||
| var stepName = step.State.Name; | ||
|
|
||
| // Check if the step is a nested process (sub-process) | ||
| if (step is KernelProcess nestedProcess && level < maxLevel) | ||
| { | ||
| sb.AppendLine($"{indentation}subgraph {stepName.Replace(" ", "")}[\"{stepName}\"]"); | ||
| sb.AppendLine($"{indentation} direction LR"); | ||
|
|
||
| // Render the nested process content without its own Start/End nodes | ||
| string nestedFlowchart = RenderProcess(nestedProcess, level + 1, isSubProcess: true, maxLevel); | ||
|
|
||
| sb.Append(nestedFlowchart); | ||
| sb.AppendLine($"{indentation}end"); | ||
| } | ||
| else if (step is KernelProcess nestedProcess2 && level >= maxLevel) | ||
| { | ||
| // Render a subprocess step | ||
| sb.AppendLine($"{indentation}{stepName}[[\"{stepName}\"]]"); | ||
| } | ||
| else | ||
| { | ||
| // Render the regular step | ||
| sb.AppendLine($"{indentation}{stepName}[\"{stepName}\"]"); | ||
| } | ||
|
|
||
| // Handle edges from this step | ||
| if (step.Edges != null) | ||
| { | ||
| foreach (var kvp in step.Edges) | ||
| { | ||
| var eventId = kvp.Key; | ||
| var stepEdges = kvp.Value; | ||
|
|
||
| // Skip drawing edges that point to a nested process as an entry point | ||
| if (stepNames.ContainsKey(eventId) && process.Steps.Any(s => s.State.Name == eventId && s is KernelProcess)) | ||
| { | ||
| continue; | ||
| } | ||
|
|
||
| foreach (var edge in stepEdges) | ||
| { | ||
| string source = $"{stepName}[\"{stepName}\"]"; | ||
| string target; | ||
|
|
||
| // Check if the target step is the end node by function name | ||
| if (edge.OutputTarget.FunctionName.Equals("end", StringComparison.OrdinalIgnoreCase) && !isSubProcess) | ||
| { | ||
| target = "End[\"End\"]"; | ||
| } | ||
| else if (stepNames.TryGetValue(edge.OutputTarget.StepId, out string? targetStepName)) | ||
| { | ||
| target = $"{targetStepName}[\"{targetStepName}\"]"; | ||
| } | ||
| else | ||
| { | ||
| // Handle cases where the target step is not in the current dictionary, possibly a nested step or placeholder | ||
| // As we have events from the step that, when it is a subprocess, that go to a step in the subprocess | ||
| // Those are triggered by events and do not have an origin step, also they are not connected to the Start node | ||
| // So we need to handle them separately - we ignore them for now | ||
| continue; | ||
| } | ||
|
|
||
| // Append the connection | ||
| sb.AppendLine($"{indentation}{source} --> {target}"); | ||
| } | ||
| } | ||
| } | ||
| } | ||
|
|
||
| // Connect Start to the first step and the last step to End (only for the main process) | ||
| if (!isSubProcess && process.Steps.Count > 0) | ||
| { | ||
| var firstStepName = process.Steps.First().State.Name; | ||
| var lastStepName = process.Steps.Last().State.Name; | ||
|
|
||
| sb.AppendLine($"{indentation}Start --> {firstStepName}[\"{firstStepName}\"]"); | ||
| sb.AppendLine($"{indentation}{lastStepName}[\"{lastStepName}\"] --> End"); | ||
| } | ||
|
|
||
| return sb.ToString(); | ||
| } | ||
| } |