Improve learning messaging#2987
Closed
conniey wants to merge 8 commits into
Closed
Conversation
conniey
requested review from
JasonYeMSFT,
KarishmaGhiya,
RickWinter,
chidozieononiwu,
g2vinay,
msalaman and
xirzec
Contributor
There was a problem hiding this comment.
Pull request overview
This PR refines the “learn” experience across MCP tool loaders by reducing schema noise in learn-mode prompts and improving the guidance text so agents can more reliably pick tools/commands and required parameters.
Changes:
- Introduces
ToolCommandInfoand updates JSON source generation to support serializing a slimmed learn-mode view (name/description +properties/required). - Updates tool loaders (Server/Namespace/SingleProxy) to emit improved learn-mode messaging and to feed slimmer JSON into sampling prompts.
- Renames
SingleProxyToolLoader.GetToolListAsync(public surface used by tests) toGetMcpClientToolListAsyncand adjusts tests accordingly.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| core/Microsoft.Mcp.Core/src/Areas/Server/ServerJsonContext.cs | Registers ToolCommandInfo types for source-gen JSON serialization and cleans up ignore-condition usage. |
| core/Microsoft.Mcp.Core/src/Areas/Server/Models/ToolCommandInfo.cs | Adds a lightweight model for learn-mode serialization of command/tool info. |
| core/Microsoft.Mcp.Core/src/Areas/Server/Commands/ToolLoading/SingleProxyToolLoader.cs | Refactors learn-mode flow to cache tool lists and pass slim JSON into sampling. |
| core/Microsoft.Mcp.Core/src/Areas/Server/Commands/ToolLoading/ServerToolLoader.cs | Improves learn-mode text and switches sampling payloads to ToolCommandInfo. |
| core/Microsoft.Mcp.Core/src/Areas/Server/Commands/ToolLoading/NamespaceToolLoader.cs | Improves learn-mode text and switches sampling payloads to ToolCommandInfo. |
| core/Microsoft.Mcp.Core/src/Areas/Server/Commands/ToolLoading/CommandFactoryToolLoader.cs | Updates XML doc text to remove Azure-specific wording. |
| core/Azure.Mcp.Core/tests/Azure.Mcp.Core.Tests/Areas/Server/Commands/ToolLoading/SingleProxyToolLoaderTests.cs | Updates tests to match renamed API (GetMcpClientToolListAsync). |
| Next, identify the command you want to execute and run again with the "tool", "command", and "parameters" arguments. | ||
|
|
||
| {toolsJson} | ||
| {_cachedToolListsJson[tool]} |
| Next, identify the tool you want to learn about and run again with the "learn" argument and the "tool" name to get a list of available commands and their parameters. | ||
|
|
||
| {toolsJson} | ||
| {_cachedRootToolsJson} |
Comment on lines
286
to
290
| Text = """ | ||
| The "command" parameters are required when not learning | ||
| The "command" parameter is required when not learning. | ||
| Run again with the "learn" argument to get a list of available tools and their parameters. | ||
| To learn about a specific tool, use the "tool" argument with the name of the tool. | ||
| To learn about a specific tool, use the "command" argument with the name of the tool. | ||
| """ |
| if (SupportsSampling(request.Server) && !string.IsNullOrWhiteSpace(intent)) | ||
| if (request.Server.ClientCapabilities?.Sampling != null && !string.IsNullOrWhiteSpace(intent)) | ||
| { | ||
| var availableTools = await GetChildToolListAsync(request, tool, cancellationToken); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
[Provide a clear, concise description of the changes][Add additional context, screenshots, or information that helps reviewers]GitHub issue number?
[Link to the GitHub issue this PR addresses]Pre-merge Checklist
servers/Azure.Mcp.Server/README.mdand/orservers/Fabric.Mcp.Server/README.mddocumentationREADME.mdchanges running the script./eng/scripts/Process-PackageReadMe.ps1. See Package READMEToolDescriptionEvaluatorand obtained a score of0.4or more and a top 3 ranking for all related test promptsconsolidated-tools.jsonbreaking-changelabelservers/Azure.Mcp.Server/docs/azmcp-commands.md./eng/scripts/Update-AzCommandsMetadata.ps1to update tool metadata inazmcp-commands.md(required for CI)servers/Azure.Mcp.Server/docs/e2eTestPrompts.mdcrypto mining, spam, data exfiltration, etc.)/azp run mcp - pullrequest - liveto run Live Test Pipeline