Skip to content

MCP Tools in Drills: Drill Update - #3324

Open
dynamicdhx wants to merge 3 commits into
mainfrom
users/dhruvbharuka/drillupdate
Open

MCP Tools in Drills: Drill Update#3324
dynamicdhx wants to merge 3 commits into
mainfrom
users/dhruvbharuka/drillupdate

Conversation

@dynamicdhx

Copy link
Copy Markdown
Contributor

What does this PR do?

[Provide a clear, concise description of the changes]

[Add additional context, screenshots, or information that helps reviewers]

GitHub issue number?

[Link to the GitHub issue this PR addresses]

Pre-merge Checklist

  • Required for All PRs
    • Read contribution guidelines
    • PR title clearly describes the change
    • Commit history is clean with descriptive messages (cleanup guide)
    • Added comprehensive tests for new/modified functionality
    • Created a changelog entry if the change falls among the following: new feature, bug fix, UI/UX update, breaking change, or updated dependencies. Follow the changelog entry guide
  • For MCP tool changes:
    • One tool per PR: This PR adds or modifies only one MCP tool for faster review cycles
    • Updated servers/Azure.Mcp.Server/README.md and/or servers/Fabric.Mcp.Server/README.md documentation
    • Validate README.md changes running the script ./eng/scripts/Process-PackageReadMe.ps1. See Package README
    • For new or modified tool descriptions, ran ToolDescriptionEvaluator and obtained a score of 0.4 or more and a top 3 ranking for all related test prompts
    • For tools with new names, including new tools or renamed tools, update consolidated-tools.json
    • For renamed tools, follow the Tool Rename Checklist and tag the PR with the breaking-change label
    • For new tools associated with Azure services or publicly available tools/APIs/products, add URL to documentation in the PR description
  • Extra steps for Azure MCP Server tool changes:
    • Updated command list in servers/Azure.Mcp.Server/docs/azmcp-commands.md
    • Ran ./eng/scripts/Update-AzCommandsMetadata.ps1 to update tool metadata in azmcp-commands.md (required for CI)
    • Updated test prompts in servers/Azure.Mcp.Server/docs/e2eTestPrompts.md
    • 👉 For Community (non-Microsoft team member) PRs:
      • Security review: Reviewed code for security vulnerabilities, malicious code, or suspicious activities before running tests (crypto mining, spam, data exfiltration, etc.)
      • Manual tests run: added comment /azp run mcp - pullrequest - live to run Live Test Pipeline

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new Azure Resilience Management “drill update” command/tool to the ResilienceManagement toolset, wiring it through service-layer update logic and documenting it in the Azure MCP Server’s consolidated tooling and command docs.

Changes:

  • Introduces resilience_drill_update command implementation, options model, and RBAC mode enum.
  • Implements UpdateDrillAsync in ResilienceManagementService and exposes it via IResilienceManagementService.
  • Adds unit + live/integration tests and updates server docs/metadata (commands doc, e2e prompts, consolidated tools, changelog, README prompt list).

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tools/Azure.Mcp.Tools.ResilienceManagement/tests/Azure.Mcp.Tools.ResilienceManagement.Tests/ResilienceManagementCommandTests.cs Adds a live/integration-style test invoking resilience_drill_update.
tools/Azure.Mcp.Tools.ResilienceManagement/tests/Azure.Mcp.Tools.ResilienceManagement.Tests/Drills/DrillUpdateCommandTests.cs Adds unit tests for the new update command (includes a theory data issue).
tools/Azure.Mcp.Tools.ResilienceManagement/tests/Azure.Mcp.Tools.ResilienceManagement.Tests/assets.json Updates recorded test assets tag.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Services/ResilienceManagementService.cs Implements drill update via ARM patch + LRO completion.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Services/IResilienceManagementService.cs Adds UpdateDrillAsync contract.
tools/Azure.Mcp.Tools.ResilienceManagement/src/ResilienceManagementSetup.cs Registers DrillUpdateCommand in DI and command groups.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Options/Drills/DrillUpdateOption.cs Defines CLI/tool options for updating a drill.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Models/DrillRbacSetupMode.cs Adds RBAC setup mode enum used by options/service.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Commands/ResilienceManagementJsonContext.cs Registers the update command result type for STJ source-gen/AOT.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Commands/Drills/DrillUpdateCommand.cs Adds the MCP command implementation + validation + error mapping.
servers/Azure.Mcp.Server/src/Resources/consolidated-tools.json Maps resilience_drill_update into the consolidated “create/update resilience management resources” tool and updates its description.
servers/Azure.Mcp.Server/README.md Adds an example prompt for updating a resilience drill.
servers/Azure.Mcp.Server/docs/e2eTestPrompts.md Adds e2e prompt entries for resilience_drill_update.
servers/Azure.Mcp.Server/docs/azmcp-commands.md Documents the new azmcp resilience drill update command usage.
servers/Azure.Mcp.Server/changelog-entries/resilience-drill-update.yml Adds changelog entry for the new tool.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

{
"name": "create_azure_resilience_management_resources",
"description": "Create or update Azure Resilience Management resources, including usage plans and usage plan enrollments, for Azure service groups.",
"description": "Create or update Azure Resilience Management resources, including usage plans, usage plan enrollments, and drills, for Azure service groups.",
[InlineData("--service-group sg1 --rbac-setup-mode Manual", false)]
[InlineData("--service-group sg1 --drill drill1", false)]
[InlineData("--service-group sg1 --drill drill1 --subscription sub", false)]
[InlineData("")]
@dynamicdhx

Copy link
Copy Markdown
Contributor Author

Tool Description Evaluator Results

Test 1

Expected Tool: resilience_drill_update
Prompt: Update resilience drill <drill_name> in service group <service_group> to use manual RBAC setup

Results

Rank Score Tool Status
1 0.656079 resilience_drill_update EXPECTED
2 0.532486 resilience_drill_get
3 0.479764 resilience_drill_resource_get
4 0.382068 resilience_recovery_plan_get
5 0.370159 resilience_usageplan_enrollment_create

Test 2

Expected Tool: resilience_drill_update
Prompt: Associate recovery plan <recovery_plan_name> with resilience drill <drill_name> in service group <service_group>

Results

Rank Score Tool Status
1 0.611092 resilience_drill_update EXPECTED
2 0.575745 resilience_recovery_plan_get
3 0.538389 resilience_drill_get
4 0.496052 resilience_drill_resource_get
5 0.447162 resilience_usageplan_enrollment_create

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants