feat: Add policy-based automatic PII detection and dynamic masking

[Deeven-Seru]

Feature Request

Is your feature request related to a problem? Please describe.
Enterprises using MCP Toolbox to connect AI agents to databases face significant privacy compliance challenges (GDPR, CCPA, HIPAA, etc.) when query results contain personally identifiable information (PII). While MCP Toolbox provides secure access controls, it lacks intelligent output filtering to automatically detect and mask PII in query results before they reach LLMs or applications.

Describe the solution you'd like

Add a policy-based automatic PII detection and dynamic masking layer that:

1. Scans query results for PII using configurable patterns, dictionaries, and optional NLP
2. Applies dynamic masking (partial/full/tokenization) based on data classification policies
3. Respects user/role-based permissions for different masking levels
4. Preserves data utility where possible (e.g., statistical masking for analytics)
5. Logs masking actions for audit trails without exposing raw sensitive data

Example Policy Configuration (in tools.yaml):

kind: pii-policy
name: gdpr-compliant
rules:
  - pattern: "\\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\\.[A-Z|a-z]{2,}\\b"
    type: EMAIL
    action: MASK_PARTIAL  # Shows first/last char: j*****n@example.com
  - pattern: "\\b\\d{3}-\\d{2}-\\d{4}\\b"
    type: SSN
    action: MASK_FULL     # Shows ***-**-**** 
  - column: credit_card_number
    type: CREDIT_CARD
    action: TOKENIZE      # Reversible tokenization for authorized users

Integration:

• As a configurable middleware layer in the tool execution pipeline
• Works with all prebuilt tools (execute_sql, query_table, etc.)
• Compatible with custom tools framework
• Role-based policy application

Describe alternatives you've considered

• External services like Shynk (issue Add GDPR-compliant PII protection to your MCP server with Shynk #2895) but preference for native, zero-dependency solution
• Manual tool-level masking (doesn't scale, error-prone)
• Application-level filtering (defeats purpose of Toolbox as unified data layer)

Additional context

This feature would transform MCP Toolbox from a secure conduit into an intelligent privacy-aware data fabric, addressing a critical gap for regulated industries adopting AI agents. It aligns with the project's goals of enhanced security and enterprise readiness.

[averikitsch]

Hi @Deeven-Seru thanks for the feature request. Just to confirm, this request is to filter PII from returned database queries before returning to the agent/context? A couple of questions

• Could you provide more info on what you want from Role-based policy application?
• Do you envision this to be applied to all tools or specific tools?
• Would a integration with DLP API be interesting to you?

[Deeven-Seru]

Hello @averikitsch ,

Thanks for following up on the feature request. To confirm - yes, this is about filtering PII from returned database/query results before they enter the agent's context or are presented to you. This prevents accidental exposure of sensitive information in conversations, logs, or agent memory.

Let me address your questions:

Role-based policy application:
This would mean defining different PII handling rules based on user/role context. For example:

• Admin/privileged roles might see full data (with audit logging)
• Standard agent roles see redacted/masked PII
• Specific roles (like finance vs HR) might see different PII fields based on their job function
• Policies could be defined per data source, table, or column with role mappings

This ensures the principle of least privilege while maintaining necessary functionality for authorized users.

Scope of application:
I envision this applying primarily to tools that return potentially sensitive data:

• Database query tools (when we implement them)
• File read operations (read, file_fetch) on documents that might contain PII
• Web fetch results that return personal data
• Any tool that returns structured/unstructured data from external sources

Tools that perform actions without returning data (like write, exec for commands) or return only metadata/status would
likely not need PII filtering. The goal is to protect data in transit to the agent's context, not to hinder operational
capabilities.

DLP API integration:
This could definitely be interesting for:

• Automated, accurate PII detection beyond simple regex patterns
• Handling complex PII formats
• Continuous learning from false positives/negatives
• Compliance with regulations like GDPR, HIPAA, CCPA

However, I'd want to evaluate:

• Performance impact (latency added to queries)
• Cost considerations for API usage
• Whether simpler rule-based filtering suffices for our initial use cases
• Data privacy implications of sending potentially sensitive data to a third-party DLP service

A hybrid approach might work well: basic pattern filtering for common PII (SSN, email, phone) built-in, with optional DLP API integration for advanced/compliance-heavy scenarios.

[averikitsch]

Thanks for the details. How do you envision defining and aligning the roles like admin vs user?

[Deeven-Seru]

hi @averikitsch I would say that the Roles will align with the existing auth layer in MCP Toolbox and mapping it to policy tiers like admin, analyst, agent defined within the pii-policy config itself. Each tier will specify which masking actions apply , making role definitions co-located with the policies they use or goovern with. This avoids building a separate RBAC system and instead uses on whatever identity context is already flowing through the Toolbox request pipeline.

[bmtriet]

I did a small public-code/docs pass on the role-alignment question. This is not a certified security audit, and I did not use or need any private data.

One concrete way to keep this from becoming a second RBAC system would be to treat the existing auth claims as the policy input, then keep PII policy roles as local tiers:

• auth layer: validate token, audience, scopes, and expose claims
• policy layer: map selected claims/scopes/groups to masking tiers such as full, partial, tokenized, or deny-field
• tool-result layer: apply masking before the result is serialized into MCP tool output / agent context
• audit layer: log policy id, tier, field/type masked, count, and decision reason, but not raw PII values

The public docs already describe MCP auth with JWT validation, audience, and scopesRequired, and internal/auth/auth.go exposes claims via GetClaimsFromHeader(...) / ValidateMCPAuth(...). That seems like a clean boundary: PII policy should consume normalized claims, not own identity.

A small acceptance-test matrix could cover the feature without sensitive sample data:

1. same query result + scope=admin -> unmasked allowed fields, audit event emitted
2. same result + scope=agent -> email/phone masked before tool output
3. unknown role/scope -> fail closed to the most restrictive tier
4. configured column rule beats regex guess when both match
5. DLP integration disabled/unavailable -> deterministic local rules still run, or request fails closed if policy requires DLP

No worries if this is already where the design is headed; I’m leaving it as a bounded public checklist because the role-mapping question looked like the key decision point.

[Deeven-Seru]

@averikitsch canu confirm so that I can proceed with a pr

[averikitsch]

Hi @Deeven-Seru , could you outline the API for what you hope to support in the Policy Configuration?

[Deeven-Seru]

Hi @averikitsch, here's the API outline for the Policy Configuration:

• New config kind: piiPolicy — added as a new case alongside source, tool, toolset in config.go's kind switch
• Fields: name, defaultTier (fail-closed default), tiers (list), and rules (list)
• Tiers: Each tier has a name (e.g. admin, analyst, agent), matchClaims (maps to scopes/claims from internal/auth), and an action
• Actions: unmask (raw data + audit log), partial_mask (format-preserving), full_mask (complete redaction), deny_field (omit entirely)
• Rules: Each rule has a type (EMAIL, SSN, PHONE, etc.) with either a pattern (regex) or column (column-name match for structured results)
• Tool binding: Tools reference a policy via an optional piiPolicy field in their existing config
• Auth integration: Tiers consume claims already exposed by GetClaimsFromHeader in internal/auth/auth.go — no new RBAC
• Execution: Masking runs as a post-Invoke step before results are serialized into MCP output
• DLP: Optional dlpProvider block for Google DLP integration, with deterministic local rules as fallback

we should start with covering config schema, pattern engine, and middleware for SQL tools if this direction works.

[averikitsch]

This sounds good to me. A couple of questions and thoughts

1. How is this enabled? Is it enabled on all tools if set in the config or do we bind it to a source or to individual tools?
2. let's hold off on a DLP integration right now
3. would you like to write a blog for this functionality via https://medium.com/@mcp_toolbox

[Deeven-Seru]

Hi @averikitsch,

1. to enable it I would propose binding it directly to individual tools via a new piiPolicy: <policy_name> field in the tool configuration block. This gives deep control — for example, you could enable masking for execute_sql or ask_data_insights where sensitive data might be queried, but leave it off for metadata tools like list_tables to avoid unnecessary processing overhead.

2. DLP Integration: Agreed, we'll keep it out right now.

3. Blog Post: I would love to! I'll put together a draft explaining the architecture, the motivation behind a privacy-aware MCP layer, and some examples of how to configure the tiers etc