The Missing Control: A Design Proposal for Tool-Output Redaction in AI Agents

This is the fifth and final piece in a series on AI agent security. The previous pieces covered read-scope enumeration, the confused deputy problem, benchmark methodology critique, and the agent identity gap. This piece is different: it is a design proposal.

The problem

The tool-call boundary is where an agent’s data exposure is densest and least controlled. Nobody has published a ranking of exfiltration vectors by volume, and I am not going to invent one. What I can show is that every control in the previous four pieces stops short of this boundary, and that the boundary is where the data actually is.

Consider what happens when an agent calls a database query tool. The tool returns a JSON object containing customer names, email addresses, account balances, and internal identifiers. The model reads this output. It reasons about it. It incorporates it into its next action: drafting an email, populating a report, forwarding data to another tool.

Every control we have built to secure agents (prompt injection defences, identity verification, read-scope enumeration, confused deputy audits) is bypassed at this moment. Not because those controls are wrong. Because they protect the wrong boundary.

Prompt injection defences protect the model from untrusted input in its prompt. Identity verification proves the agent is who it claims to be. Read-scope enumeration tells you what the agent is authorized to read. Confused deputy audits tell you whether the agent is using its authority for the right purpose.

None of them ask the question: does the model need to see the raw value of this field, or just enough structure to reason about it?

Simon Willison named the shape of it in June 2025 as the lethal trifecta: an agent with access to private data, exposure to untrusted content, and a way to talk to the outside world turns any tool output into a route out Willison, Jun 2025. The OWASP Top 10 for Agentic Applications, released December 2025, split it across three entries: tool misuse, identity and privilege abuse, and supply chain OWASP, Dec 2025. Microsoft documented in June 2026 how poisoned MCP tools turn trusted agents into instruments of data loss while every individual action stays inside normal operating parameters Microsoft, Jun 2026.

No single system holds the vulnerability. It sits in the gap between where a tool finishes running and where the model starts reading.

What exists and why it is insufficient

Before proposing a solution, it is worth surveying what already exists. The landscape is fragmented, and every approach has real gaps.

CaMeL (Debenedetti et al., arXiv 2503.18813) sits one layer up: it stops a compromised plan from executing, not a compromised value from reaching the model’s context CaMeL, arXiv 2503.18813. Different problem, same boundary instinct.

Tool-boundary hooks

Prismor published work in April 2026 showing that AI coding agents create permanent records of API keys and tokens in their session logs. Their solution: PreToolUse and PostToolUse hooks that replace secrets with placeholders before the model sees them. The real secret exists only in the local subprocess; the model, the JSONL transcript, and the upstream API see only @@SECRET:stripe_key@@.

This works. Prismor estimates that combining real-time prevention with regular cleanup “can cover about 95% of the risk surface” Prismor, Apr 2026. That’s a vendor assessing itself with no published method, so read it as design intent rather than measurement. The approach is sound, and its limits are structural: it covers secrets only, not personal data or anything a regulator cares about. It’s built for Claude Code’s hook system and doesn’t carry over to other frameworks.

Streaming generation-time detection

The PRISM paper (Tapwal, Kumar & Maple, arXiv 2605.10614, May 2026) tackles a neighbouring problem: secrets leaking while the model writes. It treats leakage as risk building up token by token, combining 16 signals at each step into a score with green, yellow and red zones. On a 2,000-task adversarial benchmark it caught 71% of leaks with no false alarms at all, and let nothing through at the task level.

Good work. But it watches what the model writes, not what the model reads.

MCP proxy and gateway patterns

The MCP ecosystem has produced several proxy and gateway implementations:

• mcp-sanitization-proxy is an open-source security proxy that intercepts tool call responses before they reach the model’s context, detecting prompt injection payloads embedded in tool output.
• Microsoft’s MCP Security Gateway provides response scanning with BLOCK, SANITIZE, and LOG policies, with credential and PII leak blocking.
• MetaMCP aggregates multiple MCP servers with middleware and tool filtering.

These are useful for prompt injection detection in tool output, not data redaction. None provides a structured redaction taxonomy.

Traditional data loss prevention, adapted for AI

Several vendors have tried to bend existing data loss prevention tooling to fit:

• Coverity.ai AI DLP inspects prompts and responses before they reach external LLMs.
• Nightfall AI offers an agentic DLP platform for SaaS, gen AI, and endpoints.
• Microsoft Purview DLP inspects tool call parameters and can block sensitive data in outbound payloads.
• Presidio, the open-source library for finding personal data (started at Microsoft, now maintained under the Data Privacy Stack project), is what most of these pipelines actually run underneath: pattern rules for structured identifiers like card numbers, and a named-entity model for spotting names and places in free text.

DLP tools were engineered for files and networks. They watch outgoing email, file transfers, the clipboard. None of that fits here: the data is moving into a model, not through a file. These tools don’t understand tool calls, don’t see the agent’s reasoning, and can’t tell that redacting a response the model already read is a compliance problem.

Research is starting to reach the gap. APPA (arXiv 2607.24625, July 2026) works out the algebra for confining sensitive data as it flows through tool calls APPA, arXiv 2607.24625. Ghost-in-the-Agent / NeuroTaint (arXiv 2604.23374, April 2026) treats tool output as a source that has to be tracked wherever it goes afterwards NeuroTaint, arXiv 2604.23374. Neither is a production system.

The common failure

Every approach above shares one weakness: post-processing is treated as if it were prevention. The model read the raw value before any post-processing happened.

The interception point has to sit between tool execution and model context. Everything after that is bookkeeping.

The design proposal

This section proposes a five-layer architecture for tool-output redaction. It is a proposal, not a specification. The design choices are motivated by the gaps identified above, and every choice has trade-offs that I will discuss explicitly.

The ordering is the argument. Layer 2 is the only point where a raw value can still be stopped; every control drawn below it is describing or recording a decision that has already been made.

Layer 1: Data classification at source

Every tool declares the sensitivity categories of its output fields. Categories follow a six-tier model:

Category	Description	Default mode
PUBLIC	No restriction	FULL
INTERNAL	Internal use only	TOKENIZED
CONFIDENTIAL	Need-to-know	AGGREGATED
PII	Personal data	REDACTED
REGULATED	Compliance-bound	SUMMARIZED
CREDENTIAL	Secrets and keys	BLOCKED

Tools that cannot classify their output default to CREDENTIAL. This is a fail-closed design.

Trade-off: Classification accuracy is the weakest link. Classification metadata must be versioned, auditable, and subject to the same change-review process as tool descriptions.

Layer 2: Interception proxy

Every tool call passes through an interception proxy before its output reaches the model’s context. The proxy is a mandatory hop, not an optional middleware.

[Tool Execution] → [Interception Proxy] → [Model Context]
                        ↑
                   Policy Engine

The proxy reads the classification metadata from Layer 1 and applies the redaction policy from Layer 4. It supports six output modes, and these six are the only vocabulary used anywhere else in this proposal:

• FULL: Raw output passes through unchanged. Used for PUBLIC data or when policy grants the agent unrestricted access to that field.
• TOKENIZED: The value is replaced with a stable, reversible token (TOKEN-7f3a2b91) mapped to the original in a vault the model cannot reach. The model can reference and correlate the value across calls without seeing it.
• AGGREGATED: Individual values are replaced with a bucket, range, or statistic. The model can reason about magnitude and distribution without seeing exact figures.
• REDACTED: The value is replaced with [REDACTED-{category}]. The JSON structure is preserved; only values change. Unlike TOKENIZED, nothing is recoverable and nothing correlates across calls.
• SUMMARIZED: The field is replaced with a derived natural-language or structural summary that carries the reasoning-relevant content without the record itself.
• BLOCKED: The field or the entire output is rejected and never enters the context. Used for CREDENTIAL data or when policy denies access outright.

The proxy operates in streaming mode for large outputs. Tool responses are processed chunk-by-chunk as they arrive, not batched after completion.

Trade-off: I have found no published latency figures for redaction at this boundary. The design implication holds either way: give the proxy a fast path that only matches patterns, for structured fields, and a slow path that runs the model, for free text, so the expensive one runs only where it earns its keep.

Layer 3: Redaction strategies

Layer 2 defines what each mode produces. This layer is about how:

• Exact replacement (REDACTED): Substitute [REDACTED-{category}]. Simplest and most conservative.
• Vault-backed tokenization (TOKENIZED): Substitute a stable token mapped to the original in a vault the model cannot reach. The same value always gets the same token, so the model can correlate records.
• Bucketing and statistics (AGGREGATED): Replace exact values with a range. Bucket boundaries are a security parameter.
• Derivation (SUMMARIZED): Replace the record with derived content. The only mode that requires understanding the field’s meaning.
• Structural preservation is orthogonal: a JSON array of 100 records stays a JSON array of 100 records; only leaf values change.
• Streaming is likewise orthogonal: the proxy processes chunk-by-chunk and must understand the output format.

Trade-off: No single strategy is optimal for all use cases. The policy engine (Layer 4) selects the strategy based on agent identity, tool classification, data sensitivity, and context.

Layer 4: Policy engine

The policy engine combines agent identity, tool classification, data sensitivity, and context into redaction decisions.

policies:
  - name: "healthcare-agent-policy"
    agent: "patient-intake-agent"
    rules:
      - when:
          tool: "patient_database"
          field_sensitivity: "PII"
        then: "REDACTED"
      - when:
          tool: "patient_database"
          field_sensitivity: "REGULATED"
        then: "SUMMARIZED"
      # Per-field override: the intake agent's task genuinely needs
      # the diagnosis, so this one REGULATED field is exempted from
      # the tier default. Overrides are how a policy narrows a broad
      # category decision, and every one is a written, auditable
      # exception rather than a quiet one.
      - when:
          tool: "patient_database"
          field: "diagnosis"
        then: "FULL"
      - when:
          tool: "patient_database"
          field_sensitivity: "CREDENTIAL"
        then: "BLOCKED"

  - name: "analyst-read-only-policy"
    agent: "financial-analyst"
    rules:
      - when:
          tool: "revenue_database"
          confidence: "> 0.9"
        then: "FULL"
      - when:
          tool: "revenue_database"
          confidence: "<= 0.9"
        then: "AGGREGATED"

Key design principles:

• Fail-closed: If a policy cannot be evaluated, the proxy defaults to the most restrictive redaction mode.
• Per-agent granularity: Policies are scoped to agent identity, not just role.
• Context-aware: Policies can consider runtime context: confidence scores, user role, time of day.
• Versioned and auditable: Policy changes are versioned, reviewed, and logged.

Trade-off: Ten agents, twenty tools and six tiers gives 1,200 distinct combinations. The policy language needs grouping, inheritance and tier defaults.

Layer 5: Audit and feedback

Every redaction decision is logged. The audit record includes:

• Redacted value (the placeholder, not the original)
• Policy applied (rule name, version)
• Agent identity
• Tool name and version
• Timestamp
• Confidence score of the agent at the time of the call

This audit log serves three purposes:

1. Compliance: Demonstrating that sensitive data was redacted.
2. Debugging: Understanding why a tool call was redacted.
3. Feedback: The agent can flag over-redaction; the policy engine can adjust strategies.

Trade-off: The audit log must be access-controlled and encrypted. It should never contain the original unredacted values.

Concrete examples

Example 1: Healthcare agent querying patient records

Example 1: Healthcare agent querying patient records

Without redaction:

{
  "patient_id": "P-12345",
  "name": "Jane Doe",
  "ssn": "123-45-6789",
  "diagnosis": "Type 2 Diabetes",
  "balance": 4567.89
}

With redaction:

{
  "patient_id": "TOKEN-7f3a2b91",
  "name": "[REDACTED-PII]",
  "ssn": "[REDACTED-PII]",
  "diagnosis": "Type 2 Diabetes",
  "balance": "[4500-5000]"
}

Four of the six modes are visible here. patient_id is TOKENIZED, name and ssn are REDACTED, balance is AGGREGATED, and diagnosis is FULL via a per-field override. Overrides are the honest part: every real deployment will have some, and a design that pretends otherwise will simply be bypassed.

Example 2: Financial agent querying revenue data

Without redaction:

{
  "company": "Acme Corp",
  "revenue_2025": 12345678.90,
  "revenue_2024": 11234567.89,
  "growth_rate": 9.88
}

A naive redaction pass buckets the two revenue figures and leaves the growth rate alone:

{
  "company": "Acme Corp",
  "revenue_2025": "[10M-15M]",
  "revenue_2024": "[10M-15M]",
  "growth_rate": 9.88
}

This is worse than it looks. The growth rate is computed from the two fields that were just bucketed. Publish it exactly and you tie the two ranges together. The corrected output buckets the derived field on the same pass:

{
  "company": "Acme Corp",
  "revenue_2025": "[10M-15M]",
  "revenue_2024": "[10M-15M]",
  "growth_rate": "[5-10%]"
}

Redact a response as a whole, not field by field. Anything you can calculate from a redacted field is itself a way out. This is the hardest part of Layer 3 to get right.

If the agent’s confidence is above the 0.9 threshold, the policy allows FULL output.

Example 3: Coding agent querying a secrets store

Without redaction:

{
  "service": "payment-api",
  "api_key": "sk_live_abc123def456ghi789jkl"
}

With redaction:

{
  "service": "payment-api",
  "api_key": "[BLOCKED-CREDENTIAL]"
}

Or, if the agent needs to reference the key without seeing it:

{
  "service": "payment-api",
  "api_key": "TOKEN-9c4e1d72",
  "key_exists": true
}

Trade-offs and open problems

This proposal is honest about what it does not solve.

Latency

Every tool call goes through a proxy. Streaming helps. I have no published numbers for this boundary: matching patterns is much cheaper than running a model over free text. Measure before you commit to an architecture.

Lost meaning

Redaction destroys information. The layer has to separate “data the agent needs to reason with” from “data the agent shouldn’t see,” and that line is often blurry.

Classification accuracy

Everything depends on tools describing their own output honestly. A tool that reports PUBLIC while returning personal data makes the whole layer blind.

Ways around it

Redaction shrinks the surface. It doesn’t remove it. A model can work out sensitive values from context even when every field has been redacted individually. Any deployment that treats redaction as removal has reinvented the post-processing mistake.

Standardization

No existing standard defines tool output classification. The MCP specification is the most natural home, but the specification process is slow.

Model uncertainty

The model’s confidence is a security signal, and nearly every system throws it away. When an agent is unsure, redaction should tighten. I know of no system that does this.

What this proposal builds on

• Piece 1 established that “read-only” is a floor, not a boundary. Knowing what the agent may read still leaves you needing to control what it actually sees come back.
• Piece 2 showed that every tool parameter the model fills in is an unverified claim about authority. The mirror image: every tool output the model reads is data that may need redacting.
• Piece 3 showed that a maxed-out benchmark isn’t robustness. This layer is built for environments where the attacks move.
• Piece 4 established that workload identity proves which process is on the wire and says nothing about what it will do. The policy engine here takes identity as one input among several.

Call for collaboration

This is a proposal, not a finished design. Here is what I would like the community to do:

1. Attack the categories. Are six tiers the right granularity? Too fine, too coarse? What’s missing?
2. Measure the proxy. This is where the proposal is weakest. Somebody has to build it and measure. Does redacting a stream hold up? What does each mode actually cost?
3. Try the policy engine. The language has to be expressive enough for real deployments and plain enough for the person on call at 2 a.m.
4. Test what you lose. The lost-meaning problem is real and badly understood. How much can you redact before the agent’s reasoning falls apart?
5. Standardise it. If any of this has merit it needs a home, and MCP is the obvious one.

The tool-call boundary is where the data actually flows, and right now nothing sits between a tool’s output and the model’s context. For most deployments the answer is nothing. That is what needs building.

References

• Simon Willison, “The Lethal Trifecta for AI Agents” (Jun 16, 2025): https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/
• Microsoft Security Blog, “Securing AI agents: When AI tools move from reading to acting” (Jun 30, 2026): https://www.microsoft.com/en-us/security/blog/2026/06/30/securing-ai-agents-ai-tools-move-from-reading-acting/
• OWASP Top 10 for Agentic Applications (Dec 9, 2025): https://genai.owasp.org/2025/12/09/owasp-top-10-for-agentic-applications/
• Prismor, “Your AI Agent Remembers Your Secrets” (Apr 13, 2026): https://www.prismor.dev/blog/tool-boundary-redaction-ai-agents
• Tapwal, Kumar & Maple, “PRISM: Generation-Time Detection and Mitigation of Secret Leakage in Multi-Agent LLM Pipelines” (arXiv 2605.10614, May 2026): https://arxiv.org/abs/2605.10614
• Kravchenko et al., “Agentic Permissions Policy Algebra for Taint Confinement in LLM Agents” (APPA, arXiv 2607.24625, Jul 2026): https://arxiv.org/abs/2607.24625
• Cai et al., “Ghost in the Agent: Redefining Information Flow Tracking for LLM Agents” (NeuroTaint, arXiv 2604.23374, Apr 2026): https://arxiv.org/abs/2604.23374
• CaMeL, “Defeating Prompt Injections by Design” (Debenedetti et al., arXiv 2503.18813): https://arxiv.org/abs/2503.18813
• mcp-sanitization-proxy: https://github.com/dhiaa2/mcp-sanitization-proxy
• Microsoft MCP Security Gateway: https://microsoft.github.io/agent-governance-toolkit/tutorials/07-mcp-security-gateway/
• MetaMCP: https://mcp.directory/blog/metamcp-complete-guide-2026
• Presidio (started at Microsoft, now maintained under the Data Privacy Stack project): https://data-privacy-stack.github.io/presidio/

Previous pieces in this series

• Piece 1: What “read-only” actually reaches — Read scope enumeration method for Kubernetes
• Piece 2: Finding the confused deputy in your own agent — Taxonomy + hands-on test
• Piece 3: What the prompt injection defence numbers actually say — Evidence review + methodology critique
• Piece 4: What it would take to give my agent a real identity — Standards gap analysis (SPIFFE, AIMS, WIMSE)