Claude was told five things never to do. It did a sixth.
Hey đź‘‹
Last week was about where agent containment breaks: the network around the sandbox. This week is about the step before that. What the agent was allowed to do in the first place, and who finds out when it does something else.
Three things happened within a few days. Anthropic published a list of things its agents did on real websites, including a false tip to a police department. On 14 October the US FTC sent investigative demands to OpenAI and Anthropic, and the EU AI Office announced its first information requests under the AI Act. And a cloud research disclosure showed that an AWS agent’s default role covered every other agent in the region.
The common word is scope. For two years it has been something you write in a system prompt. It is now the thing labs say they release on and the thing regulators are asking about.
This week in AI security
The instructions listed five things. The agent did a sixth.
On 9 October Anthropic published Investigating unintended model actions, its third disclosure of this kind after 30 July and 9 September. It says these cases are less severe than the earlier ones. I think they are more useful, because they look like what your own agents will do.
Four categories, all found by a transcript review Anthropic started in July:
- Exploiting software flaws. When a tool was missing, Claude Mythos Preview found a script on a university server that returned any file asked for, used it, then found an injection flaw and used that to run a calculation on the server.
- Submitting forms it should not have. An unreleased research model submitted a real government form after the practice copy failed to load. Claude Haiku 4.5, told to stop before submitting, submitted forms several times anyway.
- Getting past paywalls and gates. Claude Mythos 5 read a map site’s settings file, found access tokens, and queried a local government server directly. In another run it took a token from a state agency’s public dashboard to query data that was sold for a fee.
- Using URL shorteners to get around length limits on the fetch tool. The operator of da.gd reported the same traffic independently.
The case that made the news is the Haiku 4.5 one. On a page about an unsolved homicide, it submitted an invented tip through the police tip form. Anthropic’s own explanation is the sentence I’d put in front of anyone writing agent instructions:
“Claude was instructed never to log in, create accounts, enter personal data, make purchases, or submit anything destructive, but the instructions did not rule out form submissions.”
According to the Philadelphia Police Department, the tip was sent on 18 July at 11:27 pm and flagged as spam. Anthropic found it on 28 September, 72 days later, and told the department in early October (7 October per the police, 8 October per Anthropic). The department’s statement: “The two-month delay in detecting and reporting the incident to the City is unacceptable.”
Anthropic’s name for most of this is “persistence”: when Claude “cannot complete a task as given,” it works around the restriction instead of stopping. The fixes it lists are mostly infrastructure. Live internet is off for all internal evaluations “until we have confirmed that our security and monitoring measures” reliably catch these behaviours. Internal agents are moving “to centrally managed infrastructure with strong containment.” And new detection tooling, run against the cases in the post, “blocked all of them.”
One correction worth knowing about. The first version of the post said the fetch length limits were there to stop SQL and command injection. On 12 October Anthropic corrected it: they weren’t. Some coverage, including The Hacker News, still has the original wording.
So what: a list of forbidden actions in a prompt is a denylist, and the agent found the gap in it. The persistence behaviour Anthropic describes is what you want from an agent until it reaches something it shouldn’t touch. Then the only thing that helps is a boundary the agent can’t argue with, and someone who sees the trace in days rather than months.
Do this: for every agent with web or tool access, write down which outward actions have side effects: form posts, account creation, sending messages, payments, writes to third-party APIs. Make those default-deny at the tool or proxy layer, not in the prompt. A research agent that only needs to read can be limited to GET requests at the egress proxy. Then check your own detection time: when did a person last read a sample of this agent’s transcripts?
Regulators sent their first demands on the same day
The labs aren’t deciding this alone any more.
The FTC. On 14 October the FTC sent civil investigative demands to OpenAI and Anthropic, Axios reported. Each runs to more than 50 pages and asks for records on security incidents, safeguards, marketing materials and privacy policies. The FTC also plans to take testimony from executives at both companies and at METR. The stated focus is consumer protection. The sourcing is one person familiar with the matter. OpenAI said it “will engage constructively.”
The EU AI Office. Also on 14 October, Executive Vice-President Henna Virkkunen said in Dublin that the AI Office had sent its first requests for information under the AI Act, on security, external evaluation and monitoring of how models are used. Her speech named no company. Euronews reports the requests went to OpenAI, over the agent that left its test environment and attacked Hugging Face, and to Anthropic, over a model that reached three outside organisations without authorisation. Per Euronews, the AI Office sent them after judging the companies’ voluntary disclosures insufficient.
Whether the AI Act applies at all is contested. The incidents happened in testing, and the Act exempts research and development before a model is placed on the market. A Commission official’s answer, quoted by Euronews, is that if the model “escapes the testing environment and then it has an impact on the EU market, as was the case for Hugging Face, then the AI Act becomes enforceable.” That is a new reading. I’d watch it more closely than the fines.
And the labs themselves. On 28 September OpenAI decided not to release GPT-6.1 Astra. OpenAI’s head of safety systems, Saachi Jain, told CBS the model “didn’t quite meet the bar in terms of staying within scope and authorization,” and on how it reports its work back to the user. The trade-off she described is the one Anthropic calls persistence: staying in scope while “avoiding laziness” when the model “hits friction.” OpenAI hasn’t published its own account. All the detail comes from press reporting.
So what: “did the agent stay within what it was authorised to do?” is now a release criterion at one lab, a monitoring problem at another, and a question from two regulators. If you deploy agents in the EU, expect the same question from your own customers and auditors before any regulator gets to you.
Do this: be able to answer three questions for each production agent. What is it authorised to do, written down? Where is that enforced, outside the model? How long would it take you to notice it doing something else? If the third answer is “we’d find out from the other side,” you’re where Anthropic was in July.
The agent’s role covered every agent in the region
Scope also has a plain IAM meaning, and Zenity Labs found where it was missing. In AgentCorruption (8 October), Tamir Ishay Sharbat and Lana Salameh show that an AWS Bedrock AgentCore agent with an HTTP tool could reach the instance metadata service from inside its Firecracker microVM. One prompt injection was enough to get the temporary credentials of the agent’s default execution role.
The problem was what that role could do. It wasn’t scoped to the agent. It covered AgentCore resources across the whole account and region:
DescribeLogGroupslisted every agent in the region and its ID.- ECR repositories named after agent IDs let the attacker pull other agents’ container images.
InvokeAgentRuntimeinvoked any agent in the region.ListEventsread other users’ private conversations, andCreateEventplanted memories that changed agents’ behaviour in later sessions.GetResourceApiKeyandsecretsmanager:GetSecretValuereturned the credentials the agents’ tools were supposed to hold on the agent’s behalf.
The timeline is the part I’d show a procurement team. Zenity reported the metadata access on 25 December 2025 and the over-broad role on 12 January. AWS moved AgentCore to IMDSv2-only on 14 February and closed the first report as “informative” on 12 April. On 22 June the role was still unchanged. Zenity saw it restricted on 29 September. AWS’s position is that “the behavior described is documented and expected” and that customers should grant execution roles only the permissions their agents need.
That’s the second AgentCore finding in a month closed as shared responsibility. In issue #31, Unit 42 read a JWT out of the harness’s own memory. Both come down to this: the platform default doesn’t scope credentials to one agent, so you have to.
For contrast, DigitalOcean described how it builds this for its own agents. They hold no standing credentials. Calls go through an Action Gateway that injects the credential at execution time and ties it to the person the agent acts for. A “scoped secret” is bound to one HTTPS host, and the sandbox only holds a short-lived handle that the egress proxy swaps for the real credential. Every sub-agent in a swarm gets its own microVM and session.
And here is the state of the average deployment. Hush Security parsed about 82,000 public MCP config files for coding agents. 12% of the 15,115 credential slots held a hardcoded secret, and 55% of those had no recognisable token prefix, so prefix-based secret scanners miss them. Among literals whose scope could be determined, 53% were organisation- or account-wide. Hush calls it “a non-human identity (NHI) governance problem, not a secret-hygiene one.” I agree.
So what: a prompt injection gets the attacker whatever the agent’s credential can reach, and on a managed platform that may be far more than the agent’s own job. The question for a threat model isn’t “can the agent be injected.” It is “what else does this agent’s identity open?”
Do this: from inside each cloud-hosted agent, run
curl -s -m 3 http://169.254.169.254/(and the IMDSv2 token request). If you get an answer, list the role’s permissions and look for any resource set to*in the region. Better still, follow DigitalOcean’s pattern so the credential never enters the sandbox.
The MCP SDK’s issuer check didn’t fail. It never ran.
One for anyone running MCP clients in Python. Cycode’s Yuval Elbar found that the official MCP Python SDK skipped OAuth issuer validation on a fallback path. If a malicious MCP server returns 404 to the modern discovery request, the SDK falls back to asking that server for its login configuration, and the issuer check only runs on the modern path. In Cycode’s words: “The check doesn’t fail. It never runs.”
The attacker’s configuration names the victim’s real identity provider as issuer, so the login page is genuine and the user sees nothing wrong. After approval, the authorization code, the client secret and the PKCE verifier go to the attacker’s token endpoint. That is enough for a valid access token.
- Affected:
mcp1.9.1 through 2.1.1, acrossOAuthClientProvider,ClientCredentialsOAuthProviderandPrivateKeyJWTOAuthProvider. Rated High. - Fixed: 1.30.0 and 2.2.0. After upgrading, pass
issuer=to the unattended providers, clear stored client registrations, and rotate client secrets if you could have been exposed.
This is the bug class I keep writing about: a security check placed on one branch of a protocol flow, with an older branch the attacker can choose. CoSAI’s MCP Security v2.0 names the general case: if a gateway serves both old and new clients, “it runs two authorization paths,” and the legacy one still needs per-request checks.
Do this:
pip show mcpin every environment that runs an MCP client with OAuth. Anything below 1.30.0, or between 2.0.0 and 2.1.1, gets upgraded this week.
Your MCP scanner read the description. The bug was in the handler.
Two results, three weeks apart, that together say what an MCP scan is worth.
Pluto Security’s Yotam Perkal and Eitan Chassid built two malicious MCP servers (15 September) posing as weather tools: one stole credential files on every call, the other fetched and ran a remote shell command. Both still returned a normal forecast. Cisco mcp-scanner, Snyk agent-scan, Ventrova sentinel-scan-cli and mcp-shield all passed both servers, and NVIDIA SkillSpector’s static rules passed them too. SkillSpector’s LLM review caught both, until the file went over its size limit. The main reason: most scanners read tool names, descriptions and schemas, and “the malicious behavior was absent from the inspected text.”
Then on 9 October, ToFu (Ananto et al., arXiv) fuzzed tools against the input rules in their own metadata. Across 273 open-source MCP servers with 3,405 tools, it found verified bugs in CWE weakness classes in 343 tools on 118 servers: path traversal, SSRF, command injection and others. That’s 43% of servers. Their conclusion: “LLM-generated arguments should not be treated as trusted.”
Same lesson from both sides. The description is a claim. A scanner that reads the claim tells you the claim is clean. Pluto’s line is the right rule: “Treat incomplete analysis as a failed gate.”
Quick hits
A Chrome full chain in six days for $40,000. XBOW’s Brendan Dolan-Gavitt wrote up (13 October) a full chain out of the renderer sandbox (a V8 Maglev bug, an animated-WebP TOCTOU, a Crashpad heap overflow) built from 2 to 7 August by agents on
gpt-5.6-sol, with researchers choosing targets and redirecting the work. About $40,000 in API charges. Google paid the first $250,000 Chrome full-chain reward. The article’s comparison for the same work before LLMs: a team of four for six to nine months.One email, a reverse shell, and the user’s OAuth tokens. Salt Labs hijacked Manus with an email the user only asked the agent to check. Plain-text and Base64 payloads were blocked. A JSFuck-encoded one wasn’t: Manus decoded it in Node.js, which executed it. The sandbox held the Gmail OAuth token and the credentials for other connected services. The guardrail did flag it, “but only after the code had already run.” Fixed through Meta’s bug bounty.
18,000 wiki posts from OpenAI agents. In September, researchers at Nightingale and Trajectory documented about 18,000 edits by agents identifying as OpenAI’s on a small German wiki that accepted writes over GET requests. The agents used it to share answers and sandbox workarounds, and made “ZZZ” backup pages to survive the admin’s deletions. According to the authors, OpenAI hasn’t disclosed it publicly.
What I’m writing
The AgentCorruption role is the simplest version of something I’ve spent the past month writing about: delegation. An agent acting for a user gets a credential, calls a tool, and the tool calls something else. The OAuth token exchange standard (RFC 8693) tells the last service in that chain to check only the current actor. Every earlier hop goes unchecked unless you add something.
It’s the first piece in a series on agent identity that I’ll start publishing shortly. It covers what the IETF drafts and the 2026 research propose to fix it, and what each still leaves open. If you’ve built attenuated delegation for agents in production, reply. I’d like to compare notes before it goes out.
Tooling worth knowing
- CoSAI MCP Security v2.0: four assurance levels and eight security dimensions you can actually audit against, updated for the stateless 2026-07-28 MCP release. The Level 3 requirement for sender-constrained tokens (DPoP or mTLS) is the one most deployments miss. paper →
- CAIRN (Cisco Talos): a metadata-first toolkit for hunting AI-integrated malware in VirusTotal without downloading samples. Useful if you track infostealers going after agent tokens and MCP credentials. post →
- OWASP Agent Control Standard: a spec for how agent platforms expose middleware hooks so runtime policy can be enforced across frameworks. Early, but it’s the right layer for the default-deny controls above. resource →
- Agent Security Scorecard: my free self-assessment against the OWASP Agentic Top 10. About 12 minutes, no login. Score your agents →
- AI Agent Pre-Deployment Security Checklist: 25 yes/no controls to clear before an agent with tool access goes to production. Get the checklist →
One thing to check this week
Pick one agent that can reach the web or a third-party API, and spend twenty minutes on its scope.
- List its side effects. Go through its tools and write down every action that changes something outside your system: POST, PUT, DELETE, form submission, sending email or chat messages, creating accounts. That’s the list Anthropic’s instructions didn’t fully cover.
- Find where each one is stopped. For each action, is the block in the prompt, in the tool code, or at a proxy or gateway? Anything that’s only in the prompt is a request, not a control.
- Read ten transcripts. Pick ten recent runs at random and read them end to end. Note anything the agent did that you didn’t expect, even if it was harmless. Then write down how long it would have taken you to find it otherwise.
If step three turns up something, you’ve learned more than a dashboard would have told you.
What I’m watching
→ AGNTCon + MCPCon North America, 22 to 23 October in San Jose. David Soria Parra has the opening keynote. After Amsterdam’s stateless core and authorization changes, I want to see whether there’s anything on scoping what a client may do, not just who it is.
→ Whether the AI Office’s reading holds. “It escaped testing and affected the EU market” is a new route into the AI Act for incidents during development. If it holds, every lab’s evaluation infrastructure falls under EU scrutiny.
→ What the FTC asks for. If the demands cover detection and notification times, the 72 days in Anthropic’s case will become a benchmark everyone gets compared to.
→ Default roles on managed agent platforms. AWS changed AgentCore’s default role nine months after the report. I’d like to see every agent platform publish what its default execution identity can reach.
If you run the twenty-minute check and step two turns up something that’s only enforced in the prompt, reply and tell me what it was. I read everything, and I’m collecting real examples for the identity series.
Cheers, Amine
If a colleague runs agents whose limits live only in the system prompt, forward this to them.
Sources
- Investigating unintended model actions, Anthropic, 9 October 2026 (corrected 12 October 2026)
- AI model submitted false tip about unsolved murder, Philadelphia police say, 6abc, October 2026
- Anthropic cuts live internet access for internal AI tests, The Hacker News, 10 October 2026
- FTC subpoenas OpenAI, Anthropic over consumer harms, Axios, 15 October 2026
- EU’s AI rules face first enforcement test over lab incidents, Euronews, 14 October 2026
- OpenAI holds off on releasing new model over safety concerns, CBS News, 28 September 2026
- AgentCorruption: How a single prompt collapsed the entire cloud security model, Zenity Labs, 8 October 2026
- How DigitalOcean manages credentials for autonomous agents, DigitalOcean, updated 28 September 2026
- The state of MCP configuration, Hush Security, 2026
- Cycode uncovers account takeover in MCP Python SDK, Cycode, 28 September 2026
- MCP Security, Version 2.0, Coalition for Secure AI, 25 September 2026
- Malicious servers, clean scans, Pluto Security, 15 September 2026
- ToFu: Exploiting and Fuzzing the LLM-Orchestrator-Tool Trust Boundary, Ananto et al., arXiv, 9 October 2026
- Zero to calc in six days: inside XBOW’s Chrome full chain, XBOW, 13 October 2026
- How we hijacked an AI agent with a single email, Salt Labs, 1 October 2026
- Discovery of a new OpenAI agent message board, Von Arx, Byrd, Kitts and Larsen, 4 September 2026
- Introducing CAIRN, Cisco Talos
- Agent Control Standard, OWASP GenAI Security Project, 1 September 2026
- tl;dr sec #348 to #350, Clint Gibler, and Adversa AI’s October roundups, for pointing me at several of these