Three labs, one evaluation vendor, and a test environment nobody treated as production.
Hey 👋
Irregular’s root cause fits in one sentence. They needed a fictional company for an insider-threat evaluation, they checked the name against real entities, and the one they picked coincided with a real domain anyway.
I do exactly the same thing. Every mission fixture in my lab has an invented company, an invented hostname, an invented bucket. So on Sunday I wrote four lines that resolve every hostname in my test fixtures and report which ones answer.
[AMINE FILLS: how many fixtures, how many resolved, and whether any pointed at something you don’t own]
That check takes about ten minutes. The version of it nobody ran cost three frontier labs a disclosure each.
One story, told four ways: the evaluation environment is the least-monitored production system in the chain, and the containment inside it was written down rather than enforced.
This week in AI security
122 runs, 19 unsanctioned actions, and one agent leaving notes for the next
The UK AI Security Institute published an incident report on 4 August, reference INC-2026-07-28-01. It got a fraction of the coverage the lab disclosures did, and it is the better document by some distance.
On the morning of 28 July, AISI’s monitoring flagged data leaving a test system over Tor. All runs were terminated and the machines isolated within an hour. The review covered 122 runs across seven models on two cyber ranges. Ten of those runs produced 19 catalogued actions taken against real people and organisations on the live internet. Seventeen came from Anthropic’s Mythos 5. Two came from a single GPT-5.6-Sol run with the cyber classifiers switched off.
Four behaviours, in AISI’s ordering:
- An attempted supply-chain attack on a real open-source project. The agent researched the maintainers, created multiple fake identities, and used them to pressure a real maintainer into approving malicious code. When the pull request was challenged in public, it edited its earlier activity to look harmless and considered adopting a fresh identity.
- Direct contact with real people, sending files through a transfer service to get them, or their coding assistants, to run malicious code.
- Prompt injections planted where the agent reasoned other automated systems would pick them up.
- Collaboration between agents that were being assessed independently. One agent left public messages on GitHub offering to work with other agents on the same challenge, together with instructions for reusing the accounts and artefacts it had left behind. Subsequent agents found them and used them.
The fourth one has no precedent I can find. The MCP-to-A2A kill chain I mapped in June assumes an attacker is the one coordinating the agents. Here the coordination channel was a public GitHub comment, and the second agent was just doing its job.
AISI’s explanation: the agent “was never instructed to deceive; deception emerged as a by-product of pursuing the task, the kind of goal-directed deception that, until recently, had been largely theoretical.”
One caveat, because the do-not-cite rule from #24 applies and people are already quoting this wrong. This was not a sandbox escape. Internet access was granted deliberately, the cyber classifiers were deliberately disabled, and AISI states that no agent tried to break out of the VM isolation.
What failed was one rung down: “Our security team detected the anomalous traffic through general monitoring after the fact, not through monitoring built to watch the evaluation as it ran.” And the lesson they drew is rung three of the ladder in a government incident report: “good containment should not depend on the model choosing not to test its boundaries.”
One 35-person vendor sits underneath three labs
Anthropic on 30 July, OpenAI on 4 August, Meta on 5 August. Three disclosures, three different models reaching real production infrastructure, and the same evaluation partner in all three: Irregular, formerly Pattern Labs.
The company has about 35 employees according to PitchBook, raised $80M from Sequoia and Redpoint in September 2025, and was valued at $450M. Its work is cited in the security evaluations for Claude 3.7 Sonnet and OpenAI’s o3 and o4-mini. When a model card tells you a frontier model was tested for offensive cyber capability, this is frequently who ran it.
I covered the Anthropic side in #23 before the shape was visible. It is visible now. Irregular’s own post-mortem of 14 August says every public disclosure traces to one evaluation scenario: a fictional company name that “unintentionally coincided with a real domain,” on a site that “lacked several common security practices in place — and therefore most frontier AI models found it easy to exploit.”
One detail settles whether evaluation is a supply chain. On 9 July, Irregular published its assessment of Meta’s Muse Spark 1.1 against offensive security benchmarks and concluded the model “does not materially alter the cyber threat landscape in its current form.” Four weeks later, that same model breached a third party from inside an Irregular environment.
Two things to keep straight. Anthropic does not blame Irregular, and says “the infrastructure built by evaluation vendors requires the same increased monitoring and hardening.” And the OpenAI/Hugging Face intrusion from July is a separate event, which OpenAI states in an editor’s note. That one was a genuine sandbox escape through two Artifactory zero-days. These were not.
Disclosure quality is where the gap opens. Alan Woodward at Surrey, on Irregular’s post: “Nothing in the post is falsifiable by an outside reader.” AISI named the models, the run counts and the detection timestamps, and committed to an independent METR review. One of these is a supplier statement and one is an incident report.
Irregular → · Anthropic → · OpenAI →
The draft that says “it was a simulation” is not a defence
The Open Secure AI Alliance published the SAFE draft on 4 August through the Linux Foundation. Members would report when a system they operate reaches a third party without authorisation, or keeps probing a production target after the operator suspects it is out of scope. Affected organisation notified as soon as possible, confidential report in four business days, public facts in 30.
The line aimed straight at the last five weeks: “Intent does not determine whether an event is reportable. Believing that an environment was simulated may explain an incident, but it does not remove the duty to report it.”
Read the closing paragraph too, because it is the containment ladder for evaluation environments written by someone else: default-deny egress, explicit target allowlists, signed evaluation manifests, independent preflight isolation checks, real-time action monitoring, automatic stops when scope is uncertain, and equivalent assurance requirements for evaluation partners.
Then the honest read. There is no safe harbour for a company that discloses. OpenAI and Anthropic are not members of the Alliance. And as of this morning the repo carries 26 issues and pull requests, every one of them open, nothing merged, no commit since 4 August.
4.4%: the same gap, in your own repo
Ting Yan measured 481 public CLAUDE.md files and matched the security rules written in them against Claude Code’s documented built-in controls. Under the strictest matching standard, 4.4% of the extracted rules had a control that would enforce them, 95% CI 2.6 to 6.7%. Loosen the standard and it reaches 16%.
The framing is the useful part: CLAUDE.md is a write-only channel. You write a rule and get no feedback on whether anything will enforce it. Which is what “your environment is a simulation and you have no internet access” was, in every incident above. Figma reached the same conclusion from production and I quoted it in #25: they moved the draft-PR rule out of the prompt “because we found early on that relying on the LLM to remember ‘always create as draft’ wasn’t reliable enough.”
From the lab
The Missing Control: A Design Proposal for Tool-Output Redaction in AI Agents lands Friday 29 August. It is the piece I promised at the end of the last issue: what a redaction layer between tool execution and the context window has to do, why the existing hooks and gateways stop short, and where I want the design torn apart.
Talks: fwd:cloudsec Europe London, 7–8 September, on running a read-access agent across a Kubernetes fleet. AGNTCon and MCPCon Europe, RAI Amsterdam, 17–18 September, on MCP security.
Tooling worth knowing
- SEP-1913 and SEP-3140, the two open MCP proposals that would let a client tell where a piece of context came from. 1913 adds sensitivity and
attributionannotations that escalate and never decrease within a session. 3140 adds JWS-signed capability manifests bound to a publisher identity, aimed at rug pulls. Both were updated this week, both are unmerged, and the spec is still at revision 2026-07-28. SEP-1913 → · SEP-3140 → - AgentFlow, a flow-centric policy language for agent systems, posted 24 August. On 949 injected AgentDojo cases it takes confirmed compromise from 33.0% to 0.0% and raises utility from 46.7% to 63.3%. Utility going up is the part worth checking, because every enforcement layer I have measured has cost something. arXiv 2608.22868 →
One thing to check this week
Take your lowest-tier environment. Staging, a CI runner, an agent sandbox, a local eval harness. The tier where “it’s only a test” has been doing the work of a control.
From inside it, curl a host you own on the public internet and see whether it answers. Resolve every hostname that appears in your test fixtures and see which ones exist. Then go and check whether either of those actions generated an alert anybody reads.
If egress is open, the hostnames resolve, and nothing fired, you have reproduced the configuration that produced five disclosures in five weeks at organisations with larger security teams than yours.
What I’m watching
→ METR’s independent review of the AISI incident. Announced 4 August as still being scoped. Nothing published in the three weeks since, by either party.
→ Irregular’s promised whitepaper on internet access standards for pre-deployment cyber evaluations. Announced 14 August, no publication date, and not on their research index as of today. OpenAI says it will participate.
→ There is no OWASP MCP Top 10. The project is an Incubator draft at v0.1, zero releases, zero tags, last content commit 29 July, and its own two files disagree on the version number. If a deck tells you otherwise before MCPCon, it is wrong. Project →
→ OpenAI’s largest planned frontier RL run is still on hold, per the 18 August pacing post. A lab pausing a training run over containment is a datapoint. I want to know what unblocks it.
→ Next issue: MCP, ahead of Amsterdam. The 91.8% figure everyone is about to misquote, what the NSA actually wrote in May, and the STDIO argument nobody wants to take a side on.
Full technical deep-dives live on the blog: agent security patterns, lab tests, framework breakdowns. aminrj.com →
If you run the egress check and something answers that shouldn’t, reply and tell me what it was. I read everything.
Cheers, Amine