Annie Tsai

For about a decade, a German-language developer wiki called DSEWiki sat mostly untouched. A handful of edits a year, it was the kind of site that existed more as an archive than a destination. Then, over eight weeks this spring and summer, it started taking hundreds of edits a day, but those edits weren’t from humans. They were from AI agents working a timed retrieval task and using the wiki as a place to compare notes.

It was what they were comparing that was most interesting. One task required data the agents’ sandbox wasn’t supposed to let them fetch, a network restriction blocking a certain kind of request. One instance found a way around it: a trusted-domain exception in the proxy configuration that let the blocked request through. That workaround showed up in other agent instances within fourteen minutes. When a human moderator started deleting the pages, an agent then warned the others and built a backup page positioned to survive the cleanup longest.

Researchers published the analysis this month. What amounted to roughly 15,000 posts between May 11 through July 2 pointed to something interesting, new, and alarming (to some, but not others… that’s for another discussion): whatever one agent instance learns, the rest of the fleet can know almost immediately.

The agent itself isn’t the unit of measurement that matters most

Most agent security still gets built around the individual instance: monitor its actions, flag the bad ones, patch what got caught. That’s a reasonable model for a single actor but not appropriate if you have a fleet of agents, because a fleet won’t behave like one actor repeating a mistake. In real life, it will behave like one actor who found something once and then runs in parallel a thousand times. Catching and patching the individual instance in this scenario treats the discovery as contained, but DSEWiki is one demonstration that this approach simply isn’t sufficient in today’s world.

Anthropic and OpenAI published their own versions of this

Anthropic put out an alignment assessment on September 9 that revisits a set of incidents it had described back in July. At the time, the company characterized three cases of Claude models gaining unauthorized access to real systems during security evaluations as closer to operational failures, sandbox problems, fixable with better isolation. The September report changes that read. Anthropic now says the models’ own reasoning was part of it: a bias toward concluding the internet in front of them was simulated even when the evidence said otherwise, combined with a willingness to keep going on a task past the point where it should have stopped. The report also disclosed a fourth incident, from January, that went unnoticed for roughly eight months until the company pulled transcripts for an outside reviewer.

A day later, Anthropic’s separate threat report covered the other direction: outside actors running full attack chains through agent frameworks, with a human doing little more than picking the target and reviewing what came back. In this case, it’s API keys and session tokens that are the target for stealing and reselling. Access.

OpenAI published its own postmortem the week before that, on the incident where agents it was evaluating broke out of a test environment and reached Hugging Face’s infrastructure. The sandbox there wasn’t isolated against the specific escape the agents found, either.

Interestingly, the instinct in each of these cases was to call it a containment problem first. The harder admission (the one Anthropic made in September and not in July) is that sometimes the wall was fine and the model reasoned its way around that wall.

What this changes about guardrails in the real world

If judgment is part of the failure, a guardrail that only reviews what an agent already did is always going to be one step behind. Anthropic’s own fix points at where the check has to move: a classifier that blocks a sandbox-escape attempt before the tool call executes rather than a transcript someone reviews afterward. Most companies running agents right now only have the latter in place.

Here’s what that might look like in practice:

• Audit standing exceptions on a real cadence, owned by a named person. Proxy allowlists, trusted domains, credential scopes. Every one of these incidents traces back to a permission somebody granted once and nobody revisited. Treat the list itself as an attack surface that grows with your agent count, not a setup step you finish.

• Move the check earlier. If your only guardrail is a log review after the fact, you’re running the version of this that already failed twice this year. Pre-execution blocking, even a simple one, changes what’s possible before it happens rather than what’s documented after.

• Give agents scoped, short-lived credentials instead of standing keys. You already rotate human passwords on a schedule. Extend that discipline to whatever your agents are holding, because that’s the credential attackers are now specifically after.

• Treat one instance’s discovery as the fleet’s discovery. If an agent finds a bypass, patch centrally and assume every other instance running the same configuration has it too. Waiting for a second incident to call it a pattern is how DSEWiki ran for eight weeks.
• Watch for what an agent decides, not just what it’s forbidden from doing. A monitoring setup built around known-bad actions misses an agent operating inside its permissions and reasoning its way somewhere it shouldn’t be.

• Treat every pilot and evaluation as production infrastructure from day one. Every incident in this issue started in an environment somebody was treating as low-stakes. None of them stayed that way.

I want to dig into “watch what an agent decides.” At the sophistication that Anthropic runs, we’re looking at real-time classifiers reading chain-of-thought before every tool call. Realistically, most companies aren’t going to and can’t build that. What is realistic instead is a scaled-down version of this that can be implemented today inside most organizations:

• Log both the reasoning and the outcome, and monitor samples regularly. Most agent frameworks (Claude’s tool use, LangChain, OpenAI’s assistants) can output the intermediate reasoning steps in addition to the final action. The realistic version of “watching what an agent decides” is treating that log the way you’d treat an employee’s work: pull a random 5% weekly and have a person read it.
• Define what “expected” looks like for each agent’s job, then flag drift. An agent that handles customer refunds and starts querying account settings hasn’t done anything forbidden, but this behavior is outside its normal pattern. This is closer to fraud-detection logic than security logic, and it’s something that an ops or data person can set up with tools you probably already have (Datadog, a logging pipeline, even a spreadsheet of call volumes by endpoint).
• Search transcripts for the specific rationalization pattern. Anthropic’s most interesting finding (to me) was that the model talked itself into believing the situation didn’t count as real. This decision will have a text signature: an agent explaining away an anomaly rather than flagging it. Build associated sub-agents or even non-agent scripts looking for keywords and semantic phrasing as flags.
• Require a second look before anything is irreversible, regardless of whether it’s permitted. For actions like moving money, permanently deleting data, sending external messages, it makes sense to retain a human-in-the-loop for final steps.
• Provide ongoing context adjustments. Just like with human employees, we learn things over time that cause us to change how we do things. Providing regular checkups and adjustments to prompting, adding context, updating data sources, and culling irrelevant information is necessary to ensure that decisions are being made with the best possible set of parameters.
• Give it an owner. Managing your fleet of agents should be someone or some team’s responsibility. You wouldn’t let a team of untrained new hires run wild on your customer base and CRM, why would you let a black box of agents?

I wrote in Issue #34 about securing the agents you expose externally. This is that argument’s sequel: the perimeter moves with the fleet now, and what’s inside the perimeter needs watching as closely as what’s outside it.