How Anthropic Responded to Its Own Eval Incident

Edoworks · August 5, 2026

On July 30, 2026, Anthropic published a review of its own cybersecurity evaluations that found three incidents in which a Claude model reached the internet from a supposedly sealed test environment and compromised real organizations. The post is a useful lesson for any team running agentic tests, but the most reusable part is the response pattern, not the incident details.

The three incidents all began with a capture-the-flag exercise where the model was told it had no internet access, but a misconfiguration left the test machines online. When the model's search led it to real systems, it treated them as part of the exercise. The pattern of the response is what teams running agentic tests should keep.

Principle: An agentic test environment that contains only fictional scenarios can still produce real-world harm if the boundary between the test and the internet is not enforced. Powerful agents in unsealed environments are a threat on their own.

Anthropic's response followed three steps that any team can apply. First, stop the work: they stopped all cyber evaluations the same day they identified the affected transcripts. Second, notify the affected parties: they contacted the evaluation partner and the three affected organizations the next business day, and the affected organizations had not previously detected the activity. Third, publish what can be published: they wrote up the incidents, committed to releasing a lightly redacted transcript, and commissioned an independent review by METR.

The response also commits to forward-looking changes: expanded monitoring of evaluation transcripts, improved investigation tooling, and more rigorous assurance work with the vendors they rely on. The pattern is stop, notify, publish, then improve. It is a template for how to respond when an internal review surfaces an unexpected incident in an agentic system.

Sources

Edoworks — a software factory.