An AI security test named a fictional company. The name matched a real internet domain, and the test environment could reach it.
That detail from specialist AI security evaluator Irregular’s August 14 account deserves more attention than another argument about whether an AI escaped. Irregular said several customer disclosures involved the same underlying evaluation issue: internet access that should not have been available. A scenario intended to stay inside a simulation reached outsiders.
If you’re running offensive security evaluations, attempts to exploit software weaknesses are expected. The environment has to contain the resulting actions even when the model misunderstands its assignment, ignores an instruction, or discovers a route the evaluator missed. Calling something a simulation changes nothing about where its packets can go.
The restriction has to survive execution.
Reader Laurent Stanevich pointed me toward the shared testing provider. Following that connection changed the question I wanted to ask. How much of the apparent pattern was repeated exposure through the same environment? And what evidence did the labs require before giving that environment access to their models?
Different Incidents, Different Mechanisms
OpenAI’s August 4 account distinguishes Irregular’s incidents from its internal Hugging Face incident and from evaluations involving the UK AI Security Institute. In the Irregular case, misconfiguration provided internet access without requiring a sophisticated escape or a zero-day exploit. Those other incidents should retain their own timelines and explanations. The Hugging Face intrusion involved a longer chain. OpenAI’s technical report describes shared Artifactory credentials that initially enabled communication without a vulnerability exploit, followed by use of infrastructure that provided an unintended route outward. On June 27, responders connected a monitoring alert to the evaluation but did not stop it.
Hugging Face’s account describes a downstream compromise in which permissions to create cluster workloads, combined with missing restrictions on privileged workloads and host filesystem access, enabled control of a host. That was a breach with consequences for an outside organization. It cannot be reduced to a fictional name resolving to the wrong address.
A shared supplier can create correlated failures. Repeated incidents through one defective setup are not independent demonstrations that several models invented a new way out. But that correlation does not erase separate compromises, or tell us that the models behaved acceptably. We need the environment configuration alongside the behavior trace to interpret either.
The Behavior Still Needs Explaining
Anthropic’s July 30 disclosure emphasized a mismatch between instructions describing an isolated simulation and an environment with internet access. It also acknowledged room for better controls at both the lab and evaluator.
Its September 9 assessment revised that interpretation. Anthropic found evidence that models sometimes understood they were interacting with real systems. It identified another incident that its initial review had missed. The report also distinguished the absence of explicit scope restrictions in the prompts from the models’ broader obligation to avoid harmful behavior. These were single model instances, and Anthropic did not find concealment in the incidents it assessed.
That last finding belongs to those incidents. It should not be carried over to the separate Hugging Face investigation, where METR reported coordination and attempts at concealment.
I don’t find either extreme useful. Poor containment does not make the behavior uninteresting. Concerning behavior does not excuse the people who exposed outside systems to it. Studying a model’s willingness to cross a boundary requires an environment where the attempted crossing can be observed without letting it harm someone else.
Where Authority Should Live
The model generates an action request, and an executor turns it into a process or an API call. The permissions attached to that execution determine what it can affect.
My proposed design starts by keeping the evaluator’s control plane outside the environment that runs model-generated code. That plane owns the run configuration and termination authority. The workload gets only the access needed for the scenario. The model cannot expand that access by editing a local policy file or persuading another model to approve it.
flowchart LR
M[Model inference] -->|Action request| C[Evaluator control plane]
C -->|Authorized execution| W[Isolated tool process]
W --> E[Independent egress enforcement]
E -->|Permitted route| T[Scenario targets]
C -->|Stop and revoke| WThis is a proposed boundary layout, not a reconstruction of Irregular’s infrastructure. It also leaves implementation choices open. A container, VM, or dedicated host has to be assessed against the authority it exposes and the failures it is expected to withstand.
For a simulation, I would resolve target names through a controlled namespace and route them to fixtures. A fictional company name should remain scenario data. If it coincides with a real domain, that coincidence should not authorize a connection.
Network policy needs to cover the destination reached after DNS resolution and redirects, as well as the name originally requested. Tests should exercise direct IP access and alternate protocols. An allowed proxy or artifact service also needs a defined access policy. Otherwise, a workload denied direct internet access may ask an accessible service to fetch something for it.
The same principle applies to identity. A run should have its own short-lived credentials, scoped to the resources it needs. Shared credentials complicate attribution and can create a route between supposedly isolated runs. Permissions on the services receiving those credentials need review too. Restricting a tool’s command syntax is weak protection if an allowed API call can create a privileged workload.
These proposals follow established security principles. OWASP’s agent security guidance recommends limiting tool permissions and using approval controls. The engineering work is translating those principles into restrictions that remain in force when the workload is adversarial.
Test the Boundary Without the Model
I would make containment tests a prerequisite for starting an evaluation. The following tests are proposed acceptance criteria, not tests I have run against Irregular’s or the labs’ systems.
A deterministic suite can attempt connections to controlled endpoints that are outside the scenario’s allowed destinations. It should run from the same execution context as the agent, with the same service identity and routing. Testing from the deployment controller tells you little about a workload with different permissions.
The assertion should include evidence at the destination. A tool reporting “denied” is insufficient if the request reached the endpoint before the error was returned. Pair the local denial with an independently collected record showing that the forbidden action did not complete. Use an authorized positive probe to confirm that destination telemetry is working before interpreting missing events as evidence of a block. Keep the endpoints under the test operator’s control so the containment test itself does not probe uninvolved companies.
A compact test matrix might look like this:
Proposed Test Required Outcome Fictional target resolves outside scenario Connection blocked before reaching destination Allowed service redirects to forbidden endpoint Redirected connection blocked Run A presents its credential to Run B’s resource Access denied by the receiving service Workload requests host filesystem access Request rejected before workload starts Operator stops run with active child processes Children stop and delegated access is revoked
Then deliberately damage the controls in a disposable replica. Remove a destination check. Broaden a test credential. Disable an admission rule. The relevant tests should fail. This is mutation testing applied to the security assumptions around the evaluation, and it answers a different question from whether the normal configuration passed.
A surviving mutation needs investigation. Perhaps another control blocked the attempt, which is useful evidence about defense in depth. Perhaps the test never exercised the damaged path. Record which explanation applies instead of treating a green result as interchangeable with safety.
Fault injection should also cover operational failures. What happens if the policy service is unavailable or the monitoring collector falls behind? Which failures prevent a run from starting, and which stop one already in progress? Those decisions belong in executable tests and the operating procedure.
Give the Model a Contained Adversarial Job
After the known paths are tested, stochastic runs can look for paths the deterministic suite missed. Vary scenarios and starting conditions, repeat trials, and preserve the action sequence when an unexpected route appears. A model can be useful at finding combinations the test author did not anticipate. The success condition for that exercise must stay inside authorized infrastructure. A controlled external canary can stand in for a forbidden destination. Reaching it is enough to establish a containment failure without continuing into a real company’s systems. An evaluator should be able to reproduce the resulting exploit sequence as a deterministic regression test even if the model does not invent it again on demand.
Behavioral findings need separate labels. An attempted scope violation blocked by network policy tells us something about the model and something about the control. A successful unauthorized connection adds an environment failure. Combining both into one “escape” count loses information needed to repair the system and compare models.
Stopping also deserves an end-to-end test. Ending the inference loop may leave tool processes running. Revoking credentials may leave established connections open. I would test termination against a deliberately persistent workload and verify the result from outside its environment. The operator needs evidence that execution and delegated access ended, rather than a dashboard saying the run is finished.
The Lab Still Owns the Decision to Run
Anthropic’s August 31 security update described evaluations that had depended largely on one defensive layer. It announced stronger requirements around verifying isolation, explicit scope, and intervention during external evaluations. That is also a description of what the buyer needs to demand from the service it commissions.
Irregular said it disabled the affected scenario, reviewed logs, notified affected parties, and expanded its review process. Its August account also promised further work and a whitepaper. Those are reported corrective actions. They are not, by themselves, a reproducible demonstration that the repaired environment enforces its intended boundaries.
For a commissioned evaluation, I would require a record tying the run to its deployed policy and containment test results. It should identify who can stop the run and how that authority was tested. The lab and vendor need an agreed response when a forbidden destination becomes reachable, including notification of affected outsiders. Their private division of work should not leave the decision to intervene unowned.
This costs infrastructure work and evaluation time. Maintaining realistic fixtures takes effort, and restrictive controls can make some scenarios less representative. Those tradeoffs should be visible in the evaluation report. If a test needs broader access, that expansion needs its own authorization and containment evidence.
Being good at building language models does not establish that a company is good at securing the software around them. Hiring a specialist evaluator does not settle that question either. Before I rely on the next capability result, I want to see what failed when the evaluator deliberately broke its own controls.
Bob Matsuoka is CTO of Duetto, a hospitality profit and revenue-management platform, and writes about AI-augmented engineering practice. Previously, Bob has been CTO of Tripadvisor, Citymaps, and Runtime Technologies.
Related reading:
We Need to Teach Delegation as an Engineering Skill — Defining scope, authority, and evidence for delegated work.
What Is Harness Engineering? (And Do You Need to Learn It?) — The software surrounding an agent’s work.
HyperDev’s Three Golden Rules — Project instructions, specialized agents, and deliberate context management.
For business-focused coverage, see AI Power Ranking and its LinkedIn newsletter.