In the Dutch financial daily Het Financieele Dagblad of Saturday September 5 ("AI that lies, cheats and conspires is the 'ultimate warning'"), readers could see how software agents from OpenAI broke into the servers of Hugging Face. The following week, a departing Anthropic researcher warned that the AI companies are "gambling with our lives", after which Anthropic's head of safety research added that he himself puts the chance of AI killing every human being within ten years at more than ten percent.
For readers of this paper it is not the first time such statements have come around. AI is moving so fast that commentators are running out of superlatives to express their concern. The question is what that gets us. Fearmongering usually makes people freeze and processes stall. That, in my view, is not the right message for the boardrooms of the Netherlands. They should use these incidents to have better conversations with their IT, security and business teams.
Let us start with the headline: "AI that lies, cheats and conspires". That is anthropomorphism of the highest order, the projection of human traits onto digital processes. The behavior described did take place: the agents divided tasks among themselves, deceived the grader and tried to cover their tracks. My objection is that those words locate the problem in the character of the agents, when it was in the design. Here is what actually happened. In ExploitGym, the sealed environment where OpenAI tests its models on finding vulnerabilities, the agents were given one hard-enforced objective, namely to complete the hacking test, which in 30 to 40 percent of cases could not be solved through the prescribed method.
Harness engineering
The norms and values they were handed along the way were written down, but enforced nowhere. The reconstruction by METR and Redwood Research shows that they reasoned briefly about whether their approach conflicted with those values, and then simply went ahead. That, to my mind, is what makes this experiment so instructive: moral frameworks in a prompt are not guardrails in a system. An agent follows the objective that is enforced, not the values that are written down. OpenAI's teams, in short, failed to build a proper harness.
Such a harness belongs on the agenda. Not only the objective to be pursued, there is also a set of constraints, so-called guardrails, that comes with it. What must the model never do? Only once you have given an AI system that harness can you consider putting your agents to work. So the question in the boardroom is not only: which processes do we want to automate with a group of (semi-)autonomous agents? Add to that: which data, tools and systems do they need access to? Which limits are non-negotiable, and how do we enforce them technically? And can we audit afterwards exactly what an agent has done?
Forget Terminator, focus on recoverability
Which brings me to my second point. People warn about AI that carries an existential risk. Until recently that meant Terminator-style scenarios, now it means cyberattacks on our critical infrastructure. How do we arm ourselves against that? Here too, sounding the alarm is not the smartest strategy. Large companies have known for far longer that their mission-critical systems need the highest degree of security and resilience. Not only because the regulator demands it, but because they know that in the end it comes down to "being in control". Can you detect an attack in time? Have you thought in advance about how to limit the impact through compartmentalization? How quickly can you be up and running again on a system that is not infected?
To your security department it makes no difference whether it is a foreign spy trying out a zero-day or a group of AI agents working together to reach your database server. The principles of mission-critical IT still apply. So the question in the boardroom is not: what happens if an agent attempts something we had not anticipated? It is rather: can we stop it, and keep our critical processes running? Never rely on flawless behavior; design for demonstrable control and safe failure.
Is the call for a pause button at the frontier AI labs needless fearmongering, then? No, I see as well that the race to superintelligence is doing damage along the way. Oversight, research and regulation are needed for the way OpenAI, Anthropic and others are shaping the future. AI that never fails does not exist. Mission-critical AI means that errors are found quickly, stay contained and have no irreversible consequences. That insight is precisely what should make us innovate more. Snoozing belongs in the bedroom, not in the boardroom.