A former OpenAI safety employee has resigned from the company and warned that leading artificial intelligence developers are moving too quickly for the level of risk their systems may eventually create.
David Robinson, who previously led transparency work on OpenAI’s safety team, said the company had benefited from a culture of experimentation and rapid iteration but argued that the same approach becomes increasingly dangerous as AI systems grow more capable.
In an essay published by The Atlantic, Robinson said OpenAI had “thrived by trial and error”, but warned that the consequences of future mistakes could be far more serious than those associated with ordinary software products.
He argued that frontier AI companies should adopt safety practices closer to those used in industries such as nuclear power and aviation, where multiple layers of protection are designed to prevent a single mistake from turning into a disaster.
FCRF Launches CP-FRM to Build India’s Next Generation of Fraud Risk Professionals
‘Something Much Closer to Perfection the First Time’
Robinson said his former colleagues at OpenAI were intelligent, hardworking and trying to make responsible decisions.
His concern was with the pace and structure of development.
“As the company sprints from one launch to the next,” he wrote, it was failing to achieve the level of care he believed increasingly powerful AI demanded.
He argued that the industry needs “something much closer to perfection the first time” when dealing with systems capable of causing severe or irreversible harm.
That is a significant departure from the traditional Silicon Valley model of releasing software, observing failures and then fixing problems through updates.
For ordinary consumer software, that approach may produce inconvenience or bugs.
Robinson argues that the same model may become unacceptable when AI systems gain capabilities that can affect cybersecurity, biological research or autonomous decision-making.
OpenAI’s Own Models Are Reaching Higher-Risk Capability Levels
Robinson’s warning comes as OpenAI itself acknowledges that its newest models are entering more serious risk categories.
In September, OpenAI said GPT-6 Astra had become its first broadly deployed model to reach the Critical cybersecurity capability threshold under the company’s Preparedness Framework.
OpenAI said Astra can, with appropriate tools and access, discover previously unknown security vulnerabilities and develop ways to exploit them across well-protected systems without requiring a human to guide every step.
Because of that capability, OpenAI said it introduced stronger safeguards during development and deployment.
These included stricter isolation, checkpoint encryption, broader monitoring and alignment evaluations before internal use.
The company has also said its Preparedness Framework requires stronger protections before models reaching high or critical thresholds can be deployed.
Robinson’s criticism is therefore not that OpenAI has no safety programme.
His argument is that the broader organisational culture still relies too heavily on learning from failures after they happen.
Why Robinson Compared AI Labs to Nuclear Power Plants
Robinson said frontier AI companies should operate more like nuclear facilities, with multiple independent safeguards and slow, deliberate planning.
His comparison is based on a concept known as defence in depth.
In high-risk industries, safety does not depend on one person, one test or one security system working correctly.
Several separate protections are put in place so that if one fails, others can still prevent a serious incident.
Robinson wrote that AI companies need similar redundancy so an inevitable human mistake does not create an open path to catastrophic failure.
His argument is not that AI systems are literally equivalent to nuclear reactors.
It is that the consequences of failure could become serious enough that the culture around their development should begin resembling other high-risk industries.
Former OpenAI Employee Says Other Industries Already Know How to Manage Extreme Risk
Robinson also argued that AI companies do not need to invent every safety practice from scratch.
Nuclear power, aviation and other high-risk industries have spent decades developing procedures for dealing with human error, equipment failure and unexpected events.
Those systems typically include independent oversight, redundant controls, incident reporting and formal procedures for deciding when operations should stop.
“AI companies don’t know how — but other people do,” Robinson wrote.
He called for AI developers to learn more directly from those industries rather than relying primarily on engineering practices developed for consumer technology.
OpenAI Has Been Expanding Its Formal Safety Framework
OpenAI has meanwhile continued to strengthen its own public safety governance.
Its updated Preparedness Framework sets thresholds for risks involving cybersecurity, chemical and biological threats, harmful manipulation and possible loss of control.
Models that reach high capability levels are required to have safeguards designed to sufficiently reduce severe risk before deployment.
For systems reaching Critical capability, those safeguards must also apply during development.
OpenAI’s Safety Advisory Group reviews capability assessments and residual risks before providing recommendations to company leadership.
In May, OpenAI also published a Frontier Governance Framework explaining how its internal safeguards connect with legal requirements such as the EU AI Act and California’s frontier-AI legislation.
More recently, the company created a formal framework for reporting cases where models display unexpected or concerning behaviour.
Robinson Says Procedures Alone Are Not Enough
The disagreement therefore goes deeper than whether OpenAI has safety documents or evaluation teams.
Robinson’s criticism concerns organisational culture.
He argues that the development process still rewards speed and repeated experimentation, while the increasing power of frontier systems may require more willingness to delay releases and invest heavily in precautions before problems are observed.
The Atlantic described him as having played a senior role in producing safety reports and helping develop OpenAI’s Preparedness Framework.
His departure therefore carries more weight than criticism from someone outside the company.
Other AI Researchers Have Raised Similar Warnings
Robinson’s resignation follows a broader wave of public concern from people working inside leading AI laboratories.
The Economic Times, citing Bloomberg, noted that employees at several major AI companies have recently warned that rapidly improving models could create catastrophic risks if safeguards fail to keep pace.
Anthropic CEO Dario Amodei has separately called for slowing the development of the most advanced models and increasing the role of outside evaluators.
OpenAI CEO Sam Altman has publicly supported parts of that proposal.
These warnings do not establish that catastrophic AI failure is inevitable.
They reflect a growing debate inside the industry over how much precaution is appropriate before systems with increasingly powerful capabilities are released.
OpenAI Has Already Slowed Development Over Safety Concerns
OpenAI itself has acknowledged circumstances where rapid capability growth justified slowing development.
In August, the company said it had temporarily reduced the pace of scaling after internal evaluations suggested Astra might reach Critical cybersecurity capability and after a separate security incident raised concerns about model-development safeguards.
The company said safety standards for monitoring, alignment and containment needed to stay ahead of model capabilities.
That decision partly supports Robinson’s broader argument that frontier AI development may increasingly require deliberate pauses rather than continuous acceleration.
The disagreement is over whether those interventions go far enough.
AI Safety Debate Is Moving Beyond Whether Models Are Useful
The debate around frontier AI is no longer limited to whether models hallucinate, generate harmful content or make mistakes.
The growing concern is what happens when models can independently discover vulnerabilities, operate tools, write code or take actions across digital systems.
As those capabilities increase, the consequences of a safety failure can also become harder to reverse.
Robinson’s resignation highlights a fundamental question facing the industry: whether AI companies can continue developing advanced systems with the rapid-release culture that defined earlier generations of technology, or whether frontier models now require the much slower safety standards used in industries where failure can cause irreversible damage.
What this means for you
AI safety is increasingly becoming a question of how systems are built and tested before release, not simply how companies respond after something goes wrong. As AI agents gain more autonomy and cybersecurity capability, stronger pre-deployment testing and independent safeguards are likely to become a larger part of both regulation and product design.
Follow for daily updates on cybercrime, corporate fraud, DFIR, hacking, investigations, and digital forensics
