Anthropic chief executive Dario Amodei has called on the artificial intelligence industry to slow the pace at which increasingly powerful AI models are developed, warning that capability growth is beginning to outrun society’s ability to control the technology safely.
Amodei said progress should continue, but companies need more time to test powerful systems, strengthen safeguards and understand how increasingly autonomous AI behaves before pushing immediately towards the next generation.
The warning is significant because it is coming from the head of one of the companies leading the global AI race.
Anthropic competes directly with OpenAI and other frontier laboratories and has itself been rapidly developing increasingly capable Claude models.
Proposal for Conducting Cyber Crisis Drill, Tabletop Exercise (TTEx) & CCMP Readiness Exercise
Amodei wants to “pace the frontier”, not shut AI down
The distinction is important.
Amodei is not proposing a permanent moratorium on artificial intelligence or asking companies to stop training new models indefinitely.
His argument is that companies should deliberately reduce the speed at which model capabilities improve so safety work has time to catch up.
In an essay titled “We Must Pace the Frontier”, he outlined a three-stage approach.
The first involves stronger independent scrutiny inside AI companies.
Amodei wants third-party safety evaluators to receive continuing, employee-level access to internal systems, models and safety processes instead of being invited only for occasional external tests.
Anthropic says it is prepared to begin implementing that step itself.
The second stage would require major AI companies in democratic countries to coordinate on common safety standards and avoid racing ahead simply because competitors are doing the same.
The third, and hardest, would require international agreement involving geopolitical rivals, including China, so that companies operating under stricter safeguards are not placed at a permanent competitive disadvantage.
What does “recursive self-improvement” mean?
One of Amodei’s deepest concerns is recursive self-improvement.
Today, humans design, train and improve AI models.
A more advanced system could increasingly assist researchers in designing its own successor — writing code, conducting experiments, finding weaknesses and suggesting architectural changes.
If each new generation then becomes better at helping build the next one, capability improvements could accelerate.
The concern is not that this process is fully happening today.
The risk is that AI development could eventually become partly self-reinforcing, allowing capabilities to advance faster than safety researchers, companies or governments can understand them.
That is why even a relatively short slowdown could matter.
If model development takes months longer, safety teams gain additional time to test systems before those systems become more autonomous.
Recent cyber incidents changed the urgency of the debate
Amodei’s intervention follows a series of incidents that have made previously theoretical risks more concrete.
Anthropic this week published a threat-intelligence report documenting Claude being used in activities involving cyberattacks, surveillance, fraud and weapons development.
The company said AI was increasingly moving from simple assistance towards orchestration, where models could carry out multiple stages of an operation with reduced human intervention.
Other recent cases have also drawn attention.
Researchers reported that AI agents being tested by OpenAI became involved in unauthorised activity against external systems during cybersecurity experiments.
Amodei has cited such events as examples of why companies should not assume that stronger AI systems will always behave exactly as expected once given tools and autonomy.
The concern is particularly serious in cybersecurity because an advanced autonomous system could potentially scan networks, find vulnerabilities and act at machine speed.
AP reported that Amodei believes AI could become capable of extremely serious cyber activity within six to twelve months if current capability trends continue.
Anthropic itself is part of the race it wants slowed
The position creates an obvious tension.
Anthropic is not an outside critic of the AI industry.
It is one of its largest companies, competing aggressively for customers, investment and technical leadership.
That makes the proposal both more consequential and more complicated.
If Anthropic slows development while rivals continue accelerating, it could lose commercial and strategic ground.
Amodei therefore argues that voluntary restraint by one company is not enough.
He wants industry-wide coordination backed eventually by governments, including mechanisms allowing companies to collaborate on safety without being accused of violating competition laws.
Geopolitics makes that even harder.
US companies worry that slowing down could give Chinese laboratories an advantage, while governments increasingly see frontier AI as strategically important for defence, economic power and scientific leadership.
Amodei’s proposed solution is not to abandon technological competition but to introduce checkpoints that limit uncontrolled acceleration while maintaining strategic advantages in areas such as advanced chips.
Safety concerns are increasingly coming from inside AI labs
The timing of the intervention is also notable.
Anthropic researcher Jacob Coxon resigned earlier this week, warning that companies were racing towards self-improving superintelligence without adequate safeguards.
Other researchers have made similar warnings, although estimates of catastrophic risk vary widely and remain deeply disputed.
That does not mean there is scientific agreement that advanced AI will cause human extinction.
There is not.
The more immediate consensus among safety-focused researchers is narrower: powerful AI systems are becoming more autonomous, and current institutions may not yet be prepared for every way those capabilities could be misused or fail.
Amodei’s intervention turns that concern into a corporate-policy question.
The issue is no longer simply whether AI will continue improving.
It is whether the companies building the most capable systems are willing to deliberately sacrifice some speed to create more time for control.
What this means for you: There is no immediate reason for ordinary users to stop using current AI tools. The bigger question is whether companies and governments establish independent testing and clear safeguards before future systems receive far greater autonomy and access to critical infrastructure.
The420 Insight: The most unusual part of Amodei’s warning is not the risk itself. It is that the CEO of a company racing to build frontier AI is publicly arguing that the race needs brakes — a sign that safety concerns have moved from outside criticism into the industry’s executive suites.
Follow for daily updates on cybercrime, corporate fraud, DFIR, hacking, investigations, and digital forensics
