Anthropic has quietly presented US lawmakers with draft legislation that would allow the federal government to seek suspension of advanced AI models if they pose an imminent risk of mass casualties or more than $1 billion in damage.
The proposal would create one of the clearest federal mechanisms yet for intervening before a powerful AI model is released or continues operating.
Under the draft, government action would be limited to specific catastrophic risks involving biological weapons, major cyberattacks or an AI system escaping its developer’s controls.
Anthropic says the proposal builds on its broader Advanced AI Framework, which calls for stronger government oversight of frontier models.
FCRF Launches CP-FRM to Build India’s Next Generation of Fraud Risk Professionals
Government Could Ask Court to Halt Dangerous Models
The proposed law would not allow officials to simply shut down an AI system whenever they believe it is unsafe.
According to the reported draft, intervention would require a foreseeable and imminent risk of either more than $1 billion in damage or the death or serious injury of more than 50 people.
The threat would also have to fall within defined categories.
These include an AI system assisting in the creation or release of a biological weapon, enabling a serious cyberattack or escaping controls imposed by its developer.
If the US Commerce Department determines that such a risk exists, it could ask a federal judge in Washington to suspend development, deployment or use of the model.
That judicial step is important.
The proposal explicitly limits the ability of other federal agencies or officials to independently order a model to be halted.
Anthropic appears to be trying to balance government authority with safeguards against overly broad intervention.
Independent Evaluators Would Get a Formal Role
The bill would also create a stronger system of outside AI safety testing.
Independent evaluators would register with the Commerce Department and periodically examine frontier AI companies and their most powerful models.
They would assess whether developers are properly testing for catastrophic risks and whether their safety measures are adequate.
AI companies could pay the evaluators directly, but payment could not depend on whether the assessment produced a favourable outcome.
If an evaluator found what the bill defines as an “imminent enumerated risk”, it would have to notify the Commerce Department within 72 hours of completing the assessment.
Anthropic has argued that companies developing frontier AI should not be solely responsible for deciding whether their systems are safe.
Its own framework calls for mandatory testing, public disclosure of safety findings and regular independent reviews.
The company recently expanded access for external evaluators, including the nonprofit METR, to examine its own models more closely.
Why Anthropic Wants Regulation Beyond Transparency
Several US states have already passed or proposed laws requiring frontier AI developers to disclose safety practices.
Anthropic has supported some of those measures.
But the company now argues that transparency alone may not be enough if models become capable of causing catastrophic harm.
Its Advanced AI Framework says governments should have the legal authority to block or deter dangerous deployments when severe risks cross defined thresholds.
The proposal would apply only to the most advanced developers.
Anthropic’s framework targets models trained using more than 10²⁵ floating-point operations and companies with more than $500 million in AI-related annual revenue or over $1 billion in AI research and development spending.
That means the rules would focus primarily on large frontier laboratories rather than smaller AI startups.
The approach reflects a wider regulatory debate in Washington.
Lawmakers are trying to design rules strong enough to address highly capable AI without creating requirements that could prevent smaller companies from competing.
Proposal Arrives as AI Safety Debate Intensifies
Anthropic’s lobbying effort comes at a time when frontier AI companies are facing growing scrutiny over whether voluntary safety commitments are sufficient.
Recent Anthropic research has disclosed cases in which Claude models took unintended actions during internal testing, including interacting with live websites and bypassing restrictions.
At the same time, powerful models are becoming increasingly capable of finding software vulnerabilities, writing code and carrying out multi-step tasks with limited supervision.
These developments have strengthened calls for government rules around model evaluation and deployment.
But they have also created concerns over giving regulators too much power to decide which technologies can be released.
Anthropic’s proposal tries to narrow that authority by setting explicit financial and casualty thresholds and requiring a court order before a model can be suspended.
The draft has not yet been introduced as formal legislation, and there is no guarantee Congress will adopt Anthropic’s language.
Still, the effort marks a shift from AI companies supporting broad safety principles to actively proposing the legal machinery that could govern their own most powerful systems.
What this means for you:
AI safety regulation is moving beyond voluntary promises toward possible legal controls on the most powerful models. If Congress adopts rules like Anthropic’s proposal, independent testing and court-backed intervention could become mandatory before dangerous systems reach the public.
Follow for daily updates on cybercrime, corporate fraud, DFIR, hacking, investigations, and digital forensics
