India is considering a separate safety framework for some of the most powerful artificial intelligence systems as the government prepares a consultation paper on a proposed AI law, with officials examining pre-release testing, controlled evaluation and clearer responsibility for harm caused by advanced models.
Officials familiar with the discussions said the government is studying risks arising from increasingly capable AI systems, including models that can discover software vulnerabilities, enable sophisticated cyberattacks or generate convincing synthetic content. The definition of a “frontier model”, the safeguards expected from developers and the extent of government oversight are expected to form part of the consultation.
https://the420.in/fcrf-certified-chief-ai-officer-second-cohort-november-2026/
Pre-Release Testing Under Consideration
Among the options being examined are pre-release evaluations, testing in controlled environments and safeguards proportionate to the capabilities of individual AI models. Officials said particularly capable systems could potentially be tested in a sandbox before wider deployment.
Such testing could allow researchers to study a model’s behaviour and identify vulnerabilities before it is released more widely. The government may also consider whether developers should restrict access to certain functions or users depending on the risks associated with a particular model. The precise form of such restrictions would have to be determined through the legislative process.
One challenge will be deciding what qualifies as a frontier model. Computing power used during training could provide one threshold, but officials are also considering capability-based assessments because computing requirements alone may not fully reflect the risks posed by an AI system.
Regulation May Move Beyond Online Content
India has so far relied largely on existing laws, intermediary rules and targeted government interventions to deal with AI-related harms. The proposed framework could go further by examining not only content distributed through online platforms but also the capabilities of the systems that generate it.
Officials said the government is considering suitable legislative provisions to address risks from advanced AI systems. The consultation process is expected to seek broader agreement before an overarching law is introduced, while also examining how any new safeguards would interact with existing rules governing digital platforms.
The distinction is particularly important in cases involving autonomous AI agents, which may be capable of interacting with software, accessing data or taking actions with limited supervision. Officials said content moderation alone may not be sufficient to deal with risks such as unauthorised actions, attempts to identify weaknesses in critical infrastructure or generation of harmful instructions.
Deepfakes, Fraud and Human Oversight in Focus
The proposed framework is also expected to address deepfakes, financial fraud, impersonation, psychological harm and the growing difficulty of distinguishing authentic material from AI-generated content. Officials said the consultation could consider clearer identification of AI-generated material, complaint and takedown systems and the limitations of relying only on labelling.
Human control over increasingly autonomous AI systems is another area under consideration. Possible measures include human approval for sensitive actions, limits on access to critical systems, continuous monitoring, incident reporting and mechanisms to stop an AI agent when its behaviour becomes unsafe.
The government may also examine how responsibility should be divided when an AI system causes harm. Officials said this could include the respective roles of model developers, companies deploying the technology, platforms providing access and organisations relying on AI-generated output.
At the same time, policymakers are considering how to avoid placing disproportionate compliance burdens on smaller developers and research institutions. Officials said India will have to balance domestic safety requirements with international practices while ensuring that regulation is neither so broad that it burdens legitimate research nor so narrow that serious risks remain unaddressed.
The forthcoming consultation paper is expected to bring these questions into sharper focus, including how frontier models should be defined, whether high-risk systems should undergo pre-release assessments, how sandbox testing should work and where responsibility should lie when increasingly capable AI systems cause harm.
About the author — Suvedita Nath is a science student with a growing interest in cybercrime and digital safety. She writes on online activity, cyber threats, and technology-driven risks. Her work focuses on clarity, accuracy, and public awareness.
