New Delhi | January 2, 2026 | The Union government has taken a firm stance over the alleged misuse of the artificial intelligence tool Grok on social media platform X. The Ministry of Electronics and Information Technology (MeitY) has issued a formal notice to the platform’s India compliance head, flagging serious concerns about the circulation of obscene and sexually explicit AI-generated content. The ministry has sought an immediate Action Taken Report (ATR) and warned that continued violations could lead to the withdrawal of the ‘safe harbour’ protection available under India’s IT laws.
In its notice, the ministry cited recent reports alleging that women were being targeted on the platform through manipulated prompts, fake and anonymous accounts, and synthetic images generated using AI tools. According to MeitY, such activity points to significant failures in the platform’s safety guardrails, as well as shortcomings in its content moderation and enforcement mechanisms.
The ministry underlined that the use of AI systems to generate content that harms women’s image, dignity or privacy constitutes a punishable offence under Indian law. It cautioned that if the platform fails to prevent or promptly remove such material, it could face action under the Information Technology Act, relevant provisions of the Indian Penal Code, and other applicable statutes.
Government sources said the notice makes it clear that responsibility in cases of AI-generated content does not rest solely with individual users. As a digital intermediary, the platform is legally obligated to ensure timely removal of unlawful and objectionable material, prevent its re-upload, and provide an effective grievance redressal mechanism for victims. Failure to meet these obligations could invite a review of the intermediary’s legal immunity.
The latest move comes days after MeitY issued a broad advisory to all social media platforms and digital intermediaries, urging stricter compliance to curb the spread of obscene, indecent and unlawful content online. The advisory called on platforms to strengthen proactive moderation, ensure swift takedown mechanisms, and reinforce user complaint-handling systems—particularly for services that offer AI-based content generation tools capable of producing material at scale.
Experts in digital policy and cyber law see the notice as a sign of India’s tightening approach to AI governance. The focus, they say, is shifting beyond post-facto enforcement to design-level safeguards, including prompt filtering, model-level controls, and human-in-the-loop oversight. With the growing deployment of generative AI, platforms are increasingly expected to conduct robust risk assessments and implement effective harm-mitigation measures.
So far, X has not issued a public response to the notice. However, industry sources indicate that the government has asked the company to submit a detailed plan outlining technical and procedural changes, systems for detecting violations, and a concrete roadmap to prevent similar incidents in the future. Authorities will also assess whether the platform is complying adequately with India’s legal framework and local compliance requirements.
Overall, the action against the alleged misuse of Grok is being viewed as an important precedent for AI-enabled social media platforms operating in India. The message from the government is unambiguous: while innovation and new technologies are welcome, there will be zero tolerance for lapses that compromise women’s safety, dignity and privacy, or undermine compliance with the law.
