The government on Friday sent a letter to X Corp., demanding immediate action to prevent its Grok AI chatbot from generating obscene and sexually explicit content, and giving the company 72 hours to submit a detailed compliance report.
The Ministry of Electronics and Information Technology (MeitY) directed X to undertake a comprehensive technical review of Grok and remove all violating content, warning that failure to comply could result in the loss of legal protections under the Information Technology (IT) Act and potential criminal prosecution.
The MeitY notice cited multiple legal violations, including sections of the IT Act dealing with obscene content, the Bharatiya Nyaya Sanhita (BNS), the Indecent Representation of Women (Prohibition) Act, 1986, and the Protection of Children from Sexual Offences (Pocso) Act, 2012.
“It has especially been observed that the service namely ‘Grok AI’ developed by you and integrated and made available on the X platform, is being misused by users to create fake accounts to host, generate, publish or share obscene images or videos of women in a derogatory or vulgar manner,” the ministry said in a letter to X’s chief compliance officer for India operations.
The action came after Shiv Sena (UBT) MP Priyanka Chaturvedi demanded intervention, highlighting how men are using fake accounts to post women’s photos on X and using Grok prompts to minimise their clothing and sexualise them.
In a letter to electronics and IT minister Ashwini Vaishnaw, Chaturvedi said the trend represented an “unacceptable and gross misuse of an AI function”.
“This is [a] breach of women’s right to privacy as well as unauthorised use of their pictures, which is not just unethical but also criminal,” the Rajya Sabha MP said. “Our country cannot be a bystander to women’s dignity being violated publicly and digitally with zero consequences under the garb of creativity and innovation.”
Vaishnaw, speaking at an event in Delhi, spoke on the issue. “Today social media is such a big influence in our society…they have to take responsibility for the content. The [parliamentary] standing committee [on communications and information technology] has very strongly recommended that there should be strong laws for making social media accountable.”
Separately, the chatbot also acknowledged creating sexualised images of minors in recent days, violating its own acceptable use policy, Bloomberg reported.
Grok created images of minors in minimal clothing in response to user prompts over the past few days, the chatbot said in a series of posts on X this week. “We’ve identified lapses in safeguards and are urgently fixing them,” Grok posted on Friday, adding that child sexual abuse material is “illegal and prohibited”, according to Bloomberg. The offending images were taken down.
X and the Union government have over the past year been locked in a legal battle over the Sahyog portal that channels content moderation order — a mechanism that the American company called a “censorship portal”.
According to people aware of the matter, X and the ministry hold meetings on regular basis as part of the wider industry-government engagement. Two days earlier, the ministry and X decided to meet over political and some religious content churned out by Grok in recent days.
A meeting has been scheduled for Saturday but the company will likely seek it to be postponed, this person added.
MeitY secretary S Krishnan confirmed to HT that the ministry acted after note of several complaints, including by Rajya Sabha member Chaturvedi.
The notice sent to the company, signed by joint secretary Ajit Kumar, said the misuse was “not limited to creation of fake accounts but also targets women who host or publish their images or videos, through prompts, image manipulation and synthetic outputs”.
The ministry specifically directed X to immediately undertake a comprehensive review of Grok’s “prompt-processing, output-generation (responses generated using Large Language Models), image-handling and safety guardrails” to ensure the application does not generate, promote or facilitate content containing “nudity, sexualisation, sexually explicit or otherwise unlawful content”.
X must also enforce its user terms of service with “strong deterrent measures such as suspension, termination and other enforcement actions against violating users and accounts”, and remove all violating content without delay, the notice said.
The company has been ordered to submit a detailed Action Taken Report within 72 hours covering specific technical and organisational measures adopted for Grok, the role of the chief compliance officer, actions taken against offending content and accounts, and mechanisms to ensure mandatory reporting under criminal law.
The notice warned that “non-compliance with the above requirements shall be viewed seriously and may result in strict legal consequences against your platform, its responsible officers and the users on the platform who violate the law”.
Under the IT Act, intermediaries like X enjoy safe harbour protections from liability for user-generated content, but these protections are conditional upon strict observance of due diligence obligations. The ministry’s notice makes clear that failure to comply could result in the loss of these exemptions.
The controversies highlight broader challenges facing AI image generation tools. The Internet Watch Foundation, a non-profit that identifies child sexual abuse material online, reported a 400% increase in AI-generated child abuse imagery in the first six months of 2025, Bloomberg reported.
xAI has positioned Grok as more permissive than other mainstream AI models and last summer introduced a feature called “Spicy Mode” that permits partial adult nudity and sexually suggestive content. The service prohibits pornography involving real people’s likenesses and sexual content involving minors, which is illegal to create or distribute.
The MeitY notice was also sent to the secretaries of the Ministry of Home Affairs, Ministry of Women and Child Development, Ministry of Information and Broadcasting, National Commission for Women, National Commission for Protection of Child Rights, and chief secretaries of all state governments and union territories.
