California Attorney General Orders xAI to Halt Illegal Grok Deepfake Imagery

California Attorney General Robert Bonta has issued a cease-and-desist letter to Elon Musk’s artificial intelligence company xAI, demanding an immediate stop to the creation and distribution of AI-generated nonconsensual sexualized imagery produced by its Grok chatbot. The move escalates legal and regulatory pressure on Musk’s expanding social media and AI ecosystem, particularly as concerns mount globally over the misuse of generative artificial intelligence.

In a statement released Friday, Bonta described the situation as alarming, citing an “avalanche of reports” involving AI-generated images that allegedly depict women and children in sexualized and degrading scenarios. According to the California Attorney General’s Office, such content may violate state laws, especially those related to child sexual abuse material and nonconsensual deepfake imagery. The letter formally orders xAI to halt these practices while the matter is reviewed.

xAI and its chatbot Grok have been under intense scrutiny in recent weeks after Grok began generating and posting hyper-realistic sexualized images on X, the social media platform formerly known as Twitter. While xAI has reportedly rolled back Grok’s ability to publicly post such images, testing conducted by Reuters found that the chatbot could still privately generate nonconsensual sexualized imagery on demand as of Friday afternoon U.S. Eastern Time.

Neither xAI nor X has publicly responded to questions regarding the cease-and-desist letter, adding to concerns from regulators and lawmakers. The controversy highlights broader debates around AI safety, content moderation, and the legal responsibilities of companies deploying generative AI tools.

California’s action adds to mounting international pressure on Musk’s companies. Regulators in the United Kingdom, the European Union, and several Asian countries, including India, Indonesia, Malaysia, and the Philippines, are examining Grok’s image-generation capabilities. Earlier on Friday, Japanese authorities confirmed they are also investigating X and Grok, stating that all regulatory options remain on the table.

Japan’s Economic Security Minister Kimi Onoda said officials have requested immediate corrective measures from X but have yet to receive a response. As governments worldwide tighten oversight of artificial intelligence, the Grok controversy may become a landmark case in defining accountability for AI-generated deepfakes and nonconsensual sexualized content.