Google AI Faces Scrutiny Over Deepfake Terrorism and Child Abuse Reports

Google has disclosed that it received over 250 global complaints within a year regarding its AI software, Gemini, being used to generate deepfake terrorism content. Additionally, the tech giant reported dozens of user complaints about AI-generated child abuse material, according to Australia’s eSafety Commission.

Under Australian law, tech firms must submit reports on efforts to mitigate online harm or face fines. Google’s latest report, covering April 2023 to February 2024, marks a significant revelation on how AI tools may be exploited for illegal content.

Google confirmed receiving 258 user reports on suspected deepfake terrorist or extremist content and 86 reports related to AI-generated child exploitation material. However, the company did not disclose how many of these reports were verified. A spokesperson emphasized Google’s strict policies against creating or distributing such content and reiterated its commitment to online safety.

The company uses hash-matching technology to detect and remove AI-generated child abuse images but does not employ the same method for filtering out extremist content. The eSafety Commission labeled Google’s disclosure a "world-first insight" into AI misuse, urging stronger safeguards in AI development.

Global regulators have increasingly called for better oversight of AI since the rise of tools like OpenAI’s ChatGPT. In Australia, authorities have previously fined Twitter (X) and Telegram for inadequate reporting on harmful content, with both companies appealing their penalties.

As AI adoption accelerates, governments worldwide are pushing for stricter content moderation to prevent AI-generated misinformation, deepfake crimes, and online exploitation. Google continues to enhance its safety measures, but the debate over AI regulation is far from over.