Deepfake Pornography

Australia's eSafety Commissioner has launched decisive enforcement action against a major UK-based tech company for enabling the creation of AI-generated deepfake pornography of Australian school children. The company, which runs two of the world's most-visited AI nude image websites, was found to be facilitating the creation of what is classified as Child Sexual Exploitation Material (CSEM).
​

The Problem: 'Nudify' Services in Schools

This is not a niche problem: the websites in question were attracting around 100,000 Australian visitors per month and have been identified as being used to generate explicit deepfake images of students within Australian schools.
​
The eSafety Commissioner, Julie Inman Grant, highlighted the app's alarming lack of safeguards and its deliberate marketing of features that encourage CSEM creation. This included explicit "schoolgirl" and "sex mode" options, making it shockingly easy to take an ordinary photo and sexualise a child in seconds. The resulting deepfakes are often highly realistic, causing incalculable psychological and emotional harm to the young victims. Reports of digitally altered images targeting children have more than doubled in the past 18 months, with four out of five reports involving the targeting of females.

---

The Solution: eSafety’s Enforcement Action

​Australia’s eSafety Commission is helping to stamp out these problems using the strong regulatory powers under the Online Safety Act.
​

1. Formal Warning and Potential Penalties: eSafety has issued a formal warning to the UK company for breaching an industry standard by failing to provide appropriate safeguards to prevent the creation of CSEM. This is the first step in an enforcement process that could lead to severe civil penalties of up to $49.5 million if the company fails to comply with Australian safety standards.
2. Harm Removal: The eSafety Commissioner’s office has a high success rate—up to 98% in cases of image-based abuse—in working with platforms to remove harmful material. This includes synthetic content like AI deepfakes.
3. Proactive Regulation: The enforcement action acts as a major deterrent to other AI and technology companies. It highlights that under world-first Australian standards, tech providers are responsible for ensuring their services are not misused to create this harmful content.

eSafety urges any Australian who has experienced image-based abuse (including deepfakes) to report it on their website. For allegations of a criminal nature, always report to local police first, and then to eSafety.

If you want to hear more about how the eSafety Commission is dealing with the broader issues of AI-generated harm, check out this video: Australia's world-first move to protect kids from harmful AI chatbots | 7.30. This video discusses the eSafety Commissioner's efforts to regulate AI tools, which is the same body taking action against the deepfake image websites.

Ep350

Image created by AI