Children making AI-generated child abuse images, says charity

Children are making indecent images of other children using artificial intelligence (AI) image generators, according to a UK charity.

The UK Safer Internet Centre (UKSIC) said it had received "a small number of reports" from schools but called for action now before the problem grew.

It said children might need help to understand that what they were making was considered child abuse material.

The charity wants teachers and parents to work together.

It pointed out that, while young people might be motivated by curiosity rather than intent to cause harm, it was illegal in all circumstances under UK law to make, possess, or distribute such images, whether they are real or generated by AI.

It said children might lose control of the material and end up circulating it online, without realising there are consequences for these actions. It also warned that these images could potentially be used for blackmail.

Knowledge gap

New research conducted by classroom tech firm RM Technology, with 1,000 pupils, suggests that just under a third are using AI "to look at inappropriate things online".

"Students using AI regularly is now commonplace," said Tasha Gibson, online safety manager at the firm.

"In fact, their understanding of AI is more advanced than most teachers - creating a knowledge gap. This makes keeping pupils safe online and preventing misuse increasingly difficult.

"With AI set to grow in popularity, closing this knowledge gap must become a top priority."

It also found teachers were divided over whether it should be the responsibility of parents, schools or governments to teach children about the harms caused by such material.

The UKSIC wants a collaborative approach, with schools working together with parents.

"[We] need to see steps being taken now, before schools become overwhelmed and the problem grows," said UKSIC director David Wright.

"Young people are not always aware of the seriousness of what they are doing, yet these types of harmful behaviours should be anticipated when new technologies, like AI generators, become more accessible to the public.

"An increase in criminal content being made in schools is something we never want to see, and interventions must be made urgently to prevent this from spreading further."

Victoria Green, CEO of the Marie Collins Foundation - a charity which helps children impacted by sexual abuse - warned of the "lifelong" damage that could be caused.

"The imagery may not have been created by children to cause harm but, once shared, this material could get into the wrong hands and end up on dedicated abuse sites.

"There is a real risk that the images could be further used by sex offenders to shame and silence victims."

'Declothing' app dangers

The scope for AI to turn children into the generators of extreme content was demonstrated in September by an app which creates the impression of having removed someone's clothing in a photo.

It was used to create fake nude images of young girls in Spain, with more than 20 girls, aged between 11 and 17, coming forward as victims.

The images had been circulating on social media without their knowledge. So far there have been no charges brought against the boys who made the pictures.

So-called "declothing" apps began emerging on social media sites in 2019, often on messaging service Telegram as automated software with AI features - also known as bots.

Initially very unsophisticated, improvements to generative AI have allowed apps - like that used in Spain - to become much more effective in creating photorealistic fake nude images.

The Spanish bot has nearly 50,000 subscribers - implying it has had that many users, who pay a fee to create pictures, typically after being able to make several for free.

The BBC asked the maker of the bot for comment but they refused to provide a response.

Javaad Malik, a cyber expert at IT security firm KnowBe4, told the BBC it was becoming harder to differentiate between real and AI-generated images, a trend that was fuelling the use of "declothing" apps.

"It's got mass appeal unfortunately, so the trend is just going up and we're seeing a lot of revenge porn-type activities where cultural or religious beliefs cause a lot more issues for victims," he said.

Additional reporting by Chris Vallance and Liv McMahon.