AI creator on the risks, opportunities and how it may make humans 'boring'

"Humans are a bit boring - it will be like, goodbye!" That's the personal prediction - that artificial intelligence (AI) will supplant humans in many roles - from one of the most important people you've probably never heard of.

Emad Mostaque is the British founder of the tech firm, Stability AI. It popularised Stable Diffusion, a tool that uses AI to make images from simple text instructions by analysing images found online.

AI enables a computer to think or act more like a human. It includes what's called machine learning, when computers can learn what to do without being giving exact instructions by a human sitting at a keyboard tapping in commands. Last month, there was a dramatic warning from 1,000 experts to press pause on its development, warning of potential risks, and saying the race to develop AI systems is out of control.

In an interview we'll show in full on Sunday, tech founder Mostaque questions what will happen "if we have agents more capable than us that we cannot control, that are going across the internet… and they achieve a level of automation; what does that mean?

"The worst case scenario is that it proliferates and basically it controls humanity."

That sounds terrifying, but he is not alone in pointing out the risk, that if we create computers smarter than ourselves we just can't be sure what will happen next.

Mostaque believes governments could soon be shocked into taking action by an event that makes the risks suddenly real. He points to the moment Tom Hanks contracted Covid-19 and millions sat up and paid attention.

When a moment like that arrives, governments will conclude "we need policy now", the 40-year-old says.

There's been a spike in concern for example after a Republican attack advert on Jo Biden was created using fake computer generated images.

When there's a risk to information that voters can trust, that's something governments have to respond to, says Mostaque.

• What is AI and is it dangerous?

• Should we be scared of chatting to AI?

• Advanced AI risk to humanity - technology leaders

• AI could affect 300 million jobs - report

Despite his concerns, Mostaque says that the potential benefits of AI for almost every part of our lives could be huge. Yet he concedes that the effect on jobs could be painful, at least at the start.

Mostaque says he believes AI "will be a bigger economic impact than the pandemic", adding that "it's up to us to decide which direction" this all goes in.

AI could lead to 300m job losses according to one prediction.

Some jobs will undoubtedly disappear, the bank Goldman Sachs suggested an almost incomprehensible 300m roles could be lost or diminished by the advancing technology.

While no one wants to be replaced by a robot, Mostaque's hope is that better jobs could be created because "productivity increases will balance out" and humans can concentrate on the things that make us human, and let machines do more of the rest. He agrees with the UK's former chief scientific advisor, Sir Patrick Vallance, that the advance of AI and its impacts could prove even bigger than the industrial revolution.

Mostaque is an unassuming mathematician, the founder of a company he only started in 2020 that has already been valued at $1bn, and with more cash flooding in, including from Hollywood star Ashton Kutcher, is likely to be soon worth very much more. Some speculation has put the value as high $4bn.

Unlike some of his competitors he is determined his technology will remain open source - in other words anyone can look at the code, share it, and use it. In his view, that's what should give the public a level of confidence in what's going on.

"I think there shouldn't have to be a need for trust," he says.

"If you build open models and you do it in the open, you should be criticised if you do things wrong and hopefully lauded if you do some things right."

But his business also raises profound questions about ownership, and what's real. There's legal action underway against them by the photo agency Getty Images which claims the rights to the images it sells have been infringed.

In response, Mostaque says: "What if you have a robot that's walking around and looking at things, do you have to close its eyes if it sees anything?"

That's hardly likely to be the end of that conversation.

The entrepreneur is convinced that the scale of what's coming is enormous. He reckons that in 10 years time, his company and fellow AI leaders, ChatGPT and DeepMind, will even be bigger than Google and Facebook. Predictions about technology are as tricky as predictions about politics - educated guesses that could turn out to be totally wrong. But what is clear is that a public conversation about the risks and realities of AI is now underway. We might be on the cusp of sweeping changes too big for any one company, country or politician to manage.

The first steam train puffed along the tracks in Darlington more than 50 years after the steam engine was patented by James Watt. This time we're unlikely to have anything like as long to get used to these new ideas, and it's unlikely to be boring!

You can watch much more of our conversation with Emad Mostaque on tomorrow's Sunday with Laura Kuenssberg live on BBC One or here on iPlayer.

• On this week's show are Energy Secretary Grant Shapps, shadow shadow business secretary Jonathan Reynolds and boss of tech firm Stability AI, Emad Mostaque.

• Watch live on BBC One and iPlayer from 09:00 BST on Sunday

• Follow latest updates in text and video on the BBC News website from 08:00

Follow Laura on Twitter