The Mandalorian: Luke Skywalker deepfake technology 'could be used in really bad ways'

The final scenes of The Mandalorian brought back one of Star Wars' most iconic characters.

Luke Skywalker returned to save the day, looking as he did in Return of the Jedi, a film released 38 years ago.

So how did the character come back looking as if time had stood still since 1983?

In the latest Disney Gallery / Star Wars: The Mandalorian episode on Disney+, the team behind the show explained how they did it.

Deepfake and de-ageing technology

Mark Hamill, the actor who plays Luke Skywalker, is nearly 70 years old, so he couldn't play a version of the character in his 30s, right...?

Well actually he could, thanks to modern technology.

Lucasfilm began the process working with deepfake technology, it involves feeding in as many images and video clips of a person into a computer as possible. The computer AI (Artificial Intelligence) is then able to learn lots about a person's face, how it moves when they talk or make a facial expression such as smiling and can then digitally copy them.

Eventually the team at Lucasfilm decided they wanted to have Mark Hamill return to act in the scene, so they moved to use de-aging methods seen in previous Marvel and Star Wars films.

In doing so, they combined the on-set acting performances of Hamill and another actor named Max Lloyd-Jones, who physically resembled a young Luke Skywalker.

Mark Hamill inside a booth called 'The Egg' where lighting is controlled and high-res images were taken of his face

Hamill then had to film for a second time inside a special booth called 'The Egg'.

Inside The Egg, the visual effects team had control over lighting and could take super high-resolution images and textures of the actor's face.

The end result was the technology combined the performances and made the younger actor look like Luke Skywalker from 1983.

The response from fans was mixed though, with some complaining that the movements of Luke's mouth didn't properly sync with what the character was saying.

One YouTuber, who made his own deep fake version of Luke Skywalker in The Mandalorian, was given a job at Lucasfilm, after lots of people said it looked better than the original digital effects.

"We still have a few years before it really gets indistinguishable, but I'm telling you it will get to a point where you can't tell them apart," said Mandalorian executive producer Jon Favreau.

"Something people didn't realise is his [Luke Skywalker's] voice isn't real. His voice, the young Luke Skywalker voice, is completely synthesized using an application called re-speacher.

"It makes you realise it becomes harder and harder to trust your own eyes and ears when it comes to this stuff," Favreau added.

Technology could be used in 'bad ways'

Deepfakes use a type of technology called 'machine learning' to create a digital version of someone, mapping a person's face and mouth movements so that it can then copy them. Which is pretty cool in movies and TV shows, but what if that technology was used to fake things that politicians or celebrities have said or done?

"There's going to become a point where it's so sophisticated, that it could certainly be used in really bad ways," says Mandalorian director Peyton Reed.

Amateur deepfakes can be made using apps or programs, but they can usually be spotted by showing unusual flickering or blurring around someone's face.

"I think in the case of deepfake, as we're exploring these different technologies to make Luke Skywalker look young for a story we couldn't tell any other way," says Favreau.

"We're telling you how we do it and showing you this is stuff we didn't develop, this is stuff that's available readily. It's sort of a reminder that it might be used for purposes that are more misleading."

Mandalorian executive producer Jon Favreau thinks videos should have a source of verification so you can spot real clips from fakes

The movie producer added that big tech companies have a responsibility to identify fakes as the technology improves and people can't recognise what is and isn't real.

He said: "If we can anticipate these things, I wonder if certain images or videos that are released in an official capacity could have some kind of a stamp with it. If the big tech companies came together and said we should come up with a standard of verification - they do it in social media.

"Something that when you see something you know it's real, because it's becoming harder and harder to tell fake from real."