How The Muppets film was boosted by GPU chip advances

Kermit appears to be standing in front of the other Muppets, but was filmed on his own

Click here to see this and other images deconstructed

"We have a saying around here - it's not easy doing green," says Look Effects' director of marketing Pam Hogarth as she prepares to introduce her colleagues.

When The Muppets opens at cinemas in the UK later this week, the film will contain about 350 shots created by the Los Angeles-based special effects team.

However, audiences not in the know will be hard pushed to spot more than a few dozen.

Take for instance Kermit's rousing speech to his fellow performers as he tries to inspire them to save their theatre.

"I don't care what anybody says," implores the long-suffering stage manager. "I believe in you."

His audience looks him straight in the eyes... except it is all an illusion.

Most of the special effects in the film are designed to go unnoticed

When the puppeteers filmed the scene, Kermit was facing a blue screen, the colour chosen instead of the traditional green to accommodate the character's colour.

The 60-odd puppets facing him were added with painstaking care - and state-of-the-art graphics processors - some time later.

Digitally removed

Working this way involved extra effort, but it had its rewards.

"When they first came to talk to us about doing the movie I was a little curious what we would do on it," says Anthony "Max" Ivins, Look's visual effects supervisor.

"I thought we're not going to do CG Muppets, that just wouldn't be right. But it wasn't that, they wanted to give the writers and the director the freedom to just shoot the movie that they wanted to shoot.

"So we did a lot of head-to-toe puppets that were shot on a full blue-screen stage with puppeteers in blue suits.

"Some of the characters have up to four puppeteers running them. And it unleashes them if they don't have to hide from the camera."

In previous Muppet movies, many of the shots had to be designed around the teams operating the puppets, with boxes and pieces of furniture used to obscure them.

Puppeteers wore blue suits to make it possible to remove them from shots

This time, the puppeteers did not need to worry where they were standing because they were completely covered in blue suits, allowing them to be digitally removed afterwards.

But in many cases, that was easier said than done.

"Fuzzy Muppets on blue screen is not the funnest work," says Mr Ivins. "The lighting is never completely even on a Muppet."

Accelerated effects

To ensure audiences do not spot that the puppets have effectively been copied and pasted into their scenes, multiple keys - or cutouts - were taken for each frame of the film.

This allowed the visual effects artists to adjust their compositing tools so that different parameters could be applied to the parts of the characters in bright light from the parts in shadow.

Although much of this work still needed to be done manually, recent breakthroughs in GPU (graphics processing unit) technology helped speed up the process.

Computer chip maker Nvidia's Quadro graphics cards are widely used across the special effects industry - the firm claims about 95% of the market - thanks in part to the take-up of its Cuda programming language among key software developers over the past two years.

Effects programs written in the language take advantage of the GPU's skill at solving a problem by processing lots of little chunks of information simultaneously rather than attempting to process a few big chunks at speed.

When applied to appropriate tasks this offers a performance boost over what could be achieved by relying solely on the CPU (central processing unit).

Compositing is only one process that benefits.

"There are certain visual effects applications that are very keen for GPU," says Dominick Spina, Nvidia's senior technology product manager.

The Muppets' feathers and fur made them tricky to composite, adding to the workload

"Any type of simulation - fire, water, fluids, volumemetrics [for clouds or smoke] - the reason being is that there is a very large computational component to those renders, so you have to physically simulate all of the motion.

"That's very well suited to GPUs as you can send a lot of data to a chip at one time and then get the result back.

"You're optimising your CPU as well by offsetting everything that is better processed on the GPU."

Time to tweak

It may all sound quite technical, but in an industry that works to tight deadlines it can make a huge difference.

In the case of a computer-generated fireworks sequence towards the end of The Muppets film, it allowed the visual effects team to perfect the explosions and see the results nearly instantaneously afterwards.

"We used a lot of lens flare, edge rays and warp effects to create the fireworks," says Look's director of technology, Michael Oliver.

"The artists could sit there with their GPU and deploy these effects and within seconds see the results, versus other workstations that don't have GPUs which would take two minutes to render this on a CPU."

"So they could sit there and tweak the fireworks - they could make them brighter, smaller, flashier.

"Whatever the director or supervisor wanted, and they could sit with the artist and do it in real time to get the look exactly the way they wanted it."

Killer crowds

The speed gains offered by GPUs become even more apparent in cases where the effects are highly complex.

"Now we can do five iterations in an afternoon where it might have taken a day to an iteration before - so it would have taken a week instead of an afternoon to have done the same amount of testing," said Mr Ivins.

The Muppets' most graphics- intensive shot involves a view from a helicopter on to part of Hollywood Boulevard in front of the historic El Capitan theatre which doubles as the puppets' base in the film.

The shot involved replacing many of the street's advertising hoardings with Disney-approved images - including a poster for its Cars 2 movie - as well as filling the area with a computer-generated crowd.

"That's one of the times when the graphics cards came into play in a big way because the crowd is a bunch of little people, so you get millions of polygrams moving really quickly when you get that many characters in one scene," said Mr Ivins.

"That was a fun shot to do because we ran our own particle system that moves the crowd around. And there was a fine line between an active happy crowd and a riot!

A computer-generated crowd on Hollywood Boulevard was one of the movie's biggest effects scenes

"About 10% too much intensity of speed and movement and it's like: 'Uh-oh - they look as if they are all killing each other instead of celebrating.'"

Expensive tricks

Look's artists say there is one downside to the speed gains - some producers now push for "crazy schedules" to save costs, despite the firm's philosophy that "more time equates to a better product".

But Mr Ivins adds that being able to have a film's director or visual effects chief come in and see changes made in real time, rather having to wait for the dailies to arrive, can help speed up the decision-making process and avoid unnecessary work.

So when Crazy Harry explodes Abraham Lincoln's face off Mount Rushmore, and when sparks fly out of new character Walter after he is flung into an electric fence, the audience can be assured that the final results closely match the filmmakers' vision.

"The Muppets have always been about artistic integrity, not cheap tricks," says Kermit at one point in the film.

You may not always be able to trust what you see on screen, but the combination of low-tech puppetry and hi-tech graphics that brings Fozzie, Miss Piggy and the rest of the gang back to life helped all those involved live up to their leading frog's standards.