Pokemon Go: Hiding Pikachu may hold key to AR's future

Pokemon Go was the first mainstream success for augmented reality - the overlaying of computer graphics over the real world, viewed, in this case, with a smartphone.

Since its launch in July 2016, players of the game have collectively walked 12 billion miles on the hunt for the little monsters.

But sophisticated as Pokemon Go was, one critical flaw crushed the illusion of “seeing” a Pikachu in front of you. If something got in the way of your phone’s camera, the Pokemon would be rendered on top.

“It really falls apart,” acknowledged John Hanke, chief executive of Niantic, the app's creator.

“It breaks the illusion that this thing actually exists in the world, because it’s not behaving right.”

To solve the problem, Niantic has acquired Matrix Mill, a four-person company spun out of University College London and based in Covent Garden - where it will remain.

The terms of the acquisition have not been made public.

But Niantic says the deal has the potential to make AR much more realistic.

Near and far

The simple way to solve the overlay problem is to have a device that has two cameras. Each lens can take a slightly different image, and by combining the two a calculation can be made as to how far away an object is. Your eyes and brain work in much the same way.

But try this - put your hand over one eye and look around you. Despite losing your depth perception, you can still judge how far away objects are.

That’s because, unless you’re Father Dougal, you instinctively figure out when something is either small or just far away, based on your past experience of the object's typical size.

The technology allows Pikachu and other critters to disappear behind objects

This is how Matrix Mill’s software is designed to work, making it possible for a smartphone with only one camera lens to make a pretty good guess about depth.

The software sees an object, recognises it, calculates how far away it is most likely to be and determines whether the Pokemon should appear in front or behind.

“It allows the characters to join your environment,” Gabriel Brostow, one of Matrix Mill’s co-founders explained.

“Whatever is around you, your characters should understand that and be interacting with that too. Suddenly you’re not just dealing with everything on a single plate.”

The end result was demonstrated in a short proof-of-concept clip shared by Niantic on Thursday.

Where Pikachu was previously stuck to the area directly in front of you, he (it?) can now happily run around between real-world people and hide behind plant pots.

Pokemon Go was launched 20 years after the first Pokemon games were released for the Game Boy

Freeze frames reveal the tech doesn't work perfectly.

There are flickers and misjudged placements at times.

But as gamers play an updated version of the game, they will help Niantic's system become smarter at recognising objects.

Next stop: Hogwarts

Pokemon Go has been downloaded about 800 million times, but there’s no doubt that it is past its 2016 heyday, when beaches and parks were full of Pokemon hunters.

These days gamers are more likely to be fighting for survival in Fortnite.

Enter: the boy wizard.

Niantic’s next major title will be a Harry Potter game.

“What you can expect is something that draws on Pokemon Go in terms of what worked about that game," Mr Hanke said.

"It will be a game where you go outside, you are drawn to new places and you play together with your friends.

“But we’re also using it as a place to explore new gameplay mechanics and new technology.”

Little has been revealed about the Wizards Unite game beyond its logo

He declined to reveal a release date.

Opening up the platform

Also in the works are multi-player experiences that will allow up to eight players to interact in the same augmented space - a concept greatly limited, in my view, by the drawbacks of smartphone AR.

A virtual world, however exciting, lacks any sense of immersion when viewed through such a small screen.

In that regard, smartglasses can’t come soon enough.

Bigger news for the AR industry, perhaps, is Niantic’s decision to open up its “real world” technology to outsiders.

It means the technology that powers Pokemon Go, and the vast amounts of real-world intelligence gathered as a result, will be shared.

“We will continue to make games in that genre,” Mr Hanke told the BBC.

“But obviously it’s an area much bigger, in terms of the creative space, than we can occupy ourselves.

There has yet to be an augmented reality hit to match Pokemon Go's success

“I’m excited to make a platform available to other people to do things that frankly we might never imagine, because people are going to bring their own vision.”

As part of the arrangement, Niantic wants to publish the third-party games that use its tech.

But Mr Hanke would not be drawn on whether he might agree instead to charge a licensing fee in the future.

_______

Follow Dave Lee on Twitter @DaveLeeBBC

Do you have more information about this or any other technology story? You can reach Dave directly and securely through encrypted messaging app Signal on: [phone]