Image recognition aims to add to Japanese ads

Japanese advertisers want smartphone users to unlock bonus material

Tokyo is probably the most ad-soaked fragment of the planet. Bombarded by visuals shilling something, to someone, somewhere, advertisers vie constantly to attract eyeballs.

There are even TV commercials that dominate the front of 10-storey buildings and animated pitchmen placed above urinals.

So in an effort to grab attention, Japan's Mad Men have to work twice as hard to get their messages across.

The country's biggest advertising outfit, Dentsu, thinks it may have an edge with the latest image recognition technology that encourages smartphone owners to snap at ads, magazine covers and even the seemingly empty ether above their heads with their camera-equipped mobiles.

In return, users of Dentsu's Scan It app are rewarded by being whisked to the advertiser's website or receiving special offers, free music, film clips and digital coupons.

"We plan to use Scan It as content for digital and smartphone campaigns," says Mr Naoki Mori, a creative director at Dentsu, Tokyo and the mastermind behind the new marketing drive.

Bonus material

Dentsu also plans to use its tool for adding bonus material in newspapers, magazines and even product packaging, he adds.

A film detailing the history of Japanese money can be revealed by snapping a banknote

The technology is basically a much advanced version of the 2D barcodes - known as QR codes - for mobiles that surfaced recently in the West and have been used for years in Japan.

QR (Quick Response) codes are small, square printed bar-coded tattoos that resemble a splattered space invader icon and are printed on posters and billboards, and in magazines.

They provide a quick means for anyone with a camera-enabled phone to access web addresses housing everything from information on competitions to addresses and maps for retail outlets.

Instead of tapping out of web addresses, users just snap the code and let their smartphone do the rest.

Dentsu's Scan It application, available on the iPhone, works on a similar principle only it is trained to recognise normal, flat printed images using a method, which for the moment, Dentsu is unwilling to reveal.

Hidden histories

So far, Scan It is only configured to work with images in a sports car catalogue, the cover of popular Manga magazine Jump SQ and, more curiously, Japanese banknotes.

Photographing the portrait of Hideyo Noguchi, the bacteriologist who discovered the agent of syphilis who features on a 1000 Yen note, launches a film detailing the history of Japanese money.

Japanese advertisers want smartphone users to unlock bonus material

Capture Jump magazine's front cover, and a free animated Manga tale starts up on your smartphone screen.

"You can see the potential," says Paul Papadimitriou, mobile technology consultant and publisher of the mobileinjapan.com blog.

"There's lots to do with such an app: scan an ad and get to a campaign site. Imagine - scan a pack of gum and get to an micro site; scan a dedicated image in a shop and get a coupon. Treasure-hunt all images, scan them and win some prize. It's QR code redux," he adds.

The technology used means the phone actually recognises that picture, unlike technology such as Google Goggles that analyses the image then compares it those on the internet to create a list of related pictures and sites.

With images acting as QR codes alternatives, it sends users to specific sites, eliminating the guess work.

Picture perfect

Similar image recognition technology is also emerging elsewhere.

Zurich-based Kooaba, for example, has created what it calls Smart Visuals - a technology it says matches that of Dentsu's.

Kooaba's Paperboy app makes printed media behave more like a web page

By using image recognition technology and the automatic crawling of a large reference database of objects, Kooaba's Smart Visuals apps can recognise printed and digital images taken with a smartphone, web cam or any other digital camera.

"Our algorithms determine and discriminate regions in images and are able to subsequently match them to other views of the same object," says Kooaba founder Herbert Bay.

"By relying on regions in the image rather than on the whole image, we are able to identify the correct match even under challenging conditions," he adds.

Kooaba claims Smart Visuals works well for all kinds of pictures, including book covers, DVDs, games, newspapers and magazines, even buildings.

However, images of animals, plants, cars, furniture, or apparel are not common in its database and consequently do not work so well with its current software.

In future, it may be possible to add face recognition too, say both Dentsu and Kooaba. But the Swiss company is more cautious: "Besides the technological challenges there are also serious privacy issues," it says.

So far Kooaba has used Smart Visuals to create mobile apps such as Kooaba Paperboy, which makes print media interactive via smartphones.

Instead of making a mental note to download or share an article you may be reading in a newspaper, you can snap the page with your iPhone and be taken to a results page, where you can share it with friends, bookmark a digital version, or go to a related web page.

So far this neat trick is only available in a few European countries, but the firm hopes to roll out the service world-wide making a wide range of print titles interactive with a smartphone. The service will be free of charge to publishers, says Mr Bay.

Butterfly effect

Dentsu meanwhile plans to add augmented reality (AR) functions to its app similar to those demonstrated in Japan with "iButterfly" - an entertaining iPhone application that uses AR, the iPhone's camera, motion sensor and GPS functions to collect coupons.

This had users all over Japan this summer chasing virtual butterflies, only visible to iPhone users. Once spotted on screen, a deft flick with the phone would capture the fluttering AR butterfly and reward the user with a coupon.

Adding this AR function to future versions of Scan It will enable the app to interact with 3D objects as well says Dentsu. Shopping in the future then could take a sci-fi turn.

The ability to augment your knowledge about an object by just looking at it via your phone's camera would help you bring up information about a product and where to buy it.

At which point the whole world becomes a billboard and the ad mans job to stand out begins again.