Google search chief Amit Singhal looks to the future

Mr Singhal joined Google in 2000 and is responsible for its search algorithms

For more than a decade, boxes have helped Google dominate search - both the rectangle on its home page and the web browser address bars that send users to its results.

But smartphones now account for more internet use than PCs, and that changes things.

"When you have a 5in-diagonally-across screen - it's not designed to type," acknowledges Google's search chief Amit Singhal.

"So, on mobile you have to fundamentally give users new ways to interact."

To address the problem, Mr Singhal's team has developed Now on Tap.

The facility - which is being released as part of the latest Android mobile operating system - lets users get related information about whatever is on their handset's screen with a single button press.

As an example, Mr Singhal describes a text chat with his wife, in which he suggests a restaurant.

He explains his spouse could bring up driving directions and the place's opening hours simply by holding down the home button when the restaurant's name was displayed.

The feature works with any app. And if someone wants to know something specific, they can trigger a contextual voice search by saying: "OK Google".

One possibility would be asking: "Who's the lead singer?" when a song's name is displayed in Spotify.

"It's search designed for the mobile world," says Mr Singhal.

"You don't have to switch windows to type information into one window and then go to another."

Now on Tap works by analysing the text shown, allowing users to ask contextual questions

Privacy

Like many of Google's services, there's a trade-off involved.

Those using the service will end up disclosing ever increasing amounts of information about their interests and habits.

That helps Google target its ads, but also opens it up to criticism.

"They're gobbling up everything they can learn about you and trying to monetise it," said Apple's chief executive Tim Cook earlier this year.

"We think that's wrong… and we think some day, customers will see this for what it is."

Apple has made much of the fact it anonymises information gathered about users, meaning, for example, it sends location-specific alerts without recording their travel histories alongside their account IDs on its servers.

Tim Cook suggests Apple's privacy policies mark it out from ad-focused tech firms

Mr Singhal retorts that Google gives users "full control" over the data it has access to, but adds that he believes people appreciate the benefits that sharing their data makes possible.

"If Google doesn't know where my office is or what time my next meeting is, it can't [automatically] tell me there is traffic - please leave now," he says.

"We believe with opt-in and proper consent and control over your data, Google can provide you so much more value."

Virtual helpers

Resting on its laurels wasn't an option.

US-based users spend more than six times more hours using apps rather than mobile browsers - often to search for goods - according to a recent study by Comscore.

Now on Tap prevents Google being shut out of such activities when the apps aren't its own.

But the firm faces an additional threat - consumers are increasingly being encouraged to use rivals' virtual assistants:

• Apple pioneered the way with Siri, and is about to extend the service to TVs

• Microsoft has brought Cortana to Windows 10 and is expanding it to Android and iOS

• Facebook is testing M on its Messenger app

• Amazon is offering Alexa to third-party gadgets and apps

Facebook, Microsoft and Apple have designed their virtual assistants to have "personalities"

It may seem curious that Google's competitors are giving their tools personalities, while its own voice-search results seem soulless in comparison.

But Mr Singhal suggests such computer-coded characters aren't yet ready for prime time.

"One of the challenges with computer-based personalities is that human-based interactions are far more nuanced than can be encoded in an algorithm today," he says.

"A funny character or a sarcastic character may not be the right character all the time.

"Sitting across here from you, I can read your emotions. Computers don't do that.

"I deeply believe that we have to be very sensitive to human interaction, so we won't do this lightly.

"And I think that technology will take a long time to develop."

Organising memories

Google unveiled its Photos app at its developers conference in May

In the meantime, Google is investing deeply in image recognition.

Early fruits of the initiative include its recently launched Photos app, which attempts to automatically catalogue images with the appropriate keywords.

Mr Singhal shows off how he can command his phone to find and display photos of turtles he took on a recent holiday, despite the fact he never tagged them.

The tech has, however, had teething problems - not least an incident when it accidentally labelled black people as "Gorillas".

While that issue has been fixed, Mr Singhal acknowledges the limitations of the technology in its current state.

Google Photos accidentally labelled black people as "Gorillas" in June

"When we started working on speech recognition, many years back, we used to get one in four words wrong," he says.

"And if you get one in four words wrong, you can't build a viable product out of it.

"Computer vision is at that stage today.

"Today we can tell this is a chair - but we can't tell what brand of chair, what model of chair."

Such limitations didn't stop Amazon offering Firefly - a feature on its Fire phone that let users scan real-world objects to get information about them.

But Mr Singhal suggests that it's no coincidence that the handset flopped and was recently withdrawn from sale.

Amazon's Fire phone featured software that aimed to identify objects captured via its camera

"I'm glad they tried doing these things, because it pushes the envelope for all of us," he says.

"But at the same time we had Google Goggles as an app way before.

"And we learned that the technology still needs a lot of work before you can build a billion-user product out of it."

Competition probes

Unsurprisingly, Mr Singhal is not keen to discuss the intricacies of the regulatory probes Google faces.

India and the EU have accused the company of abusing its lead in search to stifle competition, and the Wall Street Journal recently reported that the US was also looking into the issue.

"How this world will experience information on mobile devices is a wild open field with numerous contenders, many of them with very deep pockets," he says.

"I feel that trying to think about the future with the lens of the past is absolutely the wrong thing to do. That's what stops innovation from happening."

That's not to say, however, that he believes that Google won't continue to lead its field.

There are reports that the US's Federal Trade Commission is investigating Google

"Having competition in this space keeps us honest," he says.

"Science progresses. We build things that don't exist otherwise.

"We have done this for 17 years. The future will be built on the solid foundation that is Google Search, and we believe that is the future that will be useful for our users."