Lost in translation: do interpreter apps work?

The BBC road-tested a host of translation apps in the Spanish Basque city Bilbao

They are going to remove language barriers forever. Or so it has been claimed.

Google recently released a real-time translator, while Microsoft-owned Skype is beta-testing its own.

Both firms make grand claims about their services, while other smaller rivals offer alternatives, but the question remains: do they actually work?

I took a selection of apps to Bilbao in the Spanish Basque Country to find out, testing them by completing tasks set for me by colleagues on the BBC's tech desk.

The first job was to find the northern city's Guggenheim museum and ask what was its most valuable work of art.

Getting to the museum was not a challenge; another Google app saw to that. The problem was getting my question across.

Bilbao to-do list

• Get to Guggenheim museum and find out what is its most highly insured work of art

• Get to Moyua metro station and ask someone there the best way to get to Plaza Nueva

• In Plaza Nueva find someone to tell the story of their first kiss

• Find Gili-Gili and ask someone inside to take a selfie posing with you and an item sold there

• Catch a taxi to Cafeteria Concha and, when you arrive, ask what their bestselling pintxo is

• Go to the city's bullfighting museum and ask the staff inside how many people the bullring could hold when full and when the original was destroyed

• Buy a one euro stamp and postcard and send it to the Tech team

Google Translate doesn't yet support spoken Basque, so I opted for its English-to-Spanish setting.

Despite near-perfect conditions - indoors with no background noise and a volunteer who was familiar with the concept of the real-time translators - it initially struggled to convey relatively simple phrases.

"OK, so I've arrived at the Guggenheim and I'm here with Begoña and I have to ask her a question, according to my list of things to do," became "ok so the rise of the guggenheim and and and i have got acid? On my list of things to do."

While it got it the second time round, the answer that came back was not exactly perfect:

It was, at least, comprehensible. An inauspicious start, nevertheless.

First kiss

At nearby Moyua metro station, the app had to face an even stiffer test - one that exposed what is perhaps its biggest weaknesses: background noise and the rigours of real life.

The weather was crisp but dry when I managed to stop a woman at the building's entrance to ask for the quickest route to the old town's main square.

By the time she had grasped the concept of listening for a translation into Spanish and answering the question clearly and slowly - delivering the directions bit-by-bit and waiting for the app to catch up - a sudden hailstorm had struck, shaking her resolve to persevere with struggling software and British tourist alike.

Nevertheless, I eventually found Plaza Nueva.

There sat 22-year-old Yurena, who agreed to share the story of her first kiss with me.

Apparently, it was at a party in Bilbao. She cuddled up with Miguel on the sofa and then "muy bien". The couple are still together, she said.

A romantic story, without doubt. Unfortunately, none of it told using Google's real-time translator, which could not deal with the fact that she was speaking at the speed of a normal conversation.

Yurena dispensed with the realtime translator, which couldn't keep up with her

In the end, Yurena had to give up and type her message manually, pressing a button to get it translated into text for me to read.

Her task was made no easier when Google Translate turned "can you speak slower, please" into "can you speak Spanish Big Show".

Una selfie, por favor

Vocre supports 38 languages, but is not free like some rival apps

Vocre is one of the most prominent other apps to provide real-time translations. And, at Bilbao's bullfighting museum, it achieved a similar rate of success to Google's.

Converting between English and Spanish, it managed to convey a question about the bullring's capacity relatively well.

The answer that came back from staff member Joaquín Vega was clearly mangled: "Around 14 thousandth people."

Imperfect, but at least it was understandable.

Such apps performed best with measured speech and low background noise, as illustrated by my experience in Gili-Gili - a local clothes shop.

Inside, where it was relatively quiet, Google Translate managed to convey my request for a posed selfie almost perfectly.

After some explanation of how the system worked, Pedro Aramaio - who spoke little or no English - agreed to pose for the photo, ticking another task off my list.

That was, however, a brief highlight.

Pedro Aramaio agreed to pose for a selfie after conversing using Google's app

At nearby Cafeteria Concha, which was significantly louder, the same app had trouble picking out any speech at all.

The barman there professed to speak very little English.

But when the software took an age to detect my request to identify his bestselling pintxo - a word for Basque bar snacks - he opted to attempt an answer in my language, rather than to persevere with the smartphone.

That's definitely not what I said

'Awkward' apps

If real-time translation apps can get it right, they could upend a lucrative sector.

According to a recent report by the Economist newspaper - which cited consulting firm Common Sense Advisory - the language interpretation industry generates about $37bn (£24bn) worth of sales every year.

But the problems I experienced in Bilbao suggest that processor-powered translations still have far to go.

Those issues are indicative of speech recognition tech's limitations in general, according to Joseba Abaitua, an academic at the modern foreign languages department at Bilbao's University of Deusto.

Mr Abaitua, who specialises in online communication, suggests that interacting via a smartphone while face-to-face with someone else is always going to be "awkward". But, he adds, apps will become more effective as people get used to them.

"In a way, you have to… make a compromise, you have to know who are you talking to - you are talking to a speech recogniser, a machine," he says.

"So, the machine can start understanding you quite well and, if your sentences are short and well recognised, the translation system… may make a good job.

"But, if you start talking unexpected things with a lot of colloquialism, then the whole system breaks down."

He adds that systems need to get much better at recognising people's different accents and ways of speaking.

None of the apps had Basque, but they did have Spanish

Smartphone-based translators also face technical limitations, such as their reliance on internet connections and limited battery life.

And, in some cases, they simply promise more than they can deliver.

One app I tried called Interpreter steadfastly refused to live up to its name.

When I attempted to use it to find a postcard shop it failed to translate either my questions or the replies I was given, despite near-ideal conditions.

A helpful shopkeeper ultimately had to regress to pen and paper to draw a crude map of the area.

Eventually, I bought a stamp and card to send back to London. But not before Google Translate managed to maul this last request.

"I speak English," the bemused card salesman said, finally putting me out of my tech-induced misery.