Coronavirus: The great contact-tracing apps mystery

Germany and Ireland have both trumpeted their success in rolling out contact-tracing apps.

But is there any evidence that they are doing what they are designed to do - warning people they could be infected with the virus?

Not yet - and the privacy-conscious way in which they are designed could mean we will never know how effective they have been.

Last month, the UK government announced that it was abandoning a centralised NHS contact-tracing app for England and switching to a decentralised version, based on the Apple-Google toolkit. This model has been favoured by privacy campaigners because the matching process takes place on users' smartphones rather on a central computer, providing a greater degree of anonymity.

But the government stressed that the new version would not arrive in a hurry, and Boris Johnson told the House of Commons that no country in the world had a functioning contact-tracing app.

The Leader of the Opposition, Labour's Keir Starmer, immediately pointed to Germany, as did many others who disputed the prime minister's claim.

Germany's app launched in mid-June

Germany's Corona-Warn-App was rolled out nationwide in June. A few days ago, the Robert Koch Institute provided an update on its progress, celebrating the fact that it had now been installed by around 16 million people.

"A successful start that speaks for great interest and acceptance among the population," a statement from the app's developer said.

Bear in mind, however, that there are 83 million Germans, and it is thought more than half of the population need to have an app before it is truly effective.

"The app works" added the institute's president Prof Lothar Wieler. He went on to say that about 500 app users had tested positive for the virus and "had the opportunity to warn others via the app".

But he then said: "We cannot say exactly how many people were warned, because of the decentralised approach of the app."

In other words, we do not know whether the software is performing its key function.

To know how well your app is performing, it is also vital to understand whether it is producing a lot of false negatives or false positives - in bald terms, whether the software is failing to alert people who have been in close contact with an infected user, and sending warnings to those who haven't.

That's important to know, because studies have indicated that Bluetooth is an unreliable way to determine the distance between two people in some common situations.

The Robert Koch Institute (RKI) told us it too didn't know because, once again, the log history it would need to tap into "remains with the users, encrypted on their smartphones".

We wondered as a workaround whether Germany's manual contact tracers might provide RKI with a record of people who had come to their attention after being alerted by the app.

No, came the answer. Local municipalities run manual contact tracing and do not share this information.

RKI added that it hoped scientific tests would eventually assess the app's impact in the "medium to long term", but had yet to work out how this would be achieved.

Switzerland faces the same issue.

Developers of the SwissCovid app have said an update next month will make changes to how it measures Bluetooth signals

Its health department also pointed the finger at SwissCovid's use of Apple and Google's model.

"We hence have the same limitations in terms of statistics. We don't know - and have no way of finding out - the number of people warned by the app or any false positives/false negatives," a spokesman told the BBC.

'Looking positive'

No such caution from Cian Ó Maidín, whose firm Nearform is behind Ireland's app.

Covid Tracker app was launched two weeks ago and has rapidly attracted 1.3 million users, about a third of Ireland's smartphone users.

Nearform is also developing a very similar app for Northern Ireland, which decided to go it alone after the NHS project hit trouble. And Mr Ó Maidín suggested that other parts of the UK and major American states could also be customers soon.

"This is a solved problem," he told the BBC.

"We have a solution that works well, it's been tested, it's been verified. We can get a government up and running from the point at which we kick off to launch in a month."

Later he diluted this to "it may not be perfectly solved at this time" but continued to brush off concerns about effectiveness.

Ireland's app is forming the basis of an app soon to be launched in Northern Ireland

"We don't have a full data set yet but the early data looks really positive," he said, directing us to Ireland's health service HSE for a more detailed response.

We have actually been chasing them on this point for days, and will provide an update if we hear back.

• Ireland's Covid Tracker app is out - where's England's?

• What went wrong with the England's contact tracing app?

• England's virus-tracing app switches to Apple-Google model

As the Irish app is based on the same model as Germany and Switzerland's, it is not clear why it should generate more data.

Although there is one potential caveat: during the installation process, users are asked to agree to the collection of "anonymous metrics" about the "effectiveness of contact-tracing processes".

In any case, Mr Ó Maidín says we should not be overly concerned about missed or bad matches.

"The perfect should not be the enemy of the good," he says.

The problem is that with so little data exposed, can we even distinguish good from bad?

Problems abound

There have also been problems with contact-tracing apps in other countries:

• Downloads of Japan's Contact-Confirming Application (Cocoa) have slowed, totalling 7.7 million on Monday - the country's population is over 126 million. A bug in the software had prevented users from being able to register testing positive for the virus, but this was fixed in an update released on 13 July. But the Japan Times reports that since then only 27 positive cases have been registered via the app.

• Italy's Immuni has been downloaded by about 4.2 million Italians, falling far short of the government's target - the country's population is about 60 million. Local authorities suggest there has been limited interest because the spread of the virus is perceived to be under control, but expect this to change if a second wave begins.

• Australia's CovidSafe has yet to identify anyone that had not already been flagged by other contact-tracing efforts. Zdnet reports that it also continues to be bedevilled by a series of flaws that have caused matches to be missed and iPhones to try to connect to unrelated devices. One ex-medical official told local news the cost involved in developing and promoting the app had been 70 million Australian dollars ($49m; £38m).

• On Monday, France's data watchdog wrote to the country's government requesting changes be made to its StopCovid app, to make it compliant with privacy laws. The last official update says it has been downloaded 2.3 million times.