Should you believe those Pokemon Go download numbers?

The augmented reality mobile phone game, Pokemon Go, seems to have been spreading across the world faster than a video of a cat singing Justin Bieber songs. But how reliable are the figures in the news that tell you how many times it has been downloaded? The BBC's Simon Maybin gives them a poke to find out.

One headline just a week after the app was first released in the US reported that there had been 15 million downloads on Apple's App Store and Google Play.

But the number wasn't put out by the app stores.

Even when Apple confirmed on Friday that Pokemon Go had broken the App Store record for downloads in the first week of release, it didn't give numbers.

"It would be nice if Apple and Google released these figures, but unfortunately they don't do that," says Abhinav Agrawal from SurveyMonkey Intelligence. His company is one of many that has been trying to work out how many people have downloaded Pokemon Go.

"The individual developer of course knows all of their own data, but they may not want to necessarily release all this data because it's a business secret or a trade secret," Agrawal says. "And this is where we step in."

Companies like SurveyMonkey Intelligence do what's known as app analytics. They get their information from a combination of app users, app publishers and the app store rankings. Making sense of it all isn't straightforward. Even the app store rankings are the source of some mystery.

"Apple and Google actually don't release any specific methodology data," Agrawal explains. "Is it for example ranked on an hourly basis? Is it ranked on a daily basis? Do they take into account just today's downloads? Do they take into account uninstalls in addition to downloads? No-one knows exactly how they do it."

Searching for Pokemon characters in Saudi Arabia

Nonetheless, SurveyMonkey Intelligence is confident enough to come up with figures - and their latest for Pokemon Go is that there have been more than 30 million downloads in the US.

But perhaps the media should be careful about presenting these kinds of numbers as facts.

"The thing I want to stress is that these are estimates," says Agrawal. "This really is the equivalent of polling. It really is a matter of building up a large audience, looking at their data, and based on advanced data science extrapolating from that."

Other estimates are available. Another app analytics company told the BBC it thinks that over the same period there have been 20 million - not 30 million - downloads of Pokemon Go in the US.

There's another reason to be wary of download figures.

"Even if we know how many times a particular app is being downloaded, it doesn't necessarily mean such apps are being used on a regular basis," says Sang-Pil Han, an academic at Arizona State University who's been researching mobile app analytics.

"I may have downloaded about 100 apps on to my smart phone, but that doesn't mean that I use every single one of my 100 apps every day."

So maybe it's not download numbers we should be paying attention to, but user numbers.

There have been plenty of those flying around too, including one that says 26 million Americans are using Pokemon Go every day.

The source for that statistic turns out to be SurveyMonkey Intelligence. Abhinav Agrawal says it comes from their panel of mobile phone users - people who have agreed to have software installed on their phones, which measures what they're doing and sends the information back to the app analytics company.

Find out more

• More or Less is broadcast on BBC Radio 4 and the World Service

• Download the More or Less podcast

• More stories from More or Less

Agrawal says they try to make the panels representative of the population at large, but he admits it's harder for them to do this than it is for opinion pollsters, who can just phone people up at random. And while pollsters always publish their sample size, companies like Agrawal's don't. So we don't know how many people are on the SurveyMonkey Intelligence user panel.

We do know that it includes people aged 13 and over. But according to one recent survey, one in four Americans between the age of eight and 12 has a smartphone. And they could well be using them to play Pokemon Go.

Agrawal also says that the figure of 26 million daily users actually came from just one particular day - and that day turned out to come at the peak of Pokemon Go fever. So even if that number was accurate at one point, it isn't any more.

What about the claim also heard and seen in headlines that Pokemon Go is the "biggest mobile game in US history"?

Again, that seems to come from SurveyMonkey Intelligence. It was based on the 26-million-users figure, which it compared with a "rumoured" peak-daily-users figure of 20 million for the game Candy Crush Saga. But given that SurveyMonkey Intelligence doesn't actually know where that figure comes from, can it be 100% confident Pokemon Go is in fact the biggest mobile game in US history?

"I don't think I can be 100% confident about anything in this industry," admits Agrawal. "But I'm actually reasonably confident."

So when it comes to figures around mobile apps, bear in mind these are estimates, not facts.

Next time you hear a big number about app downloads or daily users, treat it with the same - or perhaps a bit more - scepticism as you would opinion poll figures.

Follow @BBCNewsMagazine on Twitter and on Facebook