Ukraine war: Russians kept in the dark by internet search

In many places, searching the web is a gateway to a wider world of information, but in Russia, it is part of a system that helps trap people in an alternative reality.

Shortly after 20 people were killed in a Russian missile attack on the Ukrainian city of Kremenchuk in June, Lev Gershenzon - a former manager at Russian tech company Yandex - typed the city's name into its search engine to find out more.

The results he got back shocked him.

"The sources that ranked at the top of the page were strange and obscure," he told the BBC. "There was one blog by an unknown author claiming that the information about casualties was fake."

The Kremlin keeps a tight grip on the country's media, especially TV, which glorifies Russia's invasion of Ukraine as a mission of liberation and dismisses reports of atrocities as fake.

The internet in Russia was for a long time the main space for alternative sources of information, but after starting the war in February the Kremlin launched a crackdown on independent online media.

Digital rights watchdog Roskomsvoboda estimates that - in the first six months of the conflict - nearly 7,000 websites were blocked in Russia, including those of major independent media and human rights groups.

Our experiment

BBC Monitoring wanted to find out what people in Russia see when they search the web now.

We used a virtual private network (VPN), so it would appear as though we were searching the web from Russia.

Between June and October, we carried out dozens of searches on Russia's top search engines - Yandex and Google - for key words relating to the war in Ukraine.

Yandex is one of the big stars of Russia's homegrown tech scene. It runs the country's largest search engine and presents itself as independent of the authorities.

According to the company's own statistics, it handles about 60% of web searches done in Russia - with Google responsible for about 35%.

From the start of the war, Yandex faced criticism for the pro-Kremlin slant of the sites and stories featured on its news aggregator, Yandex News. In September, it sold Yandex News to the Kremlin-linked owner of social network VK.

But Yandex retains control of its general search engine, and here, the results of BBC Monitoring's experiment reveal an alternative reality dominated by Russian propaganda about the war.

No mention of atrocities

One topic searched for was Bucha, the Ukrainian town where hundreds of civilians were killed by Russian troops before they withdrew in early April.

The deaths shocked the world, but in Russia, many seem to believe the state media line that they were staged by Ukraine.

Yandex results about the killings of civilians in Bucha as if based in Russia (left) featured blog posts denying Russia is to blame, while Google's results in the UK spoke of evidence of atrocities

When we searched for Bucha on Yandex - using a VPN as if based in Russia and typing in Russian - the top page of results made it look like the killings had never taken place.

Three of the nine top results were anonymous blog posts denying the involvement of Russian troops. The remaining six contained no independent reporting of the events.

The discovery of mass burial sites in October in the town of Lyman, after it was retaken from Russian forces, was also reflected on Yandex from a pro-Kremlin viewpoint. Several pro-Kremlin news stories blaming the deaths on Ukrainian "Nazis" were in the top 10 results.

A typical pro-Kremlin blog post highly ranked in Yandex search results denying that the Russian military killed civilians in Bucha

Searching for "Ukraine" on the search engine in the same way also produced results heavily slanted towards the Kremlin's narrative.

Four of the nine first-page results linked to pro-Kremlin news outlets, and none to independent media.

Glimpses of independent reporting only occasionally appeared in Yandex search results with links to Wikipedia articles or YouTube.

Asked to comment by the BBC, Yandex said its search in Russia "displays content [that is] available on the internet, excluding sites which are blocked by the [media] regulator". It denied there was any "human interference" in ranking results.

Results on both Yandex in Russia (left) and Google in the UK (right) covered the mass burials in Lyman, but on Yandex, the reports depicted the bodies' exhumation as 'desecration'

So what happens if you switch from Yandex to Russia's second-largest search engine, Google?

Searching on the US-based company's search engine with our VPN set to a Russian location, and typing in Russian, still brought up pro-Kremlin media outlets, but mixed with some independent and Western sources.

Even more independent sources appeared when we searched Google with the VPN set as though we were in the UK, although still typing in Russian. There were plenty of results covering either the civilian deaths or the war.

Google told the BBC that its search "reflects the content that is available on the open web", and its algorithm is trained to "prominently surface high quality information from reliable sources".

Cleansing the results

So why did Yandex's search results differ so much from Google's?

Several specialists the BBC spoke to said they believed it was unlikely that large-scale manipulation was taking place from inside Yandex, as this would be too complicated to do.

One possibility is that the company's results are simply being skewed by the Kremlin's crackdown on independent reporting of the invasion.

Because thousands of websites have been blocked by Russia's media regulator, huge swathes of information do not appear in Yandex's search results.

Yandex's search results on "Ukraine" as if based in Russia (left) were dominated by pro-Kremlin media and featured no independent media sources, while Google's results as if based in the UK (right) showed only Western reporting

"They [the authorities] can cleanse the results completely," Alexei Sokirko, a former Yandex developer, told the BBC.

At the same time, the Kremlin is spending heavily to make sure web content is created reflecting its own worldview, he added.

Search experts Guido Ampollini and Mykhailo Orlov, from the marketing firm GA Agency, say this will also potentially skew the results users see in Yandex, as the search engine's algorithm may reward pro-Kremlin material with higher rankings and down-rank alternative views.

Artificial web traffic

Could using a VPN help Russians find out more about the war in their own language?

If they are using Yandex to find that information, then not necessarily.

When searching its engine with the VPN set to the UK and using Russian, the odd independent source appeared, but pro-Kremlin sources still dominated.

Mr Ampollini and Mr Orlov say the pro-Kremlin content appears carefully tailored to make the algorithm rank it higher.

And with one obscure news site that featured prominently in results, they also found signs of possible web traffic manipulation.

A large number of potentially artificial links to the site were found from external websites - a common technique to improve a site's search ranking.

Many civilians have been killed in Russian air strikes on Ukrainian cities

Finally, Yandex may be reflecting the fact Russian users themselves are choosing pro-Kremlin content.

Search specialist Nick Boyle, of digital marketing agency The Audit Lab, told the BBC that - unlike Google - Yandex takes into account user behaviour.

This means, for example, that a website's search ranking may be affected by the number of visits to it. Google says this is not the case for its search engine.

The GA Agency team thought it possible that many Russians are clicking on content that portrays their military positively, prompting Yandex's algorithm to reward it with higher rankings.

Lev Gershenzon believes that, no matter how the Kremlin's dominance of Yandex's search results has been achieved, it means that anyone wanting to question what they hear on state media will only get information confirming the official view.

"You open the Yandex main page and you start [searching for the] Kremenchuk [attack] to receive some alternative picture from other sources, and all you get is that 'yeah ok, you're right, it's fake' - and that's it," he told the BBC. "It's like a double punch."