For years, Western spy agencies have talked of a need to pivot to focus on China. This week, the head of the UK's GCHQ intelligence agency described it as an "epoch-defining challenge".
It follows a series of arrests across the West of people accused of spying and hacking for China. And on Monday, China's ambassador was summoned by the UK Foreign Office, after three people were accused of assisting Hong Kong's intelligence services.
These are a sign of a normally hidden contest for power and influence between the West and China bursting out into the open.
The West - the US and its allies - are determined to push back. But senior officials worry the West has not taken the challenge from China seriously enough and has fallen behind in intelligence terms, leaving the West more vulnerable to Beijing's spying, and both sides at risk of a potentially catastrophic miscalculation.
What concerns Western officials is Chinese President Xi Jinping’s determination that Beijing will shape a new international order. "Ultimately it aspires to displace the United States as the foremost power," the chief of MI6, Sir Richard Moore, told me in a rare interview in his office for a new BBC series on China and the West.
But despite years of issuing warnings, Western intelligence services have struggled until recently to concentrate their focus on Chinese activity.
Nigel Inkster, who was number two at MI6 when he retired in 2006, says China's emergence as a major global power "took place at a time when there were a lot of other pre-occupations".
As Beijing rose on the world stage during the 2000s, the thinking of Western policymakers and security officials - and the focus of the intelligence services - was dominated by the so-called war on terror and military interventions in Afghanistan and Iraq.
Shadow war: China and the West
The rise of China is the defining challenge of our times - how far to co-operate, compete or confront. Gordon Corera asks if the West has been slow to wake up.
More recently, a resurgent Russia and now the Israel-Gaza war have appeared more urgent challenges, officials in the US and Europe concede.
At the same time, there has been pressure from government and businesses alike to focus on securing access to China's huge market, rather than confronting the security risk it poses.
Political leaders frequently preferred their intelligence chiefs not to call China out by name. And businesses too preferred not to admit their secrets were being targeted.
"The pendulum did swing very far in the direction of the economic and commercial interests," says Nigel Inkster.
Chinese intelligence was already engaging in industrial espionage back in the 2000s, he says, but Western companies typically stayed quiet. "They didn't want to report it for fear that doing so would jeopardise their position in China's markets," he says.
Another major challenge has been that China spies differently from the West. That has made its activity harder both to recognise and confront.
A former Western spy says he once improbably told a Chinese counterpart that China did the "wrong type" of spying. What he meant was that Western states prefer to focus on gathering the kind of intelligence that helps them understand their adversaries. But Chinese spies have different priorities.
Protecting the position of the Communist Party is central. "Regime stability is their number one goal," Roman Rozhavsky, an FBI counterintelligence official, explains.
That requires delivering economic growth. And so China’s spies see acquiring Western technology as a top national security requirement. Western spies say their Beijing counterparts share information they have gathered with Chinese state-run companies, in a way that the West’s intelligence agencies do not with their own domestic firms.
'Special treatment'
"My agency is busier than we've ever been in our 74 years of history," Mike Burgess, the plain-speaking head of the Australian Security and Intelligence Organisation (Asio) explained to me.
"I rarely call out countries because when it comes to straight espionage, we do it to them," Burgess told me. "Commercial espionage is a completely different matter, and that's why China is getting special treatment on this one."
He acknowledged that the Western allies had been slow to understand this threat. "I think it's been going on for a long time and collectively we've missed it," he admits.
We were sitting down last October in California, where he was taking part in the first ever public appearance by the security chiefs of the so-called Five Eyes – the intelligence-sharing alliance made up of the US, UK, Australia, Canada and New Zealand.
The unprecedented gathering was a very deliberate attempt to turn up the volume of warnings about China because of the fear that many companies and organisations were still not listening. The location in Silicon Valley was also carefully chosen - a spotlight on China’s attempts to steal technology, sometimes through cyber-espionage and sometimes through recruiting insiders.
China's resources for this are on a different scale. A Western intelligence official estimates China has around 600,000 people working on intelligence and security, more than any other state in the world.
Western security services simply cannot investigate every case. According to British intelligence agency MI5, more than 20,000 people in the UK alone have been approached over professional networking sites like LinkedIn by Chinese spies to cultivate a relationship.
"People can be unwitting that they are in fact corresponding with an intelligence officer from another nation - but eventually they find themselves passing information that rips out the future of their own company," the head of MI5, Ken McCallum told me at the California gathering.
These are "epic" campaigns, Ken McCallum says, which can have serious national security implications, as well as economic consequences.
While the bulk of China’s huge apparatus is focused on domestic surveillance, it also uses its spies to limit criticism of its actions abroad.
Recently, there have been reports of Chinese spies targeting Western politics, with arrests in the UK, Belgium and Germany and an ongoing inquiry in Canada.
There have been reports of Chinese "overseas police stations" in Europe and the US. When it comes to going after Chinese dissidents in the West, security officials say Beijing's intelligence officers usually act remotely rather than using spies physically on the ground - hiring private investigators or making threatening phone calls.
In fact, the first cyber-incidents targeting UK government systems in the early 2000s came not from Russia but China and were aimed at gathering information on overseas dissidents like Tibetan and Uighur groups.
Australia has been at the leading edge of worrying about political interference. Asio says it started to detect activity starting around 2016, which include promoting candidates in elections. "They're trying to push their agenda, which they're entitled to do. We just don't want them pushing it through covert means," Mike Burgess told the BBC. Australia passed a new set of laws aimed at countering this in 2018.
In January 2022, MI5 in the UK issued an unusual interference alert alleging UK-based solicitor Christine Lee had been donating to a range of British political parties as part of a campaign to advance Beijing’s agenda. She is currently pursuing a legal case against MI5 over the claim. It was only in 2023 that the UK passed a new National Security Act which provided new powers to deal with interference and other activity by foreign states. Critics argue they have come late in the day.
The West is, of course, spying on China, just as China spies on the West. But collecting intelligence on China is uniquely challenging for Western intelligence services like MI6 and the CIA. The pervasive nature of surveillance inside the country, thanks to facial recognition and digital tracking, makes the traditional model of human intelligence - meeting agents face-to-face - near-impossible.
China swept up a large network of CIA agents a decade ago. And it is also a technically hard target for GCHQ and the US National Security Agency (NSA), which intercept communications and collect digital intelligence, partly because it uses its own - rather than Western - technology.
"We really don’t know how the (Chinese) politburo think," one Western official concedes.
- Published12 May
- Published14 May
This knowledge gap could lead to misunderstandings and this carries serious risks. In the Cold War, there were periods when the West failed to understand how insecure Moscow felt - and as a result the two sides came close to a catastrophic war that neither wanted.
Similar risks of miscalculation exist today, particularly over China's desire to regain control over Taiwan. There are also growing tensions in the South China Sea where accidental escalation could lead to conflict.
"In the rather dangerous, contested world we are living in we should always worry about conflict and we should set ourselves to avoid it," Sir Richard Moore, the head of MI6, told me.
"Particularly when you have powers that may not always understand each other as well as they should, that is where my service comes in."
The role for MI6, he says, is to provide the intelligence needed to navigate through the potential risks. "Misunderstandings by definition are always dangerous - always better when you have channels of communication open and when you have insights into the intentions of those people with whom you are competing," he says.
That makes ensuring there are open channels of communication a priority. MI6 does have contact with Chinese counterparts over terrorist threats. And the fact some military to military contacts have now resumed between the US and China has been widely welcomed.
Even though greater military and diplomatic contact between Beijing and Washington has lowered the temperature in recent months, the long-term trajectory continues to ring alarm bells.
And all the revelations about spying risks fuelling mistrust and apprehension among the public on both sides which can in turn limit room for manoeuvre in a crisis. Finding a way to live with - and understand - each other will be vital to avoid relations spiralling into a deadly conflict.
BBC InDepth is the new home on the website and app for the best analysis and expertise from our top journalists. Under a distinctive new brand, we’ll bring you fresh perspectives that challenge assumptions, and deep reporting on the biggest issues to help you make sense of a complex world. And we’ll be showcasing thought-provoking content from across BBC Sounds and iPlayer too. We’re starting small but thinking big, and we want to know what you think - you can send us your feedback by clicking on the button below.
Get in touch
InDepth is the new home for the best analysis from across BBC News. Tell us what you think.