Is the US-UK's special relationship in decline?

Is Anglo-America a failed state? In the realm of intelligence and national security, the special relationship is being tested.

As he spoke to a crowd of Americans, Britain's defence minister, Michael Fallon, had a US-UK flag pin on his lapel.

"Just as together we broke the bonds of totalitarianism and tyranny in the Second World War," he said at the Center for Strategic and International Studies (CSIS) in March, "so we faced down the threat of communism in the cold war".

Today the shared battle is against Islamic State. Citizens of both countries have been held hostage and then beheaded by IS in Syria, strengthening the resolve of leaders of both countries to combat the militant group.

Yet even with the co-operation in intelligence matters and military efforts, the relationship is strained.

President Ronald Reagan and Prime Minister Margaret Thatcher shared common political values

"Many analysts believe that some reassessment of the 'special relationship' may be in order," wrote Derek Mix in a recent report written for Congress. "There are questions about whether the relative influence and centrality of the relationship is facing a decline."

When it comes to many "global challenges", he said, "the UK may not be viewed as centrally relevant."

In recent months, US officials have criticised UK defence spending. Americans say they're "worried", as one US army commander explained to a Reuters journalist, spending would drop below 2% of the UK's gross domestic product, a threshold Nato members are supposed to maintain.

The UK has also baulked at conducting air strikes in Syria. In this and in other ways London has chosen not to follow America's lead.

After Fallon's speech, a journalist asked if he'd come here on a damage-control mission. He scowled and said he refuted "any suggestion" patching things up was the reason for his trip to the States.

A special relationship over time

• Facing the Soviet threat - under MI6's Stewart Menzies the relationship between intelligence agencies in the UK and the US deepened

• Fighting in Iraq and Afghanistan - US and British officers shared intelligence about insurgents and the Taliban

• Combating violent extremists - John Brennan, CIA director, and John Sawers, who's served as chief of MI6, faced militant groups around the world

He reminded those assembled that President Barack Obama once described the UK as America's "indispensable partner".

Privately, though, Obama advisers talk about the special relationship in different terms. "We snicker about it a little," said one White House official who asked not to be named.

The remark sounds mean, but it was not intended to be. Like others in the US administration, he believes in the special relationship. Yet it's not as important to him as it is to British officials.

Some of the friction is rooted in their distinct approaches to national security and intelligence gathering.

"The colonial hangover is brutal," said PJ Dermer, a retired US army colonel who worked with British officers in Iraq. In contrast Americans remain enthusiastic about their global role.

Sir Stewart Menzies (with his wife, Pamela), headed up MI6 during a time of close US-UK relations

"The US has acted as if it's always available - 'We surge, we do this, we do that'," said David Ucko, an associate professor at National Defense University in Washington and an adjunct fellow at King's College London. "It's far more pro-active."

Still Americans and Britons work closely together.

Michael Hayden, a former CIA director, said there's been "trails beaten down in the forest" between the CIA and its British counterpart, MI6 - to the extent that Americans at other intelligence agencies were jealous.

One US intelligence official told Hayden he wished the CIA officers would be more forthcoming. "He said, 'We'd like you to treat us like you treat the British,'" Hayden recalled.

The two spy agencies have gotten into disputes, but they usually work things out.

When Col Oleg Penkovsky, a Soviet intelligence officer, said he wanted to hand over secrets, both Britons and Americans were suspicious.

But they had different ways to approach the matter.

Michael Hayden, the former director of the CIA, said British and US officials work closely together

"We're Americans," said David Gioe, a former CIA officer who wrote his University of Cambridge dissertation about the special relationship. "We have our checklist. Out comes the polygraph."

MI6 officers saw the polygraph in a different light. "The Brits were absolutely horrified," he said. "They think it's witchcraft."

Together it was deemed unnecessary, and everybody moved on.

Charles Ries, who served as the US government's chief economic official in Baghdad, also praises the US-UK relationship. "The military guys used to joke that whenever we wanted to do something, they'd say, 'We're in,' and then they'd think about it," he said.

He paused. "That was before Iraq."

During the invasion of Iraq in March 2003, differences seemed minor - and endearing.

"They brought their tea," said Jay Bachar, who was part of the military effort.

"Not only did they have tea, but they had a well-stocked bar," Derek Harvey, who served as an Army intelligence officer in Iraq, said.

"We shared virtually everything with the States," Kim Howells, former Labour chairman of the Intelligence and Security Committee, said. "And they shared everything with us."

Yet four years after the invasion, the US had a force of 150,000 troops. UK troops numbered 5,500. And Iraq was in flames.

In the spring of 2007 Shia militants kidnapped five British men, Peter Moore, a computer specialist, and his bodyguards, Jason Creswell, Alan McMenemy, Alec MacLachlan and Jason Swindlehurst, in Baghdad and headed for a district known as Sadr City.

Britons and Americans both tried to save the men. "There was nothing really compared in terms of sheer anguish and emotional strain," said Howells.

"It's one thing to think you can mount a rescue," he said from his house in Wales. His voice trailed off. "It just became a horrible and seemingly impossible task."

British soldiers in Basra, Iraq, in 2006

One of the hardest parts, Howells says, was sorting out whom to trust. "They would tell us they knew someone," he said, describing the individuals who came forward in Iraq. "And so often you had no idea if these people were just lying through their teeth."

Sitting over a Guinness in Georgetown, Dermer, who has a deep suntan and bony hands, looked agitated. It's been almost a decade since he spent time with British officials in Baghdad, but he still gets worked up when he talks about the kidnapping.

In late 2007 he found himself in the midst of an argument with "this Shia guy", as he described him, whom he had met regularly to talk about "the future of Sadr City".

"We had formed a bond," he said. At one point the "Shia guy" said he knew the kidnappers, and Dermer started trying to negotiate their release.

The British officials stepped into the negotiations and pushed too hard, according to Dermer, demanding information their interlocutors didn't have.

Dermer saw things spinning out of control, and he turned towards the British official in the room.

"I told him, 'Shut the [expletive] up.'"

Tony Blair supported George W Bush's push for war in Iraq

"They didn't know the environment," Dermer said. He looked down at the table. "They screwed up," he said, adding "it could have contributed to what happened down the road".

Howells said he could see how friction could develop. "You know the British had one network of contacts and the Americans had another," he said.

Peter Moore arrives back in the UK in early 2010

"It's very, very difficult to arrive at a firm conclusion because you get two different pictures."

Moore was released in December 2009. But Creswell, McMenemy, MacLachlan and Swindlehurst were killed.

Americans and Britons continue to work together in the fight against militant groups in the Middle East. Yet tensions remain.

"The US is carrying the main burden," said Ucko, adding that Europeans are happy to say Americans will take the lead.

Even in small numbers, though, Britons give US-led military and intelligence-gathering operations a worldly gloss.

British accents are still "met with deference" said Ucko. "You could call it soft power."

Working with British officers has symbolic, if not operational, value.

UK defence minister Michael Fallon (right) made a US tour in March

And while Britons are now investing less in the partnership, they still care. One high-ranking UK official who has worked in Washington and Baghdad - and asked me not to use his name - said he saw first-hand what it means.

"When the relationship with the US is going well the light shines upon you," he said. "When you are no longer useful, it can become a very cold and dark place."

Follow Tara McKelvey on Twitter.