Why for the NSA every call matters

The revelation that the National Security Agency (NSA), America's eavesdropping organisation, had asked mobile phone operator Verizon for phone information relating to millions of customers highlights the growing importance of "communications data" in intelligence and forensics work.

Critical to an understanding of these techniques, which have been in use by the NSA on a large scale since 2005, and which the British security authorities would like to employ on the same scale, is that these methods do not in the main relate to what is being said, simply to the details of communications such as which phone number was called by a particular subscriber and for how long.

It was in Iraq, acting in support of the secret Joint Special Operations Command, that the NSA started to record the details of every phone call made in the country. They did not have enough translators to listen to the vast majority of these calls or to assess their possible value, that was not the point.

What they found was that within months their bulk database of call details would allow them to break open terrorist networks with dramatic speed.

Landing by helicopter to raid a target, teams from Delta Force or the SAS would find telephones, sometimes prized from the hands of dead men dressed in suicide vests.

The number might never have been registered before as one of interest, but within seconds NSA experts could discover every call made on it for months before, mapping the dead bomber's connections with other individuals. This was only possible because of the blanket recording of all call details.

The use of what special operators in Iraq referred to simply as "the database" quickly grew so refined that software was developed to map networks (spotting that some numbers relevant to only one member, for example, might be family members) and that a portable, laptop-sized, terminal tapping into it started to be carried on raids.

By 2006 teams of special operators using the terminal could hit one member of an al-Qaeda cell, and hunched over the computer in a half destroyed Iraqi house, find other members through the database, fix the location of these phones and perform a "bounce on", jumping back into their helicopters to raid the next target.

Seeing the extraordinary effects of these techniques in the field, the NSA sought to apply them to domestic communications from 2006 onwards. Since the recording of this data does not include the content of the conversations, it can be extended to US citizens more easily and does not require a warrant.

It soon became apparent to people at the eavesdropping organisation that collecting information on this scale, relating to tens of millions of calls each day, and then keeping it for months or even years would require a vast new data centre to store the information.

So it began work on its $2bn Utah facility, where a ribbon cutting ceremony was held this week.

British intelligence people working in Iraq with the UK special operators (the task forces codenamed Black, Knight, and Spartan) saw these techniques and were duly impressed. They too wanted to apply them in the domestic counter-terrorist or criminal arenas.

It had in fact been the practise for many years (since the 1984 Telecommunications Act, for example) for British operators to keep customers' "billing information" for access by the government and for this to be possible without the same level of legal proof or suspicion as eavesdropping on the calls themselves.

The moves to put broad categories of "communications data" into the so called "snoopers' charter" represents an evolution of this.