Are the UK economic growth figures reliable?

Every now and then you hear criticism of the official economic growth figures from the Office for National Statistics (ONS).

Take these comments as an example, from 23 October 2009.

"Do today's data tell us anything about what is really going on in the economy? Probably not.

"In the past, the ONS's early GDP estimates have contained no statistically useful information about growth."

Did these views, quoted in a blog by Stephanie Flanders, the BBC's former economics editor, come from some extreme outlier in the economic world? No, they came from Ben Broadbent, who was an economist at Goldman Sachs at the time and who is now deputy governor of the Bank of England and a member of its interest rate-setting Monetary Policy Committee.

GDP is a measure of all of the goods and services produced by an economy over a particular period, usually three months.

His comments were based on research by his colleague Kevin Daly, who is still at Goldman Sachs and is not a fan of the figures.

"We didn't believe the data in 2009," he says.

The economy was recovering much more slowly than other economies, but other indicators didn't agree. In particular, the employment data was difficult to reconcile with other data."

Mr Daly says there is much less debate about the period now, and that ONS reviews have agreed with his analysis.

ONS chief economist Joe Grice sees things a bit differently.

"In the second half of 2009, lots of commentators were saying that the economy was growing fast, but it's clear now that it wasn't," he says.

He adds that it's a good thing the Bank of England did not believe the people who said there was a lot of growth, because then it might have raised interest rates and "we would have been in terrible trouble by now".

How GDP became the figure to watch

Quick figures

There is no question that there are uncertainties surrounding the first estimate of UK growth measured by gross domestic product (GDP), as Joe Grice explained last year.

The first figures come less than a month after the end of the period they're measuring, which is pretty swift by international standards.

At that point only about 44% of the returns from businesses have been received.

"Only the US does it so early - the rest of Europe does it 15 days later," says James Mitchell, professor of economic modelling at Warwick Business School.

"Revisions are inevitable because we put pressure on the ONS to produce estimates so early."

GDP estimates are indeed constantly being revised, most formally one month and two months later, but they never stop being revised, mainly because of changes to the methodology.

When there are big changes to the methodology, like the ones due later this year that will change the treatment of spending on research and development for example, the changes have to be backdated, so there are likely to be revisions to some pretty old figures.

Statisticians never stop revising their estimates for GDP figures

So, with how much of a pinch of salt should the preliminary estimate be taken? Joe Grice says plus or minus 0.2 percentage points would be fair.

What that means, as with most economic data, is that what is important is the trend rather than the precise figure. This was a particular problem in 2011 and 2012 when there were several quarters of growth or contraction of 0.2% or less.

You may remember the double-dip recession that disappeared after the figure for the first quarter of 2012 was revised.

'Difficult task'

It's important to bear in mind just how difficult it is to measure GDP for an advanced economy.

Jonathan Haskel, professor of economics at Imperial College Business School, explains part of the problem with the example of an economy producing steel and cars.

You can't just count all the sales by the steel company and the car companies because some of the steel has gone into the cars, so you need to avoid counting it twice.

Instead you have to count the value added at each stage of the process.

"In the face of a difficult task the ONS is probably doing the best it can," says Prof Haskel. "It's probably not doing it any worse than anyone else."

Steel and cars: avoid doubling up

And the task is being made more difficult. "The economy is changing in ways not captured well by the statistics," says Diane Coyle, author of GDP: A brief but affectionate history.

She cites the increasing numbers of small firms and the importance of things like software and trademarks.

"We need to take the quarterly figures more cautiously than we do," she says.

She adds that the ONS is better than most at calculating the figures. Respected economist and former Bank of England committee member Kate Barker will soon publish her review of the quality of ONS figures, so we will soon get another assessment of their quality.

It should be said that the ONS is generally judged to have captured the sharp declines in GDP as a result of the financial crisis pretty well, which improved many commentators' views of its methodology.

Also, few would argue against the idea that bodies such as the MPC need to use official figures from the ONS alongside things like business surveys, which may give more immediate ideas of the state of the economy as well as hinting at what may happen in the future.

Indeed, the Bank of England is unusual in producing its own indicator called the GDP backcast, which combines the ONS figures with other factors.

Nonetheless, its official position is that the Monetary Policy Committee puts considerable weight on the ONS data and understands that it needs revising for honourable reasons from time to time.

And Ben Broadbent hasn't objected to that so far.