Reality Check: Should you care about economic models?

As the EU referendum campaign continues you will be seeing lots of headlines about how much Brexit would cost or benefit the UK economy, or indeed how much staying in will cost the economy.

Now, some of these figures will be plucked out of the air, but some of them will be based on serious economic modelling - carried out by folk with brains the size of planets, most likely swimming around in think tanks.

Should you believe all of them, some of them or none of them and how do you know which to treasure and which to discard?

Economic models - are they like fashion models?

No, they're a bit more realistic than that. But not much.

A lot of people trying to decide whether to vote to leave the European Union or stay in want to know what would happen if the UK left.

And the trouble is that we don't know.

Honestly we don't. And anyone who tells you they do is lying.

Among the key questions to which we don't know the answers are what sort of trade deals the UK would manage to do after leaving, how long they would take to negotiate, how much of the UK's contribution to the EU Budget would be saved, what that saved money would be spent on, whether the regulations that the UK government devised to replace the EU ones were better than the EU ones and what effect all that had on the economy.

There are also more subtle effects on the economy that are even harder to measure such as whether a Brexit would create some sort of feelgood factor in the UK economy, or the opposite.

So economic forecasting is a bit tricky then?

Yes it is. Remember the Budget? The Office for Budget Responsibility, which does the forecasts that the government bases its decisions on, made really big changes to the predictions it had made for the economy only about four months before. And that was without a really major event such as leaving the EU to cloud its predictions.

Also, that was looking at what will happen in four or five years, while the EU questions has ramifications for decades, during which the global economy could change beyond recognition.

The PwC report commissioned by the CBI concluded that the cost of Brexit could be as much as 5% of GDP and 950,000 jobs by 2020, figures heartily disputed by the Leave side. How would they have worked that out?

The way it works is that you build a model to predict the future based largely on how particular things happening in the past have affected the economy.

For example, your model will be programmed for what effect the pound being weaker or stronger against the euro would have on the economy, or what effect an increase in tariffs on particular exports or imports would have.

Does it work?

If you plug in all the right numbers to start with then it might do, but this is where the problems start.

Because in order to work the model you have to make some assumptions in order to have the figures to put in.

You need to take a view of what trade deals would be done and when, what difference it would make to trade, whether the pound would fall and a whole host of other variables.

And the numbers you come out with at the end are enormously sensitive to these assumptions. The margin of error on such forecasts would generally completely dwarf the effects they were trying to identify, if people bothered to try to quantify it.

Is it all a big waste of time?

George Box, one of the greatest statisticians of the 20th Century, said: "All models are wrong, but some are useful."

While the numbers may be of little value, the direction the models predict and some of the assumptions they make are quite interesting.

Friday's report from the Centre for Economic Performance said that leaving would be bad for the economy, predicting falling trade because of rising non-tariff barriers to British goods.

How big a hit the economy would take, whether it's the equivalent of £850 per household in its "optimistic" scenario, £1,700 in its pessimistic one, or between £4,200 and £6,400 per household in the long term is less important.

How do we know which reports to take seriously?

That's the tricky question.

Look at how well-respected the people conducting the research are. Look at who has commissioned it and who is paying for it.

If it's been commissioned by organisations campaigning for one side or the other then you could reasonably be a bit suspicious of its findings.

But that's not necessarily a firm rule. A well-funded, independent group can be just as wrong as a biased one.

"People are not necessarily wrong because they're biased any more than they're necessarily right because they're rich," says Will Moy, director of the fact-checking organisation Full Fact.

Also, be a bit careful with the question of EU funding. Lots of research organisations bid to do some work for the EU and many universities receive some funding from the EU. Almost none can say they have never received any European money, but it doesn't necessarily mean they are in favour of staying in the union.

Then have a look at the assumptions they are making, what they are predicting will happen and whether it seems too pessimistic or optimistic.

In particular, make sure the organisation is clarifying what assumptions it has made and how the model works. If not, don't believe it.

And remember there's also the option to decide that you don't care what even the finest economists predict, or even that you don't think the economic impact is the most important thing about EU membership.

READ MORE: The facts behind claims in the EU debate