The trouble with our trains

As Southern railway passengers brace for a very unhappy Christmas, and a lousy New Year from strikes targeted at the festive season, what, you may wonder is life like on the rest of the rail network?

Something of a triumph, according to the rail industry. "The railways are actually incredibly successful," says Paul Plummer, chief executive of the Rail Delivery Group, which represents the industry.

Certainly the headline statistics look good:

• Passenger journeys more than double since full privatisation 19 years ago

• Record investment

• Safest and fastest growing railway in Europe

But drill into the performance statistics, and a different picture emerges.

Punctuality

Take those posters the train operating companies like to pin up on stations about punctuality.

While they often convey an upbeat picture, blow away the smoke and remove the mirrors and you find that overall punctuality is now worse than at privatisation.

And that can't just be blamed on strike-hit Southern railway. Punctuality has been declining for four years.

Ironically, the number of incidents causing delays has also declined.

The headline official punctuality statistics also mask big drops in punctuality during peak hours.

On some commuter lines out of Manchester, more than six out of every 10 trains are late each day. On one commuter line into Birmingham, four out of every 10 trains have been late.

Overcrowding

Then there are the official overcrowding statistics.

Until 2013, the rail industry liked to boast that trains were no more overcrowded than at privatisation - even though passenger numbers more than doubled between 1994 and 2014.

John Ware on board Britain's trains

It sounds like a conjuring trick - until you decode something called "PiXC".

That's Passengers in Excess of Capacity, the official definition of overcrowding.

This measures the difference between the number of seats of each train journey into and out of London during the peak hours against the actual number of passengers (excluding First Class) sitting and standing at its most crowded point on the journey.

Here's the catch: before kicking in, PiXC makes a big allowance for the numbers standing.

So, like a tank filling up with water, until 2013, carriages could continue to fill up with standing "customers" but because they hadn't hit the critical "overflow" mark, they didn't score on the official PiXC statistics - even though their "customer experience" was becoming more and more unpleasant.

Since 2014, more and more services have just been spilling over past the overflow point so that in just the last two years, there's been a 50% rise in the number of officially overcrowded trains.

Today, some 160,000 commuters coming into London can't get a seat.

Six of the 10 most crowded services are in the home counties. All 10 are between 61% and 129% past the overflow point.

We don't have a clear picture about overcrowding on long-distance and cross-country lines because PiXC doesn't usually measure them.

However, we do know that, by 2005, some cross-country routes were already officially overcrowded - and since then annual passenger journeys have gone up by 600 million.

Fares

Southern is one of the UK's most troubled networks

When it comes to fares, train companies are forever reminding us that they offer many off-peak bargains.

Yet Virgin's morning and afternoon peaks from Euston now total nearly seven hours (06.16 to 09.26am and 3.01pm and 6.44pm).

This means that people wanting to get home to the Midlands or the North are limited in their choice and may arrive at their destination late.

Then there's the Friday night stampede for the first off-peak train with travellers often having to squat on the floor for hours even though they've had to stump up a lot of money for the privilege - £87.70 from London to Preston, for example.

Happy travellers?

With overcrowding, cancellations and fares all going up, and punctuality going down, you might expect a rise in official complaints.

In fact, they've more than halved - down from a million soon after privatisation to under half a million today.

You might think that's because of the train companies' commitment to customer service. All nine parent companies covering 20 passenger franchises in Britain insist they're "passionate" about this.

Well, here's the passion killer. In 2014-15, an organisation called Commute London counted 1.1 million overtly negative tweets about commuters' "customer experience".

Commute London provides live social media feedback on train services in and out of London. And very lively it is too.

On some lines, the issue is ageing rolling stock

Formal complaints seem to have been replaced by a huge rise in informal complaints.

Commute London say the sheer scale of negative tweets shows that the complainants can't simply be dismissed as coming from the ranting brigade. They include the normally polite and curious too.

This rise in social media venting against the train companies is consistent with falling satisfaction ratings.

Once again, the headline figures mask a much less rosy reality.

While the National Rail Passenger Survey shows a gentle decline in overall journey satisfaction - albeit one that, at 80% , is still high - only 73% of passengers are satisfied with punctuality/reliability, 65% satisfied with sufficient room to sit or stand, 45% satisfied with value for money for the price of their ticket and on some routes that's down to just 28%.

Who pays?

What ministers say about how much "investment" is going into the railways shouldn't always be taken at face value either.

They claim "we" are delivering "record" investment to modernise the railway.

Sounds like the government is paying in record government grant, doesn't it?

It isn't. "We" is the royal "we": government and fare payers.

Seventy per cent of that "record" investment now comes from fares, thanks to all those above-inflation fare rises since privatisation, while the share of government grant to the railways has been shrinking.

Here's the titanic challenge the railways pose for any government.

Network Rail has forecast yet a further doubling of passenger journeys over the next 25 years, to an eye-watering 3.4bn.

To keep Britain moving, a continuous series of mega projects sustained over a generation will be required.

For most commuters, crowded trains are a way of life

Although Crossrail 1 will provide 10% more capacity for London's squashed commuters, when it opens in 2018 that capacity will soon be gobbled up by the capital's rapidly growing population as it heads towards 10 million by 2030.

There will need to be Crossrail 2 and quite possibly 3; not just High Speed 2, but also an HS-3; further electrification of the network, digital signalling to release more capacity on the track, relief of a multitude of pinch points, vastly more rolling stock… the list goes on.

It's easy to get up to an investment figure well north of £150bn, never mind the cost of operating and maintaining an expanding railway.

This year's capping of regulated fares in line with inflation is a tacit acceptance by ministers that government has reached the limit of the fare-paying public's patience.

Which means higher taxes. That's if we don't want to go on getting ever more squashed until we really do end up travelling like sardines in a steel box as opposed to a tin can.

John Ware presents Panorama: The Trouble with our Trains on BBC One at 19:30 GMT on Monday 7 November and available on the iPlayer in the UK afterwards.