Hiring: NHS demand outstrips staff supply

• Aged 70, there are ageing pressures on both demand and supply for NHS staff, and planning for that has been weak.

• Unfilled posts are significant in smaller and rural hospitals and GPs. Changes to doctor training is pushing the NHS towards more centralised services.

• To address the problem, training takes a long time and recruitment is a global challenge.

The NHS is turning 70, well past the age of having its own bus pass.

While most others can retire at that age, the National Health Service is still working. Indeed, it's having to work flat out, and it's feeling the strain.

Some of that strain comes from more demands being placed on it. Expectations of what it can achieve are completely different to 1948.

And it has to put up with patients, many of whom do a poor job at looking after themselves. A lot of the strain comes from budgets that keep rising but struggle to keep up with numbers of patients and expectations of what can be done.

But even with lots more money, there's a shortage of people to do the jobs necessary. So-called workforce planning is one of its biggest headaches. I've been looking at many different facets to that workforce planning challenge. Such as:

It's big

In Scotland alone, 163,000 people work for the NHS, not all of them full time. They work the equivalent of 140,000 full time jobs.

The workforce has never been so big. The Scottish government can take credit for boosting staff numbers in most areas of what the NHS does.

There are gaps in the workforce, there's the prospect of bigger gaps to come, because that workforce planning has been weak, and the problems can't be put at the door of one government.

They reflect decisions a long time ago. And decisions made this year may not have much impact on the health service for well over a decade.

'Weak?' Who says?

The process was strongly criticised by Audit Scotland in a report published in June 2017. Auditor General Carolyn Gardner hasn't updated that report, and has yet to issue the report into the particularly challenged General Practitioner sector.

Three frameworks for different aspects of planning have been published in just over a year. But in an interview earlier this week, the public sector audit supremo showed little expectation that things will have changed that much in a year.

"We know the population of Scotland is ageing," she told me. "That affects the number of people in need of health and social care, the type of care they need, and the population available to work as doctors and nurses, and provide that healthcare.

"So there is longer term planning, which is a real challenge, and which we found the government hasn't got to grips with yet.

"On top of that, there are some very immediate challenges - high levels of vacancies for medicine and for nursing in parts of Scotland, and vacancies that are very hard to fill. And there's a real risk that as the workforce ages, we'll see a surge in retirements which will leave some real gaps."

Where are the shortages of doctors?

The overall problem of gaps might look manageable with 7.5% of consultant jobs lying vacant in March. But look under the headline numbers from last March, issued this month, and you find some areas of much greater challenges.

In clinical radiology, 50 specialists were needed. That's one in seven posts. In psychiatry, 41 posts went unfilled - roughly one in nine posts unfilled.

And overall these numbers have got much higher, more than doubling in five years, to 400-plus vacancies

For general practitioners, there's a shortage of around 850 doctors. Nearly quarter of all GP practices cannot fill their gaps.

So why hold back on recruiting?

They're working on it, and quite hard. The rate of doctor and dentist jobs going unfilled for more than six months is 4.5%. That's one in 22 jobs.

Add to that an absentee rate which is higher than target across NHS Scotland, and you find the pressure begins to tell on those having to fill the gaps. Even if the supply of staff isn't there, the demand for their services doesn't relent.

It's not all about doctors, though?

If you take nurses and midwives: in the past five years to March, vacancies for posts were up from 1,800 to 2,800.

Of those, the number left vacant for more than three months more than doubled over those five years, to a shortfall of 850 posts.

If you look at allied health professions - from speech therapists to podiatrists and dieticians - the vacancy rate is not as high, but it does hit home in certain areas, such as physiotherapy. There were 164 vacancies there.

And how is this spread around the country?

The big problem is with rural Scotland, or towns and cities with general hospitals, rather than the big teaching hospitals near universities, where it is relatively easy to secure recruits. The gulf in recruitment can be in a distance as short as Edinburgh Royal Infirmary to St John's in Livingston, half an hour's drive away.

The problem has a lot to do with the way doctors train and specialise. If you're training grade and wanting to mark out a career with a specialism, you can't be accredited if you're not seeing a sufficient throughput of patients.

So you may get to do lots of things in, say, Belford Hospital in Fort William, or Dr Gray's in Elgin, but if you want to specialise, you have to be in a city where there are lots of patients with often similar problems. (As a patient, you would surely want to be seen by a doctor who has lots of practice with your ailment, even if it's some distance form home.)

That's how doctors specialise to progress their careers. So it's more difficult to attract doctors to rural Scotland or these smaller cities.

If we dig into the numbers from the official NHS website, in Dumfries and Galloway, March saw 105 doctors and dentists in post, but 27 posts unfilled. The vast majority of those had been unfilled for more than six month. That's a fifth of all hospital medics and NHS dentistry posts.

In March, NHS Orkney had no doctor vacancies, but a couple of departures can make a big difference. In NHS Shetland, they had one anaesthetist, but should have had three. They had no doctors in general medicine, but should have two.

That puts a lot of strain on others who are working there, and costs when patients have to be flown to the mainland where the doctors are. One of the positive results is increased tele-medicine, with consultants assessing patients by video link.

Does it make any difference that women dominate the NHS workforce?

It's true that they do - 77% of the workforce is female. In nursing and allied health professions, it has always been so.

From last September, there have been more women doctors and dentists in the NHS in Scotland than male. And that is likely to continue to grow, as medical schools admit women in the majority.

That can lead to greater challenges with workforce planning. Women are, for well-known family reasons, more likely to take career breaks, or to want to work part-time.

That's one of the reasons why more than half of all GPs in Scotland are part-time. (The other factor may be that they are very well paid. On £90,000-plus, pro rata, it's easier to get by on part-time pay.)

To be clear, this is not all about women, but they have a significant role in this. The problem that arises from so many part time GPs is consistency for patients. They may see someone different every time they visit the surgery.

And it puts more pressure on the full-time GPs to run the practice and do all the administration, which is rarely an attractive part of the job.

So what to do?

There are two simple answers: let's recruit more from outside Scotland. And let's train more. Sounds simple. Far from it.

On training, if you want an extra anaesthetist, you need to recruit an 18-year-old into medical school this September. And you'll get a fully formed anaesthetist no earlier than 2029.

Among Audit Scotland's criticisms of workplace lack-of-planning was that there hasn't been much of a look to the needs of the NHS and of patients in 10 or 20 years time.

Planning has tended to be on the basis of what has gone before, with some incremental change. And alongside all the growing demand pressures from patients, there is the huge upheaval of the way services are run, which links health and care services.

Another problem: we lose a lot of people that are trained here in Scotland. They'll take their skills elsewhere. In the past decade, it is reckoned that around 3,000 Scottish-trained doctors have left the UK.

That's nothing new to Scotland. Medical expertise has been one of its significant gifts to the rest of the world, going back centuries. But it's costly, and it makes planning more difficult if there's a leakage from the system.

For all the nurses recruited into Scotland, we also lose a lot to work overseas, where their skills are sought after. Some come back later in life, of course.

That helps explain why one in three nurses are aged over 55. Audit Scotland found some are entitled to full pension from age 55, but couldn't find out how many.

What about recruiting?

There's an international market for health workers, and demand is very high, for many of the same reasons causing such problems inside the NHS. With a professional health qualification, you can pick a lot of the world in which to work.

And other health services are out to make things more attractive. I heard this week of one Scottish-trained doctor in Australia, who has 10 weeks of leave each year, wondering if he should move to a job with 14 weeks of leave.

The strengths of the NHS - in training, quality and ethos - has attracted many foreign health workers over its 70 years. It's doing so less these days. Anecdotal evidence I've picked up points to foreign doctors looking to work elsewhere, partly because the lack of resource and staff vacancies is grinding them down, partly because they want to work in a country that makes them feel welcome.

Brexit isn't far from this. Even before it is implemented, there's a very sharp drop in the number of newcomers into the health service from the rest of the European Union. Irish recruiters have been spotted in Scotland, poaching NHS staff.

From outside the EU, the Tier 2 Visa has discouraged many from applying. The "hostile environment" for immigrant workers made it awkward, expensive and time-consuming to go through the processes for getting a work permit.

Recently, the pressure for scarce places in the Tier 2 quota has pushed up the effective salary floor, taking a Tier 2 Visa out the reach of those being recruited to earn less than £50,000.

Sajid Javid, as new Home Secretary, was quick to overturn that medical constraint, which had been inflexibly applied during the Theresa May-era hostility at the Home Office.

And yet another problem with overseas recruiting: the ethical dilemma of how much the NHS can strip expensively-trained health professionals from developing countries which have poorly staffed and poorly resourced health services of their own.

One solution can be a temporary arrangement, to help training and return the recruits to their countries, with higher level skills to offer when they get back home.

So what other solutions are there?

Senior NHS figures have told me this has to be approached in alignment with budgeting, which remains tight of course, and with changes to the way health and care services are delivered.

That means a much more sophisticated approach to understanding where professionals will be needed differently, at least in the medium term.

If, for instance, dementia continues to lack a cure or more effective treatment, it is clear that there will be considerable demands for improved and much more extensive care services. But if a wonder drug is discovered, that is going to be a much less attractive career path.

Another example is in heart surgery. Those trained to wield the scalpel for open-heart surgery, and who enjoyed a very high status within the profession, have been done out of a lot of work through the angiogram camera inserted through the arteries and capable of being used to repair blood vessels.

To become more sophisticated in forecasting the workforce needs, the pressure is building to end recruitment to separate health boards. There are 22 of them, 14 territorial, and often with similar problems.

This is just one of the reasons why consolidation or centralisation or both is the direction NHS managers expect to take - to get to a reduction in the number of boards, and a radical re-drawing of acute services, putting them into far fewer centres of excellence.

A new health secretary offers an opportunity for some fresh thinking on this. Jeane Freeman was advising Labour ministers at Holyrood more than 11 years ago, when they were trying to consolidate services.

The arrival of the SNP in government, with Nicola Sturgeon as health secretary, brought a reversal of two Accident and Emergency closures in Lanarkshire and Ayrshire. That put a chill on further consolidation.

Now an SNP MSP, and representing rural communities and small towns in South Ayrshire, Ms Freeman will be under pressure from the professions to use radical surgery on the NHS if it to remain healthy well past the age of 70.