University teaching grades invalid, statistics body says

The Royal Statistical Society has attacked the way in which teaching standards are measured in universities in England - the Teaching Excellence Framework (TEF) - calling it "invalid".

Urging the statistics regulator to intervene, the society said it was "likely to mislead students who use TEF to inform their university choices".

The Office for Students said that the awards system was still evolving.

An independent review will report on the system by the summer.

The right information?

The TEF is meant to help young people make comparisons about the quality of teaching when they are applying to university.

It gives grades of gold, silver and bronze - but the statistics body is warning that it is not based on sufficiently reliable information.

• Leading universities rated 'bronze' under teaching grading system

• How the 'TEF' grading system was introduced

• Degree courses to be rated gold, silver and bronze

In December, Universities Minister Chris Skidmore described the measure of teaching as "leading the way in providing students with greater transparency and choice".

But there have been questions about whether it really measures teaching quality - as there are no actual inspections of lectures or tutorials.

It is a paper-based exercise based on available data and written submissions from universities.

And the Royal Statistical Society said "all TEF awards to date are invalid" because of its concerns about how this data is collected, analysed and reported.

How does the system work?

Higher education institutions are assessed on a range of measures, including reported student satisfaction, dropout rates and whether students go on to employment or further study after graduating.

But none of these metrics directly measures the quality of teaching and there are no actual inspections of lectures or other teaching.

Each institution's performance on each measure is then compared against other institutions with a similar intake.

Performance that looks a good deal better or worse than the benchmark is flagged to assessors, who make the final determination of a university's gold, silver or bronze award.

But the Royal Statistical Society said:

• the measures did not assess quality of teaching

• the benchmarking procedure does not properly take account of all of the differences between universities

• the flagging system is trigger-happy - too likely to flag an institution as different from the norm

And as a result, the system may mislead students about the quality of teaching they might receive.

The government is scheduled to receive a report on the TEF from the independent reviewer Dame Shirley Pearce.

The Office for Students, the higher education regulator which oversees the grading system, said it was awaiting the review and would "look forward to its conclusions".