Growth of online marking 'could improve reliability'

There was a legal challenge over last year's GCSE English exam grades

Traditional marking of exam papers by one person is giving way to a system where parts of papers are marked online by different people, sometimes overseas.

Questions from some papers taken by students in the UK last year were marked online in Australia.

A report by England's exams watchdog Ofqual says such changes could improve reliability.

Splitting up papers for marking can increase consistency, it argues.

The report was ordered after complaints from some heads and teachers about marking standards.

And the grading of last year's GCSE English exams led to a legal challenge, although the judge ruled the process had been lawful.

The report describes the tens of thousands of people employed as exam markers as "knowledgeable, nearly always holding a degree in their subject".

But it warns that "marks can never be totally reliable" and that the biggest factor influencing reliability is the design and quality of the question and the "mark scheme" - a guide to how marks should be awarded.

Australian connection

The report says moves away from traditional "pen and paper marking" towards on screen methods has made it easier for exam boards to check that examiners are marking papers correctly.

And it says it also makes it easier for papers to be marked in chunks - with one examiner marking answers to the same question.

"Existing research shows that item-level marking could improve marking reliability by reducing the effects of biases caused by the rest of the question paper," it says.

Ofqual says when one examiner marks all the questions on a paper, they might subconsciously be influenced by earlier good or bad answers a candidate has given - and give them a mark they do not deserve.

Last year, half of exam papers for GCSE and A-levels in England, Wales and Northern Ireland were split in to chunks for marking, with different examiners marking a batch of the same question.

The other half were marked by a single examiner. Most were checked in the UK, although one exam board, Pearson Edexcel, sent some questions to Melbourne, Australia, for marking.

It says 12% of "items" [questions] from its papers last year were marked online there and that these were either multiple choice or questions with simple "yes or no" answers.

A spokeswoman said technology meant it could monitor the examining process "in real time anywhere in the world".

"However, the specialist subject knowledge required for most answers requires that our examiners have experience and awareness of the UK educational context, so we only use examiners based outside of the UK for multiple choice or yes/no answers," she added.

The company was the first to use on screen marking for general qualifications from 2003 and the trend has been growing ever since.

Automated marking

Last year, two-thirds of papers were marked on screen, Ofqual says.

The watchdog predicts there will be a "further decrease in traditional marking over coming years".

On screen marking involves papers being scanned in to a computer and sent to examiners for marking using "secure systems".

A small number of papers (1%) are marked automatically, not by humans, and these are multiple-choice, says Ofqual.

Chief regulator Glenys Stacey, from Ofqual, said there had been significant developments in marking in recent years.

"We believe the increased use of on-screen marking is a positive thing, as it allows for more frequent and flexible monitoring of examiners, and reduces the logistical risks," she said.

"We are aware though, that this does bring with it risks of new administration problems, which must be managed properly.

"The biggest factor influencing the reliability of marking is the design of the assessment itself - the style and quality of the questions and the quality of the accompanying mark schemes. These are matters we intend to improve as qualifications are reformed."

The report said on screen marking was not without risk and that there had been a "small number of occasions" where systems had not correctly added marks put in by examiners and where some papers had not been scanned.