Chief Executive of exam board CCEA, Justin Edwards, updated the Education Committee on A-level results.
Education Minister Peter Weir briefed MLAs on this year's A-level results
That concludes this rather lengthy but action-packed meeting of the Education Committee.
We hope to be back with further coverage of the committee next week.
In the meantime, have a great weekend and enjoy the sunshine
Image source, NI AssemblyChris Lyttle, chair of the committee, thanks the panellists for this contribution to the meeting.
He says: “We recognise just how important these grades and qualifications are for children and young people”.
Mr Lyttle says the results process has presented stress and challenges to young people.
The clerk then runs the committee through some general business.
The next committee meeting is pencilled in for Tuesday afternoon for a briefing on the minister’s update on schools restarting.
Another meeting will be held on Friday for the committee to discuss GCSE results.
The SDLP's Justin McNulty says he's "a little bit aghast" that CCEA is "determined to hold tight" and he thinks this is "extraordinary".
He asks what preparations have been made to cope with "the deluge of appeals that are most certainly going to come into the system".
Mr McNuty also wants to know if there is capacity in the school system for resits.
Image source, NI AssemblyJustin Edwards says CCEA has arranged the appeals system such that they can "streamline and speed through the process as quickly as possible".
Regarding re-sits, he says that CCEA has been of the view that it would run "the normal examination timetable".
Mr Edwards says he believes the capacity for re-sits exists, but CCEA "will have to wait for the re-sit data post re-sits".
Image source, NI AssemblySinn Féin’s Catherine Kelly wants to know what assurances can be given that those young people from working class backgrounds “have not been further disadvantaged through this process”.
Justin Edwards says CCEA doesn’t hold data for social quintiles and “hasn’t mapped that data” but does have grammar versus non-selective data.
“In terms of outcome, there seems to be a more positive position,” he adds.
Robbie Butler (below) of the UUP ssys he spoke to many principals across his Lagan Valley constituency yesterday and there was not one who did not have "huge frustrations".
He says he's concerned about time and asks whether CCEA has any idea about the number of appeals it will be getting.
Mr Butler says that based on the discussions he has had with the schools he would estimate that there would increase five-fold compared to last year.
Image source, NI Assembly"It is very difficult to anticipate," says Mr Edwards. He says he has spoken to three schools principals from different sectors who indicated they expected appeals would be largely unchanged "but they are more satisfied schools".
"As of last night we'd only had 100 appeals, and the same time last year we had 208," he adds.
The CCEA official concedes that this years appeals system is different but that "the batching will increase the speed of the process".
He says extra resources will be made available if necessary.
Daniel McCrossan of the SDLP is up next.
“I am hugely disappointed with how you and the minister have handled this entire process,” says the West Tyrone MLA.
He turns to correspondence that he has seen, which, he says, was sent to CCEA by an academic at Queen's University and which raised concerns about the method being used to calculate grades.
“If I was to assess your handling of this Justin, as the chief executive, I’d be giving you a U,” he says adding “ to be very blunt about it, I am very, very, very unhappy with your handling of this”.
“CCEA failed and have failed the young people - 11,000 of them - and have failed schools and have failed teachers,” says the MLA.
“This model didn’t work.”
Image source, NI AssemblyMr Edwards replies that he recognises the correspondence and outlines the consultation which took place in relation to developing the method.
“I can’t comment on who or when one individual was responded to in this entire process,” says Mr Edwards.
“My organisation has worked, without break, since 20 March” says the CCEA head and outlines the bodies which he has worked with over the last few months in the development of the model.
He adds that it was not just about making “academic considerations but societal considerations of getting people their grades”.
He says “if there was a viable alternative to this situation, then let it be brought forward”.
Image source, NI AssemblyThe DUP's Robin Newton asks if there is a greater emphasis put on teacher assessments in the awarding of GCSEs.
Mr Edwards confirms that this is the case "due to the limited amount of prior performance data".
Mr Newton asks about the appeals procedure.
"I really do hope that here in Northern Ireland we've provided as many tools as we can to make the appeals process as open and easy to use for schools," says Ms Farragher (above).
"We really don't want to make this a long, arduous task for them," she adds.
Lagan Valley MLA Robbie Butler asks the “value of AS results and how robust they are”.
He wants to know how much value was given to AS levels in terms of teacher’s predictions for students.
Mr Edwards replies that there was a range of evidence collected from schools.
Image source, NI AssemblyMr Butler then says, “I think that the minister and you would be agreed that the AS grades are a pretty robust measurement”.
He adds that as a result, anyone who had their final grade lowered from their AS grade would be appealing.
Mr Butler says if that is the case, “let’s scrap the appeals and go to the data we have".
Mr Edwards says “the AS is a component” adding he would expect a school to refer back to that within an appeal.
"As there is in every other year, there may be cases where students have particular circumstances that the schools tried to represent," says the CCEA head.
Karen Mullan of Sinn Féin, the committee deputy chair, asks a technical question about A-level grades.
She asks, if there are eight pupils in a class, and their teacher calculates that all eight would secure a “strong B”, would they be likely to attain a C grade if the school did not previously have as many B grades recorded in previous years.
Margaret Farragher from CCEA says the “the final grade has been based on individual student's AS performance and rank order”.
“So I think that’s perhaps where we’ve seen a difference between what the teacher thinks the student should attain and the AS they should be awarded based on the marks they received,” she adds.
Mr Lyttle, the committee chair, asks what the situation will be for GCSE students.
Ms Farragher says “for GCSE it is very different”.
She explains that “we don’t have a qualification like the AS, so we will be using centre performance to help guide us” alongside the rank order.
Image source, NI AssemblyKaren Mullan comes back in with a supplementary and wants to know the number of downgrades by school sector.
Justin Edwards, the head of CCEA, says in the “grammar sector the match between the centre-assessed grade and actual result grade was 61.3%” and in the secondary the match was 49%.
He says percentage of over-optimism in the secondary sector was 48.1%, while it was 35.1% in the grammar sector.
Meanwhile the percentage of under-optimism in the secondary sector was 2.9%, with 3.7% in the grammar sector.
Chris Lyttle has the first question for Mr Edwards.
He says the committee had previously expressed concern about the statistical model that was to be used in calculating grades.
The committee chair says a "key concern" that has emerged since yesterday is the need for an understanding of what exactly that standardisation modelling is, and he asks the witness to explain it in more detail.
Mr Edwards refers to two packs of slides provided to the members and says he will use them for explanation.
The slides are not available on the video feed.
Image source, NI AssemblyThe CCEA official begins:
"In the A-level qualification we used a combination of centre-assessed assessment grades, that is the grade that the teacher believed, or projected the child would most likely occur, had examinations taken place.
"There is a variety of background information provided in order to give that, the rank order from 1 being the highest through.
"And then the data that we used for awarding was a combination of the AS-level outcomes, the re-sit data, the centre-assessment grades and the rank order."
He then hands over to Ms Farragher who explains the A-level model in greater detail.
Chris Lyttle asks a number of questions, including raising concerns about the use of rank ordering.
Ms Farragher says that now that the results have been announced, they could be used to help with the explanation of the system, which she says is "complicated".
"I'm thinking now that it could be useful to give real-life examples", she says, suggesting that these could be in the form of animations.
In his opening remark Justin Edwards says “this was an exceptional year” as “examinations were cancelled, probably for the first time in living memory” as a result of Covid-19.
He says CCEA was directed to achieve two things, namely for students to get their grades by 13 August, “to progress in education and journey through life and through work”, as well as ensuring those grades “maintain, as far as possible, the standard with previous years in order to maintain the value of those qualifications”.
Mr Edwards says a range of options were considered, adding that there was no perfect solution and “so we had to seek the next best option”.
These options were presented to the CCEA council and the minister, and Mr Edwards says CCEA was then instructed to proceed “with GCSE, AS and A Level qualification using a combination of teacher judgement with statistical standardisation”.
He adds that the teacher judgement component “is two component parts” comprised of grades predicted by the teacher, as well as asking “teachers to rank order the young people in their class groups”.
Image source, NI AssemblyThe CCEA chief executive adds that this ranking order was submitted to CCEA by electronic portal.
Mr Edwards says it was not just an effort by CCEA but the entire education system and thanks those involved.
He then turns to the data behind the results which that awarded to A level students on Thursday.
Mr Edwards adds that the match rate between teacher grade predictions and grades awarded by CCEA was 57.8%.
“CCEA is in a lucky position” he adds, as it has information from previous years of teacher predictions on student performance.
He says the focus has now turned to the appeals process as in other years, but says it is not a normal year.
Mr Edwards reads the CCEA helpline number: 028 90 261 260.
More information is also available on the CCEA website., external
Image source, NI AssemblyThe members discuss their reaction to the minister's briefing.
Chris Lyttle proposes that the committee's position to the minister "is that pupils are awarded their AS-level grade, their teacher-assessed grade or their CCEA-awarded grade, whichever is the highest".
The motion is passed with one vote against from Robin Newton, and an abstention from Maurice Bradley.
The members move to the next item of business, a briefing from Justin Edwards, the CEO of the awards body, CCEA.
Mr Edwards takes his seat in the alongside his colleague, Ms Margaret Farragher.
Daniel McCrossan of the SDLP comes in for a supplementary. He asks if the minister will commit to publishing the algorithm used to calculate results.
Mr Weir replies, “all I would say is probably anything given to algorithms will not be, if you like, 100% understandable as it will include a level of technical” information.
Mr McCrossan then asks the minister if he will being an independent review into CCEA processes.
Mr Weir says at this stage he doesn’t intend to have an independent review.
Mr McCrossan’s final question is, “do you believe you got this right?”
The minister replies: “Yes, but I don’t think anybody judging their own actions is obviously watertight in that regard.”
Image source, NI AssemblyThe committee thanks the minister for his time before moving to some general business.
Maurice Bradley, the minister’s DUP party colleague, says there seem to be a lot of students, “more than normal”, who are disgruntled with the results they have been awarded.
He says that some parents of children who moved schools, have been in contact with him to say their previous work at their former school has been ignored.
Mr Bradley asks the minister to review the appeals process.
Mr Weir replies that the grades are “overall up”.
He says in terms of appeals, they are being widened.
“CCEA have indicated, and I’m very supportive of this, that the change that will be made this year, is specifically to allow a challenge on the grounds that prior attainment has not been produced,” says the minister.
Image source, NI AssemblyChris Lyttle, the committee chair, says some universities have indicated that they will accept students on the basis of they grade they have been awarded, within the next 24 to 48 hours.
He asks the minister: “Can you guarantee that no pupil will lose an offer they have received from any UK university prior to that 7 September deadline”.
Mr Weir says the September date is one which has been asserted by UCAS.
“Can I guarantee that in every circumstance,” he says, “no I can’t”, but adds that any move on this would be against what has been iterated by UCAS.
Justin McNulty of the SDLP says the minister is “remarkably calm” given that “the future of pupils is in jeopardy here and their dreams are at stake”.
He adds that he has been “swamped” like his committee counterparts by pupils, teacher and parents who are “not understanding what the hell is going on here”.
Mr McNulty questions why the department asked teachers to provide predicted grades “when in essence now they feel like they have been totally ignored”.
The minister says in terms of awards, “teachers are asked for their professional assessment of grades every year” and says this year it has been used in the basis of ranking.
“As with everybody on this issue, I care deeply,” adds Mr Weir.
Image source, NI AssemblyMr McNulty asks the minister if the system used this year has been detrimental to boys in their grades.
The minister says "there is a gap in what girls achieved and what boys achieved" but adds that the pattern is similar to previous years.
Catherine Kelly of Sinn Féin (below) joins the meeting by video link.
She says many young people's lives have been "devastated by an algorithm" that has discriminated against high-achieving pupils from deprived areas.
Ms Kelly asks the minister how he can defend "a system that was flawed from the outset".
She wants to know why Mr Weir cannot show " the same leadership" shown by his Scottish counterpart.
"Do you have an understanding to the extent of the downgrading that took pace between non-selective schools compared to selective schools?" she asks.
Image source, NI AssemblyThe minister reiterates his earlier comment that "nobody has been downgraded".
"This is not a question of here's an award that is now being reduced," he says.
Regarding the situation with selective and non-selective schools, he says that "pupils at non-selective schools did better compared to their counterparts in grammar schools than they did last year or the year before. The level of attainment gap that is there in terms of A-levels and AS-levels closed this year to a greater extent than would have been in previous years".
"I think the Scottish decision was wrong," Mr Weir says.
UUP MLA Robbie Butler is up next.
He says “there seems to have been a shift in the model” used to calculate grades.
At the start of the week, he says, there appeared to be less emphasis put on the rank order than the data provided.
Mr Butler asks the minister what percentage weighting was given to each component that was considered in the formulation of results.
“There are winners and losers in exams” he adds but says the system being used must be fair.
Mr Weir says on the issue of the level of standardisation, the performance of the school will be a factor in terms of AS level.
He adds that AS levels are one of the key pieces of evidence that can be used in relation to A Level results.
“What we are doing is of a very similar nature to England, but actually we’re going slightly further than England,” says the minister, adding schools have been asked to provide any data they have gathered that can support predictions.
Image source, NI AssemblyChris Lyttle, the Alliance committee chair, intervenes and reiterates Mr Butler’s question about the percentage weighted against components used to calculate results.
Mr Weir says in terms of ranking “it’s not as straight forward as that as it’s not a percentage”.
He says there is a “statistical distribution applied to the rank order” and so it is “a different concept in that regard”.
The minister adds that “rank order is absolutely vital” to the process.
Mr Lyttle then asks the minister if he is aware of the GCSE results due to be released next week.
The minister replies that he does not know those results.
“We will get a level briefing on what the results will be relatively close to the time they are revealed,” says the minister.
Image source, NI AssemblyThe minister's DUP colleague, Robin Newton, asks whether it can be quantified if there where pupils who were assessed "more severely" by schools, who "actually got increases" in their grades.
Peter Weir says "it's a bit difficult to actually quantify as such".
He says it was not CCEA's job to interfere with the ranked lists provided by schools.
The minister says that either at school level, or at individual teacher level, "some would treat things more generously or more severely than others".
Asked about the how the Northern Ireland overall A-level results under this year's system compare with last year's, Mr Weir says there was "an increase of just under 2%".
He says the results for both selective and non-selective schools were ahead of England and Wales.
Deputy Chair of the Education Committee Karen Mullan says she doesn’t believe standardisation should not have been applied to results.
“Do you trust our teachers and their professional judgement?” she asks the minister.
“Our teachers do an outstanding job,” says Mr Weir, adding that it’s not an issue of trust.
He says the issue with teacher assessment is that there is often a “wide divergence between what they have produced as a teacher assessment and what has been the actual results of the examinations”.
“That as a methodology is one that does not produce an accurate result, or a fair result as you are dependent therefore, and particularly without any level of standardisation, then you are entirely dependent on how a teacher views it,” says the minister.
Image source, NI AssemblyFaustina Graham from the Department of Education replies to an intervention from the committee chair Chris Lyttle.
She says there was guidance provided by CCEA about evidence that could be used by teachers in predicting pupil grades.
She says it was a “more robust process this year than last year, that is absolutely true”.
“No matter what happens from school to school and across schools, you cannot stand over that what happens in one school is the same as what happens in another school,” she adds.
In an ideal scenario, she says there would have been external moderation of the grades but adds that was “impossible given our situation and the circumstances we found ourselves in”.
She says the standardisation process “is exactly the same thing” and says it’s designed to “ensure young people get a fair outcome” which she adds “does not take away from the young people we are hearing about today”.
Daniel McCrossan of the SDLP is up next to ask questions.
He says the committee must be mindful that they are talking about “children and young people, who for the last day I have seen in tears, whose dreams have been shattered, confidence battered and families are very worried and concerned for their wellbeing and their health, and mental health”.
“I have seen teachers in tears, in shock, they are numb, angry, frustrated and feel patronised,” he adds and says principals have been trying to explain the situation to parents and the public.
“Children have been failed by this system, they've been failed by the Department of Education, they have been failed by the processes of CCEA,” says the West Tyrone MLA.
Image source, NI AssemblyMr McCrossan questions the transparency of the algorithm used to calculate A Level results and asks the minister “to produce your homework to be marked”.
Mr Weir replies, “for individual children, yes, this has been incredibly stressful, it has been disappointing for some children, as also happens in other years as well”.
“There are more children today who have got a top grade, an A* to C grade than last year or the year before,” he adds.
Mr Weir says the member rightly raises the issue of appeals and confirms that “there will be a particular emphasis” to ensure those relating to university placing are worked through quickly.