Wales Pisa results: 'Little will be learned'

As Wales waits to find out the results of the latest Pisa international comparisons of 15 year olds, Dylan Wiliam, Emeritus Professor of Educational Assessment at University College London, argues that some interpretations of the results are likely to be "useless" and that we "already know" what is needed to improve education in Wales.

On 6 December, the Organisation for Economic Co-operation and Development (OECD) will release the latest batch of results from the Programme for International Student Assessment or Pisa.

Over half a million 15-year-olds in more than 70 jurisdictions all over the world have completed Pisa tests, and education experts will be making confident pronouncements about what the results mean.

Specifically, they will be offering advice about what countries need to do to in order to achieve better results next time round.

Conflicting views

We will be told that the most successful countries group students by ability (Singapore) and that they do not (Finland).

We will be told that the key to a high-performing education system is that teachers are drawn from the highest-achieving college students (Singapore and Finland) but those commentators will conveniently forget to look at the Republic of Ireland, where the entry requirements are as demanding as those in Singapore and Finland, but where the results are much less impressive.

Some will claim that giving teachers autonomy is the key (The Netherlands) while others will point to the importance of central control (Singapore).

Those who like high-stakes testing will claim that Canada, and specifically Alberta, is the system to be emulated, while those who oppose such standardised assessment will identify with Finland.

In the last set of Pisa results, published in 2013, Finland came sixth in reading and twelfth in maths

Such arguments make for interesting discussions, but there are at least three reasons why such speculations are likely to be useless, if not actually misleading.

First, the results we see are, in most countries, the result of 10 or more years of education, and so what countries are doing now is almost irrelevant.

We need to look at what was happening in schools when the students who were tested in 2015 began their education.

And what was happening in schools 10 years ago is likely to be the result of policy measures implemented 20 years ago.

Second, the results themselves are hard to interpret. All countries try to ensure that the students that participate in Pisa are a representative sample of all 15-year-olds in the system.

However, in some jurisdictions this is difficult to achieve. For example, parents who move to Shanghai for work often leave their children with grandparents in more rural areas. As a result, there are doubts about how representative the Shanghai sample can be.

What happens after school?

Even where the results are robust, it is impossible to be sure that they can be attributed to the quality of education students receive in schools.

In many Pacific Rim countries, attending additional private tuition classes after school is the norm rather than the exception.

Getting robust data on the prevalence and effectiveness of these additional classes is difficult, but one estimate found that but for the effects of private tuition, the Pisa scores for the Republic of Korea would be below the international average.

And my colleague John Jerrim at UCL's Institute of Education found that Australian children of Chinese heritage actually outperformed Shanghai. The "Shanghai effect" may be as much due to "Tiger mothers" as it is to the quality of the education system.

Third, and perhaps most important, educational systems have to be understood in the context of the societies in which they operate.

Solutions that work in countries where there is faith in the wisdom of appointed civil servants are unlikely to work as well in systems where the chief officer of a local education authority is elected by the local citizens every four years.

The truth is, despite all the heat that will be generated by discussions of the Pisa results, we will learn little.

Whether this year's results are better or worse than those from 2012, whether Wales performs as well as our industrial competitors, doesn't really matter.

We need to improve education in Wales because of the profound changes that are taking place in society and work.

Our world is becoming more and more complex, and so higher and higher levels of educational achievement will be needed to be in control of one's own life, to understand one's culture, to participate meaningfully in democracy, and to find fulfilling work.

Like many others, I will be looking at the Pisa results when they come out on 6 December.

But I won't be trying to figure out how to improve student achievement in Wales, because we already know what we need to do.

We need to create a culture in which every teacher in Wales accepts the need to improve - not because they are not good enough but because they can be even better - and when teachers do their jobs better, their students are healthier, live longer and contribute more to society.

There is no limit to what we can achieve if we support our teachers in the right way.