Chapter 1: PISA, Politics and Planning a Trip

As I approached the guard sitting by the entrance barrier to the school, I realised I was biting the inside of my cheek. I became more aware of my walking, the way the sticky Shanghai summer made smart shoes uncomfortable, and I rehearsed my limited Mandarin phrases in my head. ‘Wǒ shì lǎoshī’ (I am a teacher). ‘Wǒ shì yīngguó rén’ (I am English person). ‘Wǒ kàn xuéxiào ma’ (I look school?)

I’d arrived first thing in the morning before the children turned up in their colourful tracksuits, not wanting to cause too much disturbance to the school day, but all the same I had anticipated the guard’s initial reaction – confusion, followed by a shrug. He waited, I waited; he expected me to leave, I stayed where I was; I smiled, he picked up the phone. My Mandarin was not good enough to understand what he said on his call, but I imagine it went something like: ‘There’s a strange British woman standing outside the gates asking if she can look at the school; could you send someone down who speaks English?’ He hung up the phone. ‘Xièxiè’ (thank you). He gave me a nod.

A small, trim lady in a floral dress hurried across the courtyard a few minutes later, her expression displaying a mixture of curiosity and nerves.

‘Hello!’ I said. ‘I am so sorry to disturb you; I’m sure you’re very busy.’

She smiled and gave a polite shake of the head, ‘How can I help you?’

I told her that I was a teacher from England, and that I was interested in the education in Shanghai because their students do so very well in the international tests. ‘I’d love to come and see your school and learn what you are doing, if it’s not too much trouble. Can I come back on another day?’

Just turning up at the front gate was a last resort when my opportunity to visit a school in that kind of neighbourhood fell through. I’d already spent a week teaching in a school in a very poor area of Shanghai, and a week teaching, interviewing and observing in an experimental school in a well-to-do area, so I was keen to visit a regular neighbourhood school in Shanghai’s dense suburbs, near the house of the teacher I was staying with. My plan was to get an understanding of the school system in China’s largest city by living with its teachers, chatting to its students and listening for the cultural subtleties that aren’t picked up by ‘big data’. I was here because Shanghai’s 15-year-olds had outperformed teenagers in every other education system in tests of reading, maths and science, and I wanted to know how.

I’m a teacher by trade. I taught for three years in a secondary comprehensive school in a deprived part of London, a school that catered for young people from some difficult backgrounds and, for not unrelated reasons, a school that didn’t ‘produce’ particularly good exam results. It was hard work; there were days when I didn’t have time to eat lunch, or even use the loo, because I was running around finding students to chase up missed homework, or photocopying the worksheet that I’d stayed up until 11.30 the night before making. I moaned to my family, but I didn’t really mind about this bit – I assumed at the time that it was an inevitable part of the job.

What got to me was that the hard work I was putting in wasn’t making much difference to the children in my care. Much of it – lengthy lesson plans, extensive marking and regular data entry – was required by the school management to help them meet external targets and pass high-stakes school inspections. What time and energy I had left didn’t seem enough to overcome the systemic disadvantages that many of my students faced. Students like Dana in my Year 10G4 science class. This means she was 15 at the time, in the fourth science set out of eight, and had been working towards a ‘vocational’ qualification since she was assigned to this class two years ago. Her course was entirely coursework-based, and because it was not as rigorous as the alternative exam-based qualification, it precluded her from taking science at the next level (and therefore from studying anything science-related at university).

At parents evening, I told Dana and her mother that on this vocational course, she was working at a C-grade equivalent. Mum’s eyes lit up. ‘That’s great!’ she said. ‘I knew Dana was good at science. She wants to be a science teacher.’ Dana smiled and agreed, ‘Yeah, I’ve arranged to spend some time at my auntie’s school so I can get more experience.’ By this stage in the English system, without the possibility of taking science at the next level, it was too late for Dana to achieve – or even work towards – that goal. But it needn’t have been. Had she been in a better school, had better teachers, better resources and better support from the beginning – in other words, had she been educated in a better system – she could have at least had a shot.

I wanted to understand how education systems could be run better – how they could support their students to get better outcomes and have better opportunities without running their staff into the ground – and I looked beyond our borders to find the answers. But how can you tell which countries are doing ‘better’? Are there any objective measures that compare educational outcomes in different countries? And are they educational outcomes that we ought to care about?

The Politics of PISA

‘Europe is Failing its Students’1

‘Worldwide Survey Finds US Students Are Not Keeping Up’2

‘PISA Tests: UK Stagnates as Shanghai Tops League Table’3

‘PISA Report Finds Australian Teenagers Education Worse Than 10 Years Ago’4

‘Norway is a Loser’5

‘OECD Study: Finnish Teenagers are Best Readers’6

‘Canadians Ace Science Test’7

Every three years, the papers are flooded with headlines like these. They refer to the results of an international test called PISA – the Programme for International Student Assessment – which covers reading, maths and science. Each country that chooses to participate enters a representative sample of 15-year-olds to sit the papers, and in 2000 when the programme began, 43 countries took part. In the following 15 years, as the tests became more famous, more and more countries have signed up: 71 participated in PISA 2015, making up approximately nine-tenths of the world’s economy. You might think this is an odd way to express the number of children taking an education test, but there’s a good reason for it; the organisation that runs these tests is an economic organisation – the Organisation for Economic Co-operation and Development (OECD).

Why do so many countries choose to take part in this testing programme? There are two answers to this question, one quite straightforward, the other more cynical. Firstly, the OECD suggests that the tests measure ‘to what extent students at the end of compulsory education can apply their knowledge to real-life situations and be equipped for full participation in society’. This gives governments information that complements the results of another international test – the Trends in International Mathematics and Science Study (TIMSS), which measures how well eighth grade students have learnt the maths and science curriculum in their countries, rather than whether they can apply it.8

The results of both studies are broken down by subject, by type of question and by student background, allowing governments who opt in to the testing programmes to see where their education system’s strengths and weaknesses are, and to make those areas the target of education reform, capacity building or additional financial support – in an ideal world. Additionally, because PISA is carefully designed to measure students’ ability to apply and use knowledge, rather than just memorising it and reproducing it, participation in the programme can be a useful way of tracking the extent to which the education system is doing this successfully. This was one reason that China chose to enter students from Shanghai into PISA in 2009, and from several other Chinese cities in 2012 and 2015.

What is the more cynical reason that countries choose to take part in international tests? Well, it stems right from the inception of the PISA programme. An unusual coalition of US Republicans and French Socialists first set the ball rolling for PISA’s design and implementation. Ronald Reagan, reeling from the dispiriting findings of the 1983 ‘Nation at Risk’ report on American education (the clue to its conclusion is in the name), wanted to implement national reform. However, he was met with resistance by state-level governments who considered education to be solely within their own mandate. He was therefore looking for a way to make education policy an international issue, so that he could bring it under presidential control.

Across the Atlantic, the French Minister of Education Jean-Pierre Chevènement sought to demonstrate the failings of what he considered to be an elitist French education system in order to justify his own educational reform. What both men needed was an international education survey that would allow for comparisons of educational outcomes between countries, and they looked to the OECD to provide it. It took a while to develop (as attempting to accurately assess problem-solving ability in three subjects across many different cultures is an enormously ambitious undertaking), but in 2001 the first PISA test results were published, and they were used as an excuse for reform by governments around the world. In Norway, the Secretary of State at the time of these early PISA results described using Norway’s poor performance as a ‘flying start’ to implement their reforms.9 In America, the PISA results were used as a key justification for a federal school accountability programme known as ‘Race to the Top’. And in New Zealand, the use of OECD data in defending controversial reform has been described by some educators as an ‘OECD hangover’.10

Using PISA results as an impetus for reform is, in itself, no bad thing. The German people went through ‘PISA shock’ in 2001 when they realised that what they had thought was a world-leading education system was in fact below average in reading, maths and science, and one of the most inequitable in the OECD. It provoked educational discussion, soul searching, and a TV series, and led to various evidence-based reforms across its many states that brought about an improvement in their education system and their PISA standing over the next 10 years.

Of course, one of the major selling points of the PISA tests is that it allows us to identify success in a particular area of education, and learn lessons from other systems that appear to be doing this better. Certainly that is what politicians say they’re doing. But this is sadly not always the case. Politicians (along with the rest of us) have been known to cherry-pick evidence; to choose to mention only the data or the features of top-performing systems that back up their pre-existing ideas, and ignore the evidence that throws doubt on their proposed reforms.11 It is therefore particularly important that the wider public know a little more about what these successful systems are doing, and what the analysis that’s been conducted on the data suggests, so that we can hold our politicians to account before they conduct expensive and potentially ineffective reforms.

My Motivation and Approach

There is information about all of these matters out there already, but not enough people are reading it. It’s in the form of country reports and cross-country analysis by the OECD (who produce new publications that I want to read at a rate I can't keep up with), and additional reviews, analysis and commentary by consultants, academics and journalists.12 This book includes some of this analysis, and its conclusions are consistent with the conclusions drawn by the OECD – but they are not driven by it. While the analyses by various consultants and academics are immensely valuable and carried out by people far more talented than myself, they offer only half of the picture. They include the effect sizes, but leave out the people. They can tell you what correlates with what, but not how each thing interacts with a culture. You can learn about the ‘what’, but not the ‘why’ or
the ‘how’.13

As I read about ‘top-performing’ countries and their education systems from rainy England, I craved a more holistic, visceral understanding of them. ‘What does that look like in the classroom?’ I wondered. ‘Do the Singaporean parents think they have a good education system, or is the grass always greener?’ ‘What does it mean for the children in Finland that they don’t start school until aged seven, and would I want my own children to go to school there?’ I could read all about the governmental approaches of various nations in these OECD reports, and about the effects of specific policies in the academic literature, but I couldn’t find out how education worked as a whole in each particular country context without seeing it for myself; I couldn’t join the dots to see the bigger picture.

At the time I was asking myself these questions, I was just finishing my masters and had no permanent job, no mortgage, and no children; nothing to keep me in the country. So I decided to go on what has since been described by a friend as my ‘geeky gap year’. I started drafting emails to teachers in top-performing systems abroad whose addresses I’d found on the internet, asking if I could come and help out in their schools for a bit, and asking whether I could stay with them for a few weeks too (no wonder my mum wasn’t a fan of this plan at the time). I then realised that no one would say yes in case I was a lunatic, so I set up my iPad and made a little video introducing myself to would-be hosts, attempting to appear as sane as possible.

Of the top 10 PISA performers at the time, I chose Shanghai and Singapore because they got phenomenal scores, Japan because it was a large country rather than a small city-state, Finland because it has until recently been one of the only Western countries to outperform the East Asians and Canada because they performed well despite being culturally and geographically diverse. I sent out emails to teachers in Finland, the first country on my intended itinerary, and waited. I was half thinking that no one would reply and that I’d have to revert to my backup plan of sitting in coffee shops in Helsinki with a big sign saying ‘I’d like to hear your views on education, come and talk to me!’ when, much to my delight and amazement, a teacher called Kristiina emailed me back – we’ll meet her in the next chapter. I found educators all over the world to be remarkably generous and open-minded in welcoming me into their schools, and I owe them a huge debt of gratitude for their essential role in the making of this book.

My approach in each of the countries I visited was to stay with educators to get a deeper understanding of their lifestyle and culture; to visit schools through informal means to avoid being directed towards ‘sparkly’ schools that were unrepresentative of the rest of the system; and to stay in one of the schools in each place for at least a week until the staff got to know me and felt relaxed in my company.14 I spent about four weeks in each country and three of those inside schools; sometimes teaching, sometimes helping out, always asking questions. Some of my interviews were formal and recorded, others were informal chats on the subway on the way home, or over a bowl of noodles. I talked to enthusiastic school caretakers, distinguished school leaders, frantic high-school students and despairing parents. I learnt a huge amount about different approaches to (and ideas behind) education this way, and I will share some of this with you in this book as I take you on a tour of education in some of the world’s ‘top-performing’ systems.

What Matters? My Outlook

When I first started giving talks about my work at conferences, a common question at the end of my presentation was the kind of question that is actually a comment: ‘Why should we care about PISA? There is more to education than test results.’ This is an excellent point; if we don’t care about PISA, then we certainly shouldn’t care what countries that do well in PISA are doing. It is an excellent and oft-made point, because there is a perception that the furore around PISA results has caused governments to focus on a very narrow measure of education, to the detriment of other important educational goals, such as developing children’s knowledge of the arts, their appreciation of citizenship and their personal and social attributes. I don’t doubt that this is happening in England and some parts of America, and I don’t defend it; it is an example of governments giving PISA rankings too much weight, rather than using the broader data as helpful information to inform the direction of their education systems. Nevertheless, I don’t think this makes PISA (or TIMSS) irrelevant.

Just as most people would agree that there is more to education than reading, maths and science, most people would agree that this broader education ought to include reading, maths and science. One of the questions I set off to research was whether top-performing PISA systems only did so well in these subjects at the expense of everything else, or whether there are ways of bringing about improved understanding and ability in maths, reading and science without hugely increasing the time spent on them. It is important that young people reach at least a basic level in these subjects to prepare them for their lives after school. And at the moment, in the UK, we are not preparing all of them sufficiently: 17 per cent of British 15–16-year-olds who took the PISA test in 2012 did not attain the baseline proficiency level in reading (Level 2), which means the OECD consider them to ‘lack the essential skills needed to participate effectively and productively in society’. In mathematics, the number failing to reach the baseline level was 22 per cent, which is more than one in five.

Box 1: What does Level 2 actually mean?

Failing to reach Level 2 means not being able to answer questions like this:

Level 2 PISA maths question:

On one trip, Helen rode 4 km in the first 10 minutes and then 2 km in the next 5 minutes.

Which one of the following statements is correct?

A. Helen’s average speed was greater in the first 10 minutes than in the next 5 minutes.

B. Helen’s average speed was the same in the first 10 minutes and in the next 5 minutes.

C. Helen’s average speed was less in the first 10 minutes than in the next 5 minutes.

D. It is not possible to tell anything about Helen’s average speed from the information given.

In reading, almost one in five 15–16-year-olds are working at the following level, or below:

Level 1a PISA reading question:

BRUSHING YOUR TEETH

Do our teeth become cleaner and cleaner the longer and harder we brush them?

British researchers say no. They have actually tried out many different alternatives, and ended up with the perfect way to brush your teeth. A two-minute brush, without brushing too hard, gives the best result. If you brush hard, you harm your tooth enamel and your gums without loosening food remnants or plaque.

Bente Hansen, an expert on tooth brushing, says that it is a good idea to hold the toothbrush the way you hold a pen. ‘Start in one corner and brush your way along the whole row,’ she says. ‘Don’t forget your tongue either! It can actually contain loads of bacteria that may cause bad breath.’

What is this article about?

A. The best way to brush your teeth.

B. The best kind of toothbrush to use.

C. The importance of good teeth.

D. The way different people brush their teeth.

As you can see from the questions included here, the baseline proficiency level is pretty basic, especially for someone who has been in school for 11 years and is close to the end of their compulsory education. The OECD hasn’t plucked this level out of thin air; it is based on studies of young people in Australia, Canada, Denmark and Switzerland, which followed them over a number of years and found that those who performed below this baseline level (Level 2) often faced severe disadvantages in their transition into higher education and the labour force.15 This is not only a problem for the economy. Whatever you want to do with your life – becoming a pro skateboarder, opening your own restaurant or starting a family – lacking these basic skills makes your life more difficult and reduces the opportunities available to you.

The proportion of students that achieve at least these basic levels in any system is therefore one outcome measured as part of PISA which I believe to be important – but that isn’t all we’ll look at. Another key consideration is what proportion of young people achieve the highest PISA levels – it would be no good having a country where everyone knew the basics but no one was brilliant at solving complicated problems. This in measured by another OECD metric: the proportion of young people who attain Levels 5 and 6. The most famous measures are, of course, the average scores that give us the headlines and act as a simple figure to summarise each country’s performance.

But we are not quite done. How would you feel if every child that scored above a Level 6 was from a wealthy background, and every child that missed the baseline was living on or below the poverty line? That wouldn’t be quite fair, would it? While much of the cause of the worldwide association between background and results is due to students’ home lives, education systems can make this worse or they can make this better (though they needn’t be held responsible for fixing it completely). This measure – the impact of student background on outcomes – is called ‘equity’. Along with the proportion of students who attain at least a L2, the proportion who attain L5 and L6 and the average measures, we have quite enough to be getting on with. Where relevant, though, I will also bring in things which are harder to measure such as children’s mental health and reported happiness as we journey through our five countries.

Caring about particular outcomes does not automatically reveal the way in which it’s best to pursue them. There’s a key distinction between your values or preferred goals, and ‘what works’ to bring these desired outcomes about. The former should be shouted from the rooftops by anyone with a stake in education – parents, teachers, politicians and business people. Their values and goals are valid and important in any democratic country – but how should those outcomes be brought about? That is a question that has to be answered by looking to the world; looking at the evidence. By looking at the numbers, the associations and the analysis, but also by watching the children, listening to the teachers, looking for the big ideas and connecting all of these to work towards a more complete picture of how it might all fit together.

In this book, I will share with you a part of that bigger picture. I will join up some of the dots to show you what education looks like in five of the world’s most celebrated education systems, and to give some clues as to why they are so successful. You’ll learn a bit about their history, elements of their culture, how they deal with the challenge of educating children of different abilities, the various ways in which governments make teaching attractive and how parenting styles and attitudes affect their children’s results. You’ll also learn how psychology can help explain what underlies the success of their approaches. Come with me to Finland.