Take it with a PISA salt?
I’ve recently been exploring the story behind the headlines when it comes to the PISA tests. It’s a story that needs telling as it places some pretty large caveats on any interpretation of the international league tables published by PISA. There are many PISA sceptics who cite such arguments as:
- The tests are only taken by, and percentages quoted about, students in school that day. It ignores the non-attenders who can account for significant proportions of society in many countries.
- Only approximately 10% of students take the full test. The rest take only part of the test and their answers to the questions they didn’t take are estimated using a Rasch model. Critics argue this is inaccurate. (http://www.oxydiane.net/IMG/pdf/Uncertainties_and_Bias_in_PISA.pdf)
- Shanghai is allowed to appear as a ‘country’. All other countries are not allowed to cite high-performing cities as ‘countries’.
To be fair, the PISA report does go into detail about all the uncertainties involved in their analyses. PISA are not intending to mislead. It is people ignoring the detail and taking the league tables at face value that is the issue. As this TES article points out, if the uncertainties involved mean the UK come anywhere from 14th to 30th, these are important and should be acknowledged before we all look to Shanghai for guidance on how to teach maths.
Before we go any further, I want you to imagine if this was what your day job looked like:
- 92% of your students read for pleasure every day.
- Education was highly regarded in society and considered in greater regard than economic success.
- The culture in society was one in which lack of intelligence is not considered a barrier to success. Hard work is considered the route to successful outcomes.
- Parental support is high, with them fully supporting the school policies, but also expecting additional tuition if their child is not making expected progress. They ensure their child studies at home and even set extra work themselves if the school has not set a sufficient amount. Relatives and friends place pressure on parents if they are not providing additional private tuition for their children.
- Students have a ‘remarkable’ work ethic in a highly pressurised and competitive environment.
- High performing schools are required to take 30% of their students from the poorest areas.
- Students state that the idea of not completing homework is simply unimaginable.
- Rather than 80%-90% contact time, as typical in the UK, you have 25%-30% contact time; typically 2 classes (of 40 or more). Planning, marking and preparation are all done in the school working day and not taken home. Non-contact time also includes providing 121 tutorials for pupils not making progress.
- Group planning by teachers is the norm.
- In the first 5 years of teaching, all teachers are expected to observe their colleagues in over 360 lessons. There is a minimum of 10 per year after that.
- Public or demonstration lessons are taught by ‘master teachers’. Being asked to deliver one of these is considered as reaching the pinnacle of your profession.
- Teachers carrying out action-based research is expected every year. They have to produce 2 papers annually of their results.
- Teachers in their first five years receive 240 hours of CPD (in addition to the observations).
- There is no grading system for lesson observations. They are purely developmental.
- 8 n.o. 40 min lessons per day, 2 of which are self-study, followed by further supervised silent study at school and 2-3 hours of homework each day.
- A maths lesson every day compulsory up to 18 years of age.
- A reduced quantity of content on the maths curriculum with the focus on depth rather than breadth.
- Mixed-ability classes.
- Problem-solving focus, ensuring pupils can answer complex, deep questions on topics before considering it as learned.
- An agreed national syllabus and textbooks allow teachers to focus on knowledge, skills and understanding rather than re-planning to meet new exam specs or regulatory criteria. There is a 10 year plan for education from the government meaning policies are given time to embed themselves. The specifications for maths haven’t changed since 2004.
What could you do with 25% contact time and only 2 classes?
This system is not fictional, but that of Shanghai. These findings come from a report by the National College for School Leadership: Report on research into maths and science teaching in the Shanghai region.
Although the Shanghai region is considered as a country by PISA, it is clear that we could learn from them. However, many of the differences are cultural and outside of our control as teachers at the coal face in the UK.
What do you think we can learn? Surely, it’s the whole package that’s getting the results. We can’t pick and choose. It works because of the culture, not any one single policy.
On the other hand, is the idea of an international comparison between education systems flawed? What if the tests were more like GCSE maths exams in the UK? Would the increased breadth of knowledge our students have place them higher in the league tables than their ‘depth rather than breadth’ competitors in The East?
PISA; take it with a PISA salt?
Approach with caution. The detail is important here.