Educational Myth-busting: School Rankings

Educational Myth-busting: School Rankings

Charles Ungerleider, Professor Emeritus, The University of British Columbia

[Permission to reproduce granted if authorship is acknowledged]

It isn’t often that I have the luxury of watching television, but, when I see Discovery Channel’s MythBusters I smile. The hosts use their knowledge of science to entertainingly debunk myths. I’ve often thought that, if the program did some education myth-busting, it would be doing a public service.  The problem with my idea is that busting educational myths depends on logic and evidence, topics that don’t have much visual appeal.

One persistent myth about Canadian schools that deserves to be busted is that you can tell a lot about a school from its rank among other schools. In fact, school rankings are, at best, misleading and, at worst, dishonest.

The methodology used to produce school rankings compares each school with every other school being ranked. Those comparisons are mathematically adjusted to distribute the results (school scores) so that half of the schools fall below the average school score and the other half of the schools above the average.  When I point that out to parents, someone correctly points out, “That means that, if all schools improved by 50%, there would still be half of the schools below the average score.” 

People who produce school rankings based on student achievement conveniently ignore some of the other flaws in the logic. For example, they ignore that student achievement is a product of all the prior in-school and out-of-school experiences that a student has had up to the point when achievement is assessed. One implication of a highly mobile society is that holding a school accountable for the achievement of the students currently enrolled in that school probably places too much weight on the school’s influence and understates other influences. 

What we know from the lengthy history of studying student achievement is that, while schools – and especially the quality of instruction that students receive in schools – matter, factors outside of school matter more! Parental influence is one of the factors affecting how and how well students achieve in school. The amount and quality of interaction between parents and their children makes a difference. The value that parents ascribe to school and the respect they have for teachers affect how their children view their school experience and their teachers. 

Family income affects student achievement in several ways. Families living in impoverished circumstances have children who are less healthy. Those children are typically less likely to have seen a dentist or a doctor than their more advantaged peers. One reason is that parents struggling to make ends meet often work multiple low-paying jobs. If they take time from work, they lose pay. If employers think they are taking too much unpaid time off, they risk being replaced. Under such circumstances, the necessity of a visit to the dentist or physician can become a luxury that they cannot afford. 

On the other hand, more advantaged parents have time and resources they can invest in their children. They can ensure that their children receive regular dental and medical checkups. They can afford more nutritious food. They can attend school meetings and meetings with their children’s teachers. In addition, they can afford experiences such as after school activities and summer camps that are beyond the reach of their less advantaged peers. All other things being equal, children whose parents are better off reap educational advantages and benefits. 

Children who live in more challenging circumstances sometimes struggle with school work. If they are in classes where the students come from varied backgrounds, they benefit from having peers who are not struggling – especially if the more advantaged peers are in the majority. If children live in impoverished communities and attend school with children who also live in impoverished circumstances, they are deprived of the positive peer influences of those youngsters who live in advantaged circumstances.

Although the truth behind school rankings is not as visually appealing as investigating the cause of an explosion or a failed missile launch, it should make everyone – especially parents – think twice about the conclusions they draw from school rankings.   

Enjoy your Spring break (blog will resume in a few weeks)

Teaching: Complexity and Making a Visible Difference

Teaching: Complexity and Making a Visible Difference

Charles Ungerleider, Professor Emeritus, The University of British Columbia

[Permission to reproduce granted in authorship is acknowledged]

Teaching is complex work. It involves making an enormous number of decisions each day. All but the most trivial decisions require consideration of factors that interact with one another in seemingly countless ways.

Many decisions arise unexpectedly: decisions about responding to conflict among students, unexpected interruptions, unpredictable misbehavior, etc. Many decisions can be anticipated: decisions about lesson content, sequencing, selection of illustrations and examples, questions, etc.

On its own, instruction is massively complex--too complex to rely on experience alone. Discerning what works and what does not by examining one’s experience would be like looking for a blade of grass in a corn field. Carefully planned and executed studies are helpful guidance but relying on a single study would be foolish. It would be ideal if the carefully planned and executed study was repeated many times under the same conditions. There are unfortunately too few carefully planned and executed studies that have been replicated multiple times.

Instructional decision-making is too important to be left to the imperfections of our own experience or even to a single study however well it might have been performed.  Although imperfect, looking across studies of the same topic can provide more guidance. Over the course of the last 30 years or so, meta-analytical work has provided needed guidance for policy and practice, despite the difficulty of distinguishing the signal from the noise in the complex classroom environment.

John Hattie points out that any number of decisions a teacher might make can lead to some observed change in student performance because in education “almost everything works.” He recommends that, to minimize the risk of adopting policies and practices that may only make a marginal difference, teachers adopt practices that have a visible impact.

In his books, especially in his 2009 book Visible Learning, Hattie reports the results of his meta-analysis of studies that purport to address the same phenomenon and identifies practices that have a visible impact on student achievement.  All the practices Hattie enumerates that make a visible difference in student learning are, as one might expect, associated with how teachers teach rather than the conditions affecting their work.  That doesn’t mean that working conditions are unimportant, but it suggests that the preparation, support and professional development of teachers are where most of the attention should be directed.

Hattie identifies way-points to educational excellence that are specifically focused on teachers and teaching. They are:

1. Teachers are among the most powerful influences in learning.

2. Teachers need to be directive, influential, caring, and actively engaged in the passion of teaching and learning.

3. Teachers need to be aware of what each and every student is thinking and knowing, to construct meaning and meaningful experiences in light of this knowledge and have proficient knowledge and understanding of their content to provide meaningful and appropriate feedback such that each student moves progressively through the curriculum levels.

4. Teachers need to know the learning intentions and success criteria of their lessons, know how well they are attaining these criteria for all students, and know where to go next in light of the gap between students’ current knowledge and understanding and the success criteria of: “Where are you going?”, “How are you going?”, and “Where to next?”.

5. Teachers need to move from the single idea to multiple ideas, and to relate and then extend these ideas such that learners construct and reconstruct knowledge and ideas. It is not the knowledge or ideas, but the learner’s construction of this knowledge and these ideas that is critical.

6. School leaders and teachers need to create school, staff-room, and classroom environments where error is welcomed as a learning opportunity, where discarding incorrect knowledge and understandings is welcomed, and where participants can feel safe to learn, re-learn, and explore knowledge and understanding. (Hattie, 2009, P. 238-239)

Because teachers are the greatest influence on student achievement, governments that want to improve outcomes should make their preparation, support and professional development a priority.

__________________ 

Hattie, John. (2009). Visible Learning. Abingdon, Oxon: Routledge.