Data comparability in elementary and secondary education is hard to achieve in Canada

 Charles Ungerleider, Professor Emeritus, The University of British Columbia

[permission to reproduce granted if authorship is acknowledged]  

I received an email from colleagues complaining about how difficult it is to obtain comparable data about elementary and secondary schooling in Canada. “That’s the truth,” was my frustrated and, I am sure, frustrating reply.  

The decentralized nature of Canada's education system makes gathering comparable data from each of the Canadian provinces and territories in the context of elementary, secondary, and, for that matter, post-secondary education challenging. Each province and territory in Canada has jurisdiction for its education systems. That autonomy leads to subtle, but important, differences that make inter-provincial comparisons difficult. Differences among provincial systems, policies, and curricula lead to variations, albeit often minor, in what is taught, how it is taught, standards for student achievement, and almost everything else. As a result, data collected from one region may not be directly comparable to another.  

Further complicating the matter is the fact that methods, standards, and timing of data collection vary across provinces and territories. This includes differences in the types of data collected, the tools, and technologies used for data gathering, and the metrics for measuring educational processes, performance, and outcomes.  

Those seeking to make comparisons will no doubt notice that the academic year, the age at which children enter, and the number of hours of instruction differ from province to province. These variations affect the comparability of data related to enrollment, progression, and attainment.  

For those who seek to make comparisons and are cognizant of the need to control the variations affected by system differences and the demographic differences among provinces, obtaining data is a challenge. The willingness of provincial and territorial governments to participate in national data collection initiatives varies, influenced by political, fiscal, and administrative priorities.  

Canada lacks a centralized data repository.  Without a centralized national database for educational data, collecting, standardizing, and comparing data from different regions is complex and resource intensive. Canada needs a coordinated approach and standardized guidelines for data collection and sharing across Canada's educational jurisdictions to facilitate meaningful comparative analysis. The Councils of Ministers of Education, Canada (CMEC) tries to address the need for comparable data.  

The Pan-Canadian Education Indicators Program (PCEIP) is a partnership between CMEC and the Canadian Education Statistics Council, a body jointly chaired Statistics Canada and CMEC. It tries to provide a set of statistical measures on education systems in Canada using in accordance with common data standards and definitions. When I served as deputy minister of education for BC, I co-chaired the Canadian Education Statistics Council with the Chief Statistician of Canada. I saw first hand how challenging it was to produce data that would enable comparisons across jurisdictions.  

Despite considerable effort, the available data about elementary and secondary schooling are meagre. In fact, as far as achievement data are concerned, PCEIP depends greatly upon the Pan-Canadian Assessment Program (PCAP). PCAP, conducted by the Council of Ministers of Education, Canada (CMEC), assesses the abilities of 13- and 16-year-old students in reading, mathematics, and science. PCEIP also depends on the OECD’s Programme for International Student Assessment (PISA) that examines the performance of 15-year-olds in mathematics, reading, and science.  

Those of us interested in cross-jurisdictional comparisons look to PCEIP, the Canadian National Household Survey, the Census, and the Labour Force Survey and try to assemble data that would enable meaningful interpretation. It isn’t easy.  

We also seek data from the provinces and territories, but transparency and availability of educational data vary significantly among Canadian provinces and territories. Some regions are known for being more open and forthcoming with their educational data. I wish all provinces were as open about data as British Columbia. More than a decade ago, BC became the first province to ‘publish’ its data under an open license.  

BC makes a data catalogue available to the public. I often seek data about student performance, enrollment statistics, and graduation rates from the data warehouse. There are data that I want that I cannot find there. Attendance data are something I am looking forward to seeing. But getting there means the province will need to establish common definitions and data collection procedures to ensure that the data provided by one school or district is comparable with another.  

Ensuring comparability in data is crucial for accurately understanding trends and their broader implications. This requires standardized methods for collecting data, uniform definitions across studies, consistent measurement units, and synchronized timeframes for data collection. These goals are not overly complex but achieving them demands significant political commitment and administrative effort. British Columbia has exemplified such commitment and effort. However, the relative consistency in the PCEIP reports over time suggests a limited commitment from provincial politicians, regardless of their party, to understanding educational trends and their implications.  

Reflections on online learning

 Charles Ungerleider, Professor Emeritus, The University of British Columbia

[permission to reproduce granted if authorship is acknowledged]  

Readers of my blog will know that I believe quality instruction makes a tangible difference in learning, and that well-educated and well-prepared teachers are essential. I have been thinking quite a bit about on-line learning, something that occupies much of my time.   

In the 1960s, around the time I was preparing as a teacher, John Carroll, an educational psychologist, proposed that the amount of time learners spend on learning a task, relative to the time required, is a critical factor in determining their success in learning.[1] One of the constraints of face-to-face schooling is that it imposes constraints on the amount of time learners have to achieve mastery.  

The advantage of online education aligns with Carroll's emphasis on learning time and individual differences in learning needs. In Carroll's model, the aptitude of a student, including their prior knowledge and learning abilities, significantly influences how they learn. Online learning facilitates this by allowing learners to tailor their learning to their individual aptitudes. For example, a student who finds calculus challenging can allocate more time to this subject, revisiting complex problems through videos and online forums until they grasp the concepts.  

Online learning provides learners with the flexibility to access educational resources at any time. Someone interested in history, for example, can spend more time investigating a topic of interest.  

Online learning requires perseverance. Self-motivated students can spend additional time mastering subjects of interest or difficulty. This is evident when a learner studying programming online spends extra hours developing an application. That learner is achieving mastery and exhibiting the willingness to invest time in learning.  

Quality of instruction is also a critical factor. Online learning offers access to varied instructional methods, materials, and instructional models (teachers). A learner who prefers demonstrations might explore YouTube while one who prefers print can look for articles and e-books.  

Carroll's model points to the importance of student understanding. Learners who find one video difficult to comprehend can look for another that is clearer or better aligned with the learner’s prior understanding. Self-discipline and motivation are two critical characteristics of successful online learners. The learner's willingness to engage with the material is crucial. Online learning offers flexibility and some level of individuation, but it demands a high level of self-regulation from students.  

Internet-based learning offers a flexible, somewhat individualized educational experience that can align with the needs, aptitudes, and time that different learners need to achieve mastery. However, it also requires enormous self-discipline and motivation. 

---

[1] Carroll, John B. (1963) "A Model of School Learning." Teachers College record (1970), vol. 64, pp. 1-9.

Should cell phones be banned in school?

 Charles Ungerleider, Professor Emeritus, The University of British Columbia

[permission to reproduce granted if authorship is acknowledged]  

Quebec’s new policy banning cellphones in classrooms has taken effect after the holiday break. The ban applies to all public elementary and secondary schools in Quebec but allows teachers to determine whether the students with whom they work may use phones for educational purposes. Enforcement of the ban falls to individual school boards. Some boards already have restrictions on cellphone use. Ontario's 2019 cellphone ban has been criticized for ineffective enforcement. Restrictions on cell phone use is consistent with what is occurring in countries like France, China, and the UK. Anecdotal evidence indicates there have been positive outcomes like reduced distractions and improved academic performance.  

Debate about student cell phone use in schools involves various claims about their impact on educational environments, student behavior, and teaching practices. Proponents of restrictions on cell phones argue that they are major distractions in classrooms. They argue that restricting their use can help students concentrate better on their studies.  

Limiting phone use, proponents of restrictions say, can help reduce instances of cyberbullying during school hours by reducing the opportunity that students have for engaging in harmful online behaviors. Some proponents say a ban can encourage more face-to-face interactions among students. This, they argue, will improve social skills, and reduce dependence on digital communication.  

A ban on cell phone use may reduce the risk of cheating on exams or assignments. And limiting cell phone use in schools may also contribute to healthier lifestyle habits among students.  

Those opposing restrictions say that cell phones are a crucial communication tool. A ban might hinder students' ability to communicate with parents or guardians in case of emergencies or changes in plans. Opponents of restrictions say that smartphones can be effective educational tools. Eliminating them from schools removes a resource that can enhance learning. Opponents also say that enforcing a ban consistently across all schools in a province can be difficult and determining and applying appropriate consequences for violations of the ban may be almost impossible.  

I asked myself what questions I would ask If I were contemplating a proposal to ban cell phone use in schools. Here are some that occur to me:

How does cell phone use or its absence affect student learning, attention in class, and overall academic performance?

What is the impact of cell phone use on student behavior, including instances of cyberbullying, social interactions, and development of social skills?

How would a ban on cell phone use affect students’ ability to communicate with family, especially in emergencies?

What does the evidence say about the effect of cell phone use on students’ mental health?

Would a ban on cell phones exacerbate or mitigate the digital divide among students?

What time is lost to instruction because of monitoring and disciplinary actions?

Would a ban on cell phone use have a material impact on students’ readiness for a technology-driven world?

Answers to such questions would provide useful information about whether a ban on cell phone use should be pursued.

The Devil is in the Details

 Charles Ungerleider, Professor Emeritus, The University of British Columbia

[permission to reproduce granted if authorship is acknowledged]  

We are wired to be resistant – and even blind – to ideas and policies advanced by people whose values and beliefs differ from our own. Our desire to maintain connection to those who share our values disposes us to resist the ideas of those whose values differ. We have strong attachments to our own ideas and values that make it difficult to evaluate policies and claims that others make – especially if their values are different.  

We resist change. Confirmation bias, selective perception, and cognitive dissonance are among the mechanisms at work. It takes effort to overcome these mechanisms and biases. I think it is worth making the effort.  

What brings this to mind is the recent passage of The Better Schools and Student Outcomes Act, 2023 (Bill 98) in Ontario. The proclaimed legislation has provisions that purport to enhance education in that province.  

The bill encompasses five key areas that would be supported by future regulatory and education policy reforms. The bill aims to increase accountability and transparency in Ontario's education sector by aligning provincial priorities and expectations with district and classroom-level implementation. Boards of education throughout Ontario will be required to fulfill government-established goals for student achievement and improve transparency and accountability regarding board performance and funding. It sounds like the provincial regulation in British Columbia under the banner of the Framework for Enhanced Student Learning.  

The second key element in the bill attempts to address effective governance by introducing standardized processes and expectations for trustees, Directors (Superintendents) of Education, and supervisory officers in school districts. This element seems designed to ensure equitable and unbiased trustee conduct across all boards and to equip trustees and Directors of Education with the necessary competencies to effectively address the government's student achievement objectives.   

In a third section, the Act seeks to optimize the use of the capital assets of school boards to expedite the construction of schools and enhance school capacity. This entails leveraging surplus property for public education and other provincial priorities, addressing accommodation needs in areas experiencing high growth, streamlining school planning and design processes, and promoting more efficient approvals procedures.  

A fourth dimension of the bill is aimed at fortifying teacher preparation and oversight by ensuring that preparation addresses the current demands of classrooms. This dimension also addresses student safety by establishing fair and effective disciplinary processes for teachers and registered early childhood educators.  

The aims of the fifth major element in the bill is less evident to me. It appears to want to foster greater uniformity in approaches to student learning and provide resources for parents to engage with their child's education.  

Like most jurisdictions the Education Act in Ontario empowers the Minister to make such regulations as are necessary for the administration of schools and school boards. While new legislation was probably not necessary, it seems that the Ontario Minister of Education is using the Bill to signal increased accountability. According to the Minister:

These new measures will focus on getting back to the basics of education: strengthening reading, writing and math, and other STEM disciplines.

We are following through on our commitment to parents through new measures that will better refocus school boards on academic achievement and the development of life and job skills.

These reforms include the new authority for our government to set binding priorities on school boards that focus on boosting student achievement focused on reading, writing and math.   

Given evidence of a steady decline in reading and mathematics performance in most provincial and national jurisdictions, I suspect that few would argue that increasing achievement in foundational areas is unnecessary. The devil, as the saying goes, is in the details. What, at first, seems promising in the Ontario Better Schools and Student Outcomes Act can wind up producing undesirable outcomes or unforeseen problems. The regulations that the bill enables will deserve scrutiny and the implementation careful evaluation.