The fallacies of a sampling approach to student assessment

 

Charles Ungerleider, Professor Emeritus, The University of British Columbia

[permission to reproduce granted if authorship is acknowledged]

My previous blog examined some of the claims made about large scale student assessment: that they prompt teachers to teach to the test; waste valuable time and resources; do not assess everything that is important; are stressful for students and teachers; do not take into account differences among students; and allow some individuals and organizations to make invidious comparisons among schools.

Another argument that opponents make about large scale student assessment is that they should be administered to a percentage of students (a sample) rather than all the students at a particular level (census approach). That, it is argued, would save resources, and prevent those who misuse the results from doing so. It is doubtful that a sample approach to large scale student assessment would achieve those desired outcomes. But there is a more serious problem: sampling will not work if one is really concerned about equity of outcomes for all students because it reduces the ability to identify factors that impede success for each and every student.

 A sampling approach (as opposed to a census approach) has other deficiencies. Sampling prevents examining the trajectories of students. In other words, with a sample, it is almost impossible to tell whether the students who performed poorly at grade 3 had improved by grade 6. Sampling doesn’t allow the system to see what has happened to transient students, a particularly vulnerable population that is easy to overlook. It also increases imprecision by increasing measurement error. If you want to know how students’ prior performance relates to their subsequent performance, you need to survey all students.

I can illustrate the benefits of a census approach from a study in British Columbia. In 2006, the Minister of Education described high school completion rates of students for whom English was a second language (ESL) to be better than any other group the Ministry assessed (Victoria, Parliamentary Debates, p. 3530). After a closer examination of the data, Bruce Garnett and I (Garnett and Ungerleider, 2008) confirmed the accuracy of the Minister’s statement, but found that the high achievement of the numerous Chinese speakers masked the fact that smaller subgroups of the ESL population were faring poorly in the school system. We found that the strong performance of Chinese speakers (the largest ESL group in the data set) pulled the aggregate ESL graduation rates upwards. The graduation rates of all groups except Chinese speakers were very low, generally below 60%. The worst outcomes were among ESL speakers of Spanish, Vietnamese and Filipino languages. Identifying the differential success of various ESL groups would not have been possible if the data set had been generated on a sample basis. You cannot systematically address problems that you cannot identify.

Another advantage of a census approach is that it permits analyses that can help to identify factors over which the system exerts influence that facilitate or impede educational progress of groups of learners (for example, First Nations, Metis, and Inuit students, second language learners, students with special needs, etc.).

Most important, if equity among students is a priority, samples simply do not work. Even with carefully drawn samples it is difficult to detect small sub-populations of students to support meaningful analyses. I can illustrate this with reference to British Columbia’s student population which, at the time of the calculations below, was about 40,848 students at grade 4. With a student population of that size, we could draw a sample of 1,481 student, a number sufficient to meet the requirements for sound statistical analyses of the grade level population.

However, if you wanted to break down results by school board or wanted to study the performance of sub-groups of students, the sample would not work. Here is an illustration of why it doesn’t. The illustration assumes that the assessment is intended to produce results that fall within a confidence interval of +/- 2.5% 95 times out of 100 at the school board level.

School Boards	Number of Students available for assessment in 2019/20	Number of students required for a sample with a confidence interval of 2.5 at a 95% level of confidence
Abbotsford	1524	765
Alberni	294	247
Arrow Lakes	34	33
Boundary	99	93
Bulkley Valley	144	132
Burnaby	1740	816
Campbell River	397	316
Cariboo-Chilcotin	305	255
Central Coast	26	26
Central Okanagan	1690	805
Chilliwack	1022	614
Coast Mountains	271	230
Comox Valley	631	448
Conseil Scolaire francophone	601	432
~~~~~~~~~~~~~~~~~~~~~~~~	Table truncated*	~~~~~~~~~~~~~
Richmond	1377	726
Rocky Mountain	288	243
Saanich	484	368
Sea to Sky	409	323
Sooke	868	555
Southeast Kootenay	466	358
Stikine	15	15
Sunshine Coast	235	204
Surrey	5459	1199
Vancouver	3493	1067
Vancouver Island North	94	89
Vancouver Island West	35	34
Vernon	610	437
West Vancouver	501	378
Grand Total	40,848	22,200
*FULL TABLE AVAILABLE UPON REQUEST

         

In very large boards such as the Surrey School Board, the number of students sampled (1199) would be a relatively small proportion of the grade 4 student population (5,459). In a smaller board such as Vancouver Island North, the sample required (89) would encompass almost all the 94 grade 4 students in the Board. In the Conseil Scolaire Francophone Board the sample required would be more than two-thirds (432) of the 601 grade 4 students.

Overall, if you wanted to break down results by school board or wanted to study the performance of sub-groups of students, you would need to sample more than half of the total number of students at grade 4.

It is inconsistent for those concerned about the role education plays in helping to achieve social justice to want to restrict large scale student assessments by arguing in favour of sampling students. Those determined to achieve social justice ought to want to shine a light on discrepancies, not obscure them. My hunch is that, when considering the evidence of the limitation posed by sampling, advocates for social justice will see the benefits of a census approach to large scale student assessment.

The Assessment Controversy – Part Two

 The overlooked benefits of large-scale student assessment

Charles Ungerleider, Professor Emeritus, The University of British Columbia

[permission to reproduce granted if authorship is acknowledged]

In last week’s blogpost, I expressed my dismay that advocates for social justice are often strongly opposed to large-scale assessment, one of the most powerful tools for demonstrating the value of schooling and drawing attention to places that would benefit from additional resources. I acknowledged that assessment information can be used to show that, despite effort and resources, there has not been visible evidence of improvement. But I am an optimist who believes that such information can help make a difference for all students and especially for those for whom school success has been elusive.

Opponents of large-scale student assessments have mobilized many arguments to make the case against them. They say large-scale student assessment prompts teachers to teach to the test; wastes valuable time and resources; doesn’t assess everything that is important; is stressful for students and teachers; doesn’t take into account differences among students; makes invidious comparisons among schools; etc.

Teaching to the test is a sin for some critics of large-scale assessments. I am uncertain how teaching to the test is a bad thing if the test assesses things that are regarded as important and they are assessed well . . . and no one has the test in advance. Poorly designed and developed large-scale assessments that measure things that are not socially valued will have a short shelf life.

Opponents of large-scale assessments often quote the amount of money that is devoted to such assessments, claiming that the money could be better spent. As far as I can determine, there have not been studies of the value-for-money of large-scale assessments and the information they provide. I would be keen to see such studies. In fact, I think we need more program evaluations that consider the effectiveness, efficiency, and economy of large-scale assessments and many of the other practices we employ in schools.

The argument about a “waste of time” is a variation on the waste of money argument. Something of value (time) is being used poorly when it is used for large scale assessment. Here too we need some careful study of time allocation and use in education so that we can compare the time devoted to assessment with other practices to which we devote time.

It is true that large-scale assessments do not assess everything that is important. We should file that claim in the “all or nothing” fallacy drawer. I would like to see more assessments rather than fewer. If what is measured counts, I would like to see well-designed and developed assessments in social studies, the arts, mental health, and other areas to which we devote our scarce time and resources.

I do not dispute that some large-scale assessments can cause stress for teachers and students. There is ample evidence from contexts in which assessments are “high stakes” that they are stressful and sometimes prompt cheating. But, in their absence, we would not know whether graduates possess the knowledge that the graduation diploma is supposed to signify.  

Large scale assessments do not consider differences among students at the time of administration, but they are crucial to producing equitable outcomes for students who may have entered school with differences that schools are supposed to eliminate. Without the information provided by large scale assessments, we cannot know if Charles who performed poorly at grade 3 has improved by grade 6, or whether a gap between Indigenous learners and their non-Indigenous peers has diminished over time. The analysis of the data from large scale student assessments allows the system to identify the things over which the system has control that can facilitate or impede student success. Such information is essential for student and system improvement.

As I said in a previous blog, some individuals and organizations use the information from large scale student assessments to make invidious comparisons among schools. But those who make such comparisons are unaware or willfully ignore (my hunch is the latter) that between school differences have a relatively small influence on student achievement when compared to within school differences. Eliminating large scale assessment because the results are sometimes misused is like eliminating the food guide because it is sometimes ignored.

A superficial consideration of large-scale student assessment magnifies some of its imperfections and minimizes its benefit. One of the things that a good education should enable is an appreciation of appraising costs and benefits dispassionately. Unfortunately, we are not there yet. 

The Assessment Controversy

 Charles Ungerleider, Professor Emeritus, The University of British Columbia

[permission to reproduce granted if authorship is acknowledged]

Assessment of learning is among the most controversial issues in education, and one of the most misunderstood. At its core, assessment is the planned process for collecting information to aid in decision-making for many different purposes.

Assessments are used to provide feedback to learners about how well they have mastered a body of knowledge. The information that assessments provide can help teachers to improve instruction for groups or individual students. Assessments are the basis for informing parents about the progress their children are making in meeting expectations at grade levels or in specific subjects.

The information provided by assessments communicates to external audiences whether a student has met standards for graduation or the award of a certificate of accomplishment. Post-secondary institutions and employers use the information provided by assessments for admission or for judging whether a person can perform the tasks required of someone in the position for which the person has applied.

Assessments allow policy makers to determine how well the system is achieving the result it is expected to achieve (ensuring levels of literacy or numeracy, for example). Information from assessments provides assurance to internal and external audiences that the resources and responsibilities assigned to the institution are being used effectively (and being fulfilled). Claims for financial resources depend in part on information provided by assessments. When responsibility for vulnerable populations is shared among institutions, assessment information helps those institutions to know if they are living up to their shared obligations (for example, the education of children and youth in care).

In the competition for scarce resources, assessment information can call attention to places where the development of policy, the allocation of resources, the refinement of practices, and the initiation of programs should occur (improving mental health, for example) or where improvement is needed (Indigenous education, anti-racism, etc.).

Institutions that rely on public resources (tax dollars) use assessment information to respond to the public’s interest in knowing whether to place confidence in those institutions. There are multiple dimensions to ensuring the public’s confidence. One is over-all system performance, another is the performance of one’s children and grand-children, and a third – related, of course, to the first two – is equity. Are the outcomes students achieve from the contribution of schooling influenced by who the students are or where they live, or do students thrive regardless of their identities and places of residence?

Opposition to large-scale assessment seems contrary to the aspirations of those who seek social justice and who advocate for additional resources for the support of vulnerable populations. Yet, that opposition does occur. Having the confidence of parents of children who are vulnerable seems crucial to making the case for more and better resources. Being able to demonstrate the differences those resources and efforts make for individual students who face challenges, and for their similarly challenged peers, strikes me as an essential ingredient in the arguments that social justice advocates employ.

I am not naïve. Assessment information was once used to justify sexism and racism. More recently it has been used to make invidious comparisons of one school to another based on dubious assumptions using questionable methodologies. I devoted an entire blog to the misuse of large scale assessment to compare schools.

I am an optimist, I strongly believe that assessment information can make a tangible difference to the education of all students and, especially, students who, because of circumstances beyond their control or the control of their parents, face the most challenges to succeed.

Teachers are Better Together

 

Charles Ungerleider, Professor Emeritus, The University of British Columbia

[permission to reproduce granted if authorship is acknowledged]

Last week’s blog tried to address some of the complexities affecting funding in public education. And, just to be clear, when talking about school funding, we are talking about personnel – teachers mostly. The complexities of preparing, recruiting, and deploying teachers are many. I won’t address all those issues. I want to talk about deployment, especially in the context of improving student outcomes.

A consistent argument in education is that those close to students are in the best position to know what and how to address the challenges that confront those students. The education system, for the most part, recognizes the truth of that argument. Ministries of education establish the broad framework within which the system operates. In consultation with teachers, parents and experts of various kinds, Ministries establish curricula, set standards and the means of measuring progress toward those standards, and provide resources to school boards. School Boards are locally elected to govern school districts and to hold their superintendents accountable for the management of schools. 

The professional staff – teachers and administrators – may exercise considerable professional latitude in how they organize for, and carry out, their instructional and other responsibilities. One hopes, of course, that they do that in accordance with accumulated professional knowledge and close attention to what they understand about the students for whom they are responsible. If they are on top of their game, teachers build their instructional program on what students know and can do now, and where the program they devise will take them.

Despite the impressive experience possessed by their peers, most teachers plan on their own. They will share knowledge generously with one another, but only on occasion will they plan a school’s instructional program together. That is too bad. That does not mean that individual effort is unimportant. Every teacher makes a difference. Good teaching is indispensable to student success. Improving student outcomes is too great a burden to place on teachers individually. But the absence of genuine collaboration reflects the design of timetables and teaching assignments (something that has been referred to as an “egg-carton” culture) and an unfortunate belief that teaching is more art than a skill.

Let me make clear that teachers do collaborate with one another episodically and unsystematically. However, the collaboration that has occurred over the past quarter century is constrained by a structure that limits collective discussion and decision-making.

Greater gains in student performance can be achieved by teachers working together at the school level. The collective knowledge and experience of the school’s staff is indispensable to organizing and planning to educate students beyond the boundaries of the current knowledge and experience.

Teachers know their students well. After students have been in school for more than a year, they are known by several teachers. As she progresses, the knowledge that those staff members have about her accumulates and can be seen from multiple perspectives. Teachers who feel stymied because their efforts with the student do not seem to be paying off can consult with peers about approaches they have found successful.

The conventional pattern is for students to be assigned to class groupings that remain permanent throughout the school year. The teacher assigned to that grouping is responsible for determining the instructional program. In intermediate grades and continuing through high school, both teachers who enroll a class (enrolling teachers) and teachers who provide specialized support (non-enrolling teachers) must work together to address learners’ needs that can only be understood by knowing how students learn and socialize throughout the school community.

Helping each student to progress is essential. Collectively optimizing the success of all students is even better. School staffs are increasingly attentive to using data to inform decisions. What is often missing is collective decision-making about the way the school implements instruction. Rarer still is collective decision-making about student placement. It is almost unheard of for a school staff to decide to organize (and reorganize) learning groups dynamically as student performance is monitored throughout the year.

Last year in British Columbia, the ratio of students to teachers was about 19 students for each teacher. That ratio improves when you add in the non-enrolling teachers (teachers of students with special needs, students for whom English is not their first language, teachers who run libraries, counsellors, etc). With the non-enrolling in the mix, the ratio drops to about 16.6 students to every teacher.

Picture a school to which enrolling and non-enrolling teachers were assigned consistent with the requirements of the teachers’ collective agreement. But, instead of the conventional determination of teaching assignments, the teachers and principal would consider the data about student performance and outcomes together and determine what course of action they would pursue as a school and how they would monitor performance. They would take collective responsibility for student success.

Instead of making assignments based on non-enrolling formulae, school districts would assign the same number of teachers generated by the formula to the schools but allow for collaborative professional judgement among school staffs to determine teacher placement. Such placements might be planned to differ during the day or over the semester or year. For example, in situations where it is determined that students would benefit from large-group explicit instruction, one teacher could take responsibility for fifty students, freeing up other teachers to work collaboratively to improve student opportunities and outcomes.

My prediction is that, if teachers worked well together instead of approaching the task as masters in their own classroom, students would be more successful, and the teachers’ work would seem less onerous and be more satisfying. Teachers would be better together than they would be working primarily on their own with limited interaction with their colleagues.

Money, Money, Money, Money

 

Charles Ungerleider, Professor Emeritus, The University of British Columbia

[permission to reproduce granted if authorship is acknowledged]

The most frequently discussed topic in education is . . . school funding. Well, maybe mathematics and readings, but, after them, it is definitely school funding. Why? Scarcity. There is never enough money. Have you ever heard anyone in education say, “you know, we should give this money back because we really don’t need it”? No, never.

Raising funds for schools from local property taxation was quite common for most of the history of public schools – and in many places still is. That practice creates vast inequalities between school districts. Communities that have a large tax base can raise money that communities without a decent tax base – or communities whose citizens send their children to private schools – cannot or will not raise. In the United States, there is a $23billion dollar gap between school districts serving predominantly Black students and those serving predominantly white students.

Inequalities such as these affect the quality of schooling. From maintaining the buildings and equipment to paying teachers, severe inequalities affect student outcomes. Those who can, flee districts where conditions are poor and where teachers are poorly paid. The less affluent – families that cannot afford to move and whose children lack the advantages that money can buy – are left behind. However, children from economically disadvantaged families attending schools in relatively more affluent communities fare better that students who live in homogeneously poor communities.

Although there are inequalities among school districts in Canada, the inequalities are much smaller than in the United States because most of the basic funding that school districts receive comes from provincial governments. Provincial governments use formulas to ensure equity in the distribution of funds. The funding for school districts that are more remote is adjusted to account for the additional cost of transportation. The funding for school districts in northern climates is adjusted to reflect that fact that heating schools is more expensive and lights are on longer because there are fewer hours of daylight during the school year. As I have written in another blog, the outcomes students achieve in Canadian schools are better in part because of greater equity in school funding.

No matter. Almost everyone wants more money, and almost everyone wants new formulas for the disbursement of government funding. But there is little agreement about what they want the money for. It is hard to make a case for additional funding when jurisdictions want the money for different purposes or simply want more money and the discretion to use it as they see fit.  And, by the way, if there isn’t going to be more money don’t bother changing the funding formula because changing the formula will just take money from one district and give it to another – creating what are perceived as ‘winners’ and ‘losers.’  

Governments are typically unwilling to allocate funds for specific purposes without accountability for the use of the money. Thus, when funds are allocated for a specific purpose they are designated (‘targeted’ or ‘earmarked’) for that purpose alone. School boards don’t like designated funding because it fetters their discretion and requires them to be accountable for the use of the funding.

The acceptance of designated funding can produce unanticipated consequences. Consider special needs funding. Many local school boards receive designated funding for students who manifest special needs. Where this is the case, the number of students with special needs sometimes increases over time. It isn’t clear whether, in their attempt to get more funding for student needs, school boards identify greater numbers of students whose conditions attract funding or whether the funding draws attention to students with conditions that, though recognized, went unaddressed prior to the board’s receiving additional funding.

Money matters. In education, there is a natural desire to want to do more for students. No one wants to do less. Claims for additional resources sometimes try to paint a picture of students whose needs are neglected - ‘students falling through the cracks.’ But I have not ever seen evidence of neglected students.

Claims for additional resources for education would be easier to justify if (a) there was agreement on where those funds were most needed, and (b) those seeking the additional funds could show that the funding they received in the past made a visible (measurable) difference for students.

Rethinking Education for Mass Unemployment

 Charles Ungerleider, Professor Emeritus, The University of British Columbia

[permission to reproduce granted if authorship is acknowledged]

 COVID-19 has caused business failures and high levels of unemployment. Many of the failed businesses and lost jobs will not return . . . ever. As deeply concerning as that is, Smart technology, artificial intelligence, robotics, and algorithms (STARA) may also produce permanent, unemployment of unprecedented levels. Social policies such as guaranteed annual wages may help to mitigate the economic impact of mass unemployment. I think it is time to look ahead and consider the part that education should play in preparing successive generations for their responsibilities as productive citizens.

For much of its history – especially in the post WWII period – education has been associated with preparation for employment. Even when a smaller proportion of the population attended secondary school, those who did were often people preparing for the professions and the clergy. In the post war period, the link between schooling and work became stronger for the growing number of students. “Stay in school and get a good job,” parents admonished their offspring. Mine certainly did.

It paid off for most of us who stayed in school through to graduation. The booming post-war economy was fueled by consumer-demand for goods. The growing population needed more teachers, doctors, nurses, police . . . almost everything. Generous benefits made it possible for soldiers to attend school, obtain training and post-secondary education.

Those who heeded the advice about staying in school benefitted in direct proportion to their educational tenure. High school graduates were more likely to find steady employment and earned higher wages than those who did not graduate. Those who acquired training and post-secondary education did better than the high school graduates. That relationship persists today . . . for those who are employed . . . though not as strongly as it once did.

STARA has begun to change that relationship for many people and the trend is projected to grow exponentially. Most of the high school graduates who once found employment in the burgeoning industrial sector have been replaced by robots. The same is true in the resource sector. Fallers, miners, and derrick-hands are fewer in number because of mechanization of the work.

The post war service industry exploded as consumer wages increased. High school students once earned income from part-time work in drive in restaurants, diners, and service stations. Today many of those service jobs are performed by early school leavers and seniors who do not get the benefits typically associated with full-time employment. Those positions will also decline as the fast-food sector automates. Robots can work 24/7 (except for downtime for maintenance) and do not get benefits.

Some argue that “. . . automation displaces workers who are doing highly automatable work and tasks, but it does not affect the total number of jobs in the economy because of offsetting effects. . . . It is important to keep in mind that even though technology can be a net job creator, it does not mean that the new jobs created will show up right away, be located in the same place or even pay the same as the ones that were lost. All it means is that the overall need for human work has not gone away” [my emphasis]. As Statistics Canada analysts Marc Frenette and Kristyn Frank point out “. . .  even the most carefully chosen statistical methods can fail to accurately predict the future.”  

The dislocation of employment that may arise from automation and the difficulty in predicting the future suggest that it is prudent to anticipate and plan for an education for a society characterized by increasing unemployment and, possibly mass unemployment. What might such an education entail?

Literacy will continue to be important. People will need to read and the ability to evaluate media messages and images (media literacy). Literature courses should increase in number. There should be greater emphasis on poetry, composition, especially for artistic expression and written argumentation, i.e., rhetoric

History, economics, government, philosophy, ethics, sociology, and law will take a more prominent part in education because citizenship will become more active. People will take a fuller part in the affairs of their communities and societies. Scientific and environmental knowledge will allow citizens to understand the impact of their decisions. Languages should flourish, too, because the world will become increasingly integrated.  

Elective courses offered at the margins of a school career today will become more important in an education that takes into account mass unemployment. Music, carpentry, photography, painting and sculpture, electronics, and filmmaking will flourish.

These areas of study look familiar, but their priority in the curriculum and their focus will have been transformed. The cultivation of the human mind and its capacity for understanding will replace the cultivation of marketable skills. To the extent that there is a desire for the development of marketable skills, those courses will become the elective courses at the margins of the school curriculum.

Of course, I am no more adept at predicting the future than anyone else. But there is consensus that there is simply not enough work to engage everyone today and that STARA will increase unemployment, eventually on a mass scale. It is prudent to anticipate today what education for such a future might entail.

Will limiting Donald Trump’s access to social media make addressing cyberbullying easier?

 Charles Ungerleider, Professor Emeritus, The University of British Columbia

[permission to reproduce granted if authorship is acknowledged]

It took a seditious incursion by an angry mob into the American Congress to limit Donald Trump’s four plus years of online deceit and cyberbullying. During that period, the world’s population was exposed to his use of social media to promote aggression toward almost everyone. His behaviour was egregious, providing license to behaviour for which children and youth are admonished by parents and teachers: name calling, demeaning characterizations, misogynistic rants, taunts, incitement of violence, and lies.

The last four years have shown that connectivity is a two-edged sword. Better and more rapid access to information on the one hand, increased susceptibility to manipulation and damage on the other. Online aggression – also called cyberbullying or cyberharassment – is a form of bullying that has serious negative consequences for the victim and for the perpetrator. According to a 2017 meta-analysis of factors predicting cyberbullying perpetration and victimization by Chen and her colleagues, those with greater access are more likely to become involved in online aggression.

Online aggression can lead to anxiety, stress, depression, poor self-esteem, negative school performance, a higher chance of dropping out, and suicide. Pediatricians, mental health workers, and social workers express concerns about the exposure that children and youth have to social media. It is common to find parents giving their smartphones to infants to distract or entertain them. Children and youth – like most of us - are rarely untethered from their phones.

Access to information and communication technology has given us enormous advantages and opportunities. Evident in the extreme from Trump’s behaviour, those same technologies can victimize. In a wired world, denying or restricting access are not realistic means of addressing the issue. In the case of children and youth, denying or restricting access encourages students to seek covert access. Children and youth are reluctant to disclose their victimization for fear that their access will be limited or cut off completely.

Technological solutions – apps that detect and prevent the transmission of negative language and harassment or apps that facilitate the reporting of harassment – are of limited usefulness. As is often the case, education is a more promising approach, but it too has limitations.

In 2019, my colleagues and I reviewed 35 programs designed to address cyberbullying among children (ages 6-11), youth (ages 12-17), and young adults (ages 18-25). The programs focused on: creating awareness of cyberbullying; developing knowledge about cyberbullying topics; equipping participants with skills to address cyberbullying (coping strategies, social skills, skills to form improved/respectful relationships, self-empowerment, and empathy); on creating safe, respectful, bullying-free school environments and policies; building teacher capacity to address cyberbullying and bullying issues; and increasing knowledge and awareness among parents.

We found that most programs have not been systematically studied. But we were able to formulate recommendations that ought to help educators think about selecting or designing programs to address cyberbullying. Not surprising, the first recommendation is that you need to know what you are trying to achieve and for whom. Are you trying to influence behaviour change among young children or youth? Develop empathy for others or specific skills for improved communication and better relationships with others? Is the program aimed at a specific segment of the population or the entire school? You get the idea.

Clarity of purpose is indispensable to the selection of the relevant practices and to ensuring that they are aligned with the goals. Programs aimed at increasing awareness of unhealthy behaviours differ from programs that seek to equip students with the skills they need to extricate themselves from situations in which those behaviours are present.

Some cyberbullying programs treat cyberbullying like ‘bullying with technology,’ but there are important differences between bullying and cyberbullying: the anonymity of perpetrators, the almost limitless opportunities for victimization, and long-lasting impact because the attacks are difficult to expunge. So, it is important to pay attention to the similarities and differences between cyberbullying and traditional bullying.

Making certain that the approach is age appropriate and sensitive to vulnerable populations (racialized students, LBGTQ2S, for example) is important. Equally important is developing the capacity of those who will implement the program. Most established programs go off the rails when applied to other contexts because those in the new context do not faithfully implement the program.

Trump makes me think of two significant challenges that cyberbullying programs must face. One is motivating behaviour change. The other is the example set by socially prominent models. Trump’s visibility and the position he occupied made his behaviour seem acceptable, and he has shown that he has no desire to change. I am guessing that it will be a bit easier to address cyberbullying when Trump is gone, but it still won’t be easy.

If you want brief descriptions of the 35 programs or a copy of the full  report that my colleagues and I prepared for Public Safety Canada, please send me an email at oneducationcanada@gmail.com

More effective online instruction during COID-19 and beyond

 

Charles Ungerleider, Professor Emeritus, The University of British Columbia

[permission to reproduce granted if authorship is acknowledged]

There is no doubt that the fall term was extremely challenging for everyone. Many of the teachers with whom I have talked or corresponded over the holidays are feeling guilty about the quality of their online instruction. They know that they have not been prepared for teaching online and believe that students are suffering as a result.

I have tried to explain to some of them that they know more about online instruction than they think they know. I’ve done this by asking them questions about their classroom teaching methods to get them to recognize and make use of what they already know. I have distilled those conversations into an interview format, calling upon what the teachers said in response to my questions. My questions are in bold with a composite of the teachers’ answers beneath edited for economy and clarity.

“What do you do in your classroom to reduce the challenges students face when they confront new material?

I check to see what students already know about the material. When I do that, I build upon what students know and can do and avoid asking them to engage with material for which they do not have the necessary prior knowledge or experience. I review with them what they know and can do.

I break the new material into small chunks or steps so that students are not asked to bite off more than they can chew. What I mean by that is that it is easier for me to check to see if they have mastered a small amount of learning before I move on.

“What do you do if there are students who are struggling to get to the next step?”

I build scaffolds and use cue cards or models to help them along. I create diagrams of the processes I am asking them to follow or a timeline or a chart. I use key terms or phrases as reminders of the ideas they are trying to wrestle with. I prepare lots of examples so that, if a student is struggling, I have a second or third or fourth example I can use. If I have checked for prior understanding, I can usually link the material back to something the student already knows or use an example within the student’s frame of reference.

If there is a process or procedure that I am asking the students to follow, I try to model it for them. As I am modelling the procedure, I talk about what I am thinking at each step. I try to phrase what I am thinking in terms that are familiar to the students. Linking to the student’s prior knowledge or experience reminds them of what they know and shows them they are being asked to make a small step up – not a big leap.

I try to build in frequent opportunities for practice for two reasons. First, it allows me to observe them working through the problems, checking their work for accuracy, and giving them hints or prompts that will help them to succeed at the task. It builds their confidence to see their own success. Second, if more than one or two students are struggling, I know that I haven’t been successful in communicating the ideas or the process that I expect them to learn. I need to go back and start that step again, but with a different approach, examples, etc.

“What do you say or do for the students who are on track to succeed or who have already succeeded in the task?”

I let them know that I recognize their success. But I try to avoid saying, “good for you.” Someone is always saying “good for you,” but that’s too vague. They need to know what specifically they have done well. I say things like “you were careful in following each step,” “you remembered to look for X,” or “you showed your work each step of the way.” If they are on track, but not quite there, I try to nudge them in the right direction. “Check to see if you have all of the steps in the right order,” “Where would I look to see X?” or “Here’s an example that might be helpful” are some of the things I say to give them a gentle push.

If they are working through a problem on paper or showing me their work on a white board, I sometimes say, “let’s mark each of the steps to see if they are all there.” Gently, but firmly providing little corrections and feedback goes a long way to building understanding and confidence. Asking students to use an assessment rubric that is familiar to them lets them check their own work. If they do that often enough, checking their own work may become habitual. That makes them more independent learners.

“What do you do when you are confident that students have achieved the next level?”

Even when they have achieved the next level, they need practice . . . and lots of it. I try to create opportunities for practice that allow them to represent what they know. “Draw a diagram or make a chart,” “Find examples that fit the model we’ve been working with,” “Teach [another student] or teach me how to do that.”

“Is there anything that you’ve said that you cannot do online?”

Not really. But when I teach online, I have tried to overpackage the experience and made a lot of other mistakes. I really haven’t checked for prior learning or broken things up as I would have in class. I’ve spent too much time talking at students rather than with them. The lessons are too long. Not enough examples. Too little practice. It’s hard to give feedback online. But, no, you are right. In the classroom I have done what I described. Now I need to put it into practice online.

Managing student behaviour, technology, and the learning environment is very difficult in an online situation. My hunch: Online teaching and its technologies threw teachers with sound teaching practices off their game. If they did what they do with their students – inventory what they know – the task might be a bit easier . . . but not easy.