Post-pandemic belt-tightening coming to a school district near you

 

Charles Ungerleider, Professor Emeritus, The University of British Columbia

[permission to reproduce granted if authorship is acknowledged]

The roll-out of emergency funding in response to the pandemic was accomplished more quickly in Canada than in the US because of our Westminster-stye of government. But provincial governments are beginning to issue warnings that the COVID emergency funding will not continue indefinitely. Ontario’s Deputy Minister of Education wrote a cautionary letter to Directors of Education (Superintendents in other jurisdictions) in school boards throughout the province with the subject line “2021-2022 School Year.” She wrote about “the extraordinary steps that school boards and their staff have taken to safely support the learning journey for Ontario students in what continues to be unprecedented times.”

She points out that the actions of boards and their staff were made possible in part because the Government of Ontario disbursed more that $CDN 1.6 billion dollars to support the safe reopening of schools. The letter enumerates the hiring of 7,000 “one-time additional staff” in every group of personnel from principals to custodians, the latter indispensable to ensuring the health and safety in schools.

Having recounted the Provincial Government’s assistance, the letter turns to its main purpose. She reminds Directors that the boards they lead reported to the Ministry that there had been an enrollment decline of approximately 40,000 students attributed to COVID. The letter recaps that the Ontario Government helped to mitigate the impact of the decline by providing $400 million in “one-time stabilization funding” that enabled boards “to maintain teaching and education worker positions and a high standard of programming.”

The fourth paragraph advises that in planning for the coming school years “school boards should take a cautious approach in their planning given the uncertainty in enrolment and adjust accordingly for the probable loss of one-time funding that was provided for  2020-2021” The letter was prompted because labour agreements require staffing decisions to be made this Spring for the Fall. That school boards will layoff “more than the typical number” of staff is understandable.

I recount this announcement because similar letters will likely be issued across the country regardless of the political stripe of the governments. Provinces and territories do not have the revenue to maintain emergency and stabilization funding. And, although low interest rates make borrowing less expensive, such borrowing would be politically risky.

The good news is school boards that have strategic educational plans and regularly evaluate their program will be better able to adjust to the withdrawal of emergency and stabilization funding. Having a plan allows boards to prioritize the allocation of scarce dollars. Regular evaluation enables them to know which programs are effective, efficient, and economical. That knowledge enables them to know which programs might be eliminated in times when funds are limited.

The bad news is that most schools boards do not have strategic educational plans that provide a reference point and help boards to prioritize spending. Most school boards do not evaluate programs, leaving decisions about program reductions subject to anecdotal information about their effectiveness and efficiency.

Boards facing immediate decisions about the allocation of scarce resources do not have the time to develop strategic education plans and evaluation frameworks to help them make the difficult decisions they are facing. But it is not too late for boards to develop plans and evaluation frameworks that will guide their long-term planning and their responses to exigencies like the ones they are currently facing. Resources are always scarce. Even when they increase, there is never enough funding for everything one would like to do for students. It is better to have a strategic educational plan and evaluation framework than to take the “journey” without a road map.

All teachers teach to the test (or at least they should)

 Charles Ungerleider, Professor Emeritus, The University of British Columbia

[permission to reproduce granted if authorship is acknowledged]

Opponents of large-scale assessments say that they prompt teachers to teach to the test. As I have said in an earlier blog, if the assessments measure performances on things that are important to be able to do or know, teaching to the test is not a bad thing.

But all teachers “teach to the test.” What I mean by that is all instruction begins with establishing a clear objective or destination. Although they may not (should not) wait for external assessments to determine whether the students have achieved the objective or reached the destination, teachers use classroom assessments that they have devised to monitor student progress along the way and, often, at the end of a unit.

Sometimes called backward design or backward planning, the process starts with the end point of the unit or lesson and works backward toward the beginning. Once the goals (and big ideas) are established, teachers develop the instructional sequence that they think will help the students reach the goals, including the ‘way points’ or indicators of progress that teachers assess along the path.

The assessments may entail simple observation on the part of the teacher, the completion of a task or problem the teacher has set, a quiz, a demonstration, a dramatization, a graphic representation, essays, presentations, project work, portfolios, etc. All teacher assessments are high stakes in the sense that cumulatively they will figure in a teacher’s overall appraisal of a student’s performance.

Teacher-made assessments are very “costly.” They are more costly than large scale assessments when you add up the amount of time teachers spend creating and marking the various assessments that they use.

With both teacher assessments and large-scale assessments, validity is a big deal. Are the judgments or decisions made based on the assessment(s) justified by the data generated by them? For instance, I do not eat at [restaurant name] because the first time I ate there I had food poisoning. The tenuous connection between my decision and the data is insufficient to justify my conclusion. Can a recommendation for a gifted program be made because the student produced an imaginative science-fair project? Does a two-minute audition justify a director’s decision to offer or deny a leading role in a school play?

Reliability is another important consideration in assessment. Does the assessment accurately measure whatever it is trying to measure? As I have mentioned in an earlier blog, when I was a child our furnace was fueled with oil from a large tank in our basement. Each fall my father would climb a step ladder and put a stick into the tank to determine how much oil we needed. This was important because the truck that delivered the oil served many customers. The driver, to ensure that there was sufficient oil in the truck, would ask in advance, “how much do you need?” My father would tell him. But my father’s measurement of the oil in the tank was very inconsistent. Sometimes he would insert the stick in the tank on an angle. In the low light in the basement, he would sometimes misread the level on the stick. His appraisal of the volume of oil we needed was often mistaken because of the measurement errors he made. The driver would be annoyed when my father’s assessment was too low, and the driver would have to return to finish filling the tank.

Individual teacher assessments are often unreliable. Fatigue, the pressure of time, ambiguous instructions, and many other factors detract from the reliability of teacher assessments. It is fortunate, however, that teachers make (or should make) many individual assessments before arriving at a judgement about performance (assigning a grade for the year, for example). Although notoriously imprecise, the use of multiple assessments throughout the year is assumed to average out the measurement error.

Interpretation (by parents and other teachers, for example) of the judgements that teachers make about student performance in an area of study is very challenging because teachers vary in what they consider in making an assessment. For some, the assessment reflects work habits, punctuality, task persistence, engagement, etc. There is little consistency across teachers and, often, little consistency from one assessment to another for the same teacher.

Lack of consistency about the features that a teacher is taking into account and inconsistency across teachers compromise the validity of the judgements and decisions made on the basis of assessments. These undisclosed aspects of a teacher’s assessment might be one reason why some teachers complain about “teaching to the test.” The “test” is only one of several things that a teacher uses to gauge student achievement.

That all teachers “teach to the test” is definitely not an indictment of what they do. Teaching to the test is a crucial element of all instruction and should be celebrated.  

Enjoy Your Spring Break, Charles

I will post again on March 31st

The Challenge of Classroom Assessment

 

Charles Ungerleider, Professor Emeritus, The University of British Columbia

[permission to reproduce granted if authorship is acknowledged]

Teachers’ classroom assessments are high stakes. They are the basis for decisions about promotion of students from one grade to the next, the awarding of graduation diplomas and scholarships, and they are often consequential for admission to post-secondary studies.

High stake decisions are not the only purposes to which classroom assessments are put. Teachers use classroom assessments to inform students about whether they have mastered the knowledge they are supposed to have acquired, to plan and modify the teachers’ instructional plans, to provide opportunities for students to practice and apply what they have learned, to determine the knowledge and understanding students need to progress to the next level, to communicate to the student and the student’s parents how the student is doing, to motivate students and more.

Teacher classroom assessment occurs daily and is labour intensive. Observation of student performance happens frequently albeit unsystematically throughout the day. Teachers create numerous kinds of assessments. Assessments include quizzes, tests, and opportunities for demonstration and oral presentations. Appraising performance on each of the teacher-created assessments is time-consuming.

The challenge with teacher assessment is that there isn’t agreement among teachers about what should be assessed. Teacher assessments are driven by a teacher’s professional judgment and, thus, by the teacher’s values. Some teachers believe that assessment should be confined to performance in the subject being assessed. Others believe that assessment should consider student attitude, motivation, or work ethic. Most teachers are aware that a student’s background, gender orientation, and/or personality are not relevant considerations for assessment.

Some teachers argue that assessment should be confined to the student’s performance in the classroom. Others say that homework is an extension of the classroom. Those opposed to assessing homework point out that homework is a form of practice – the most important facet of which is the teacher’s feedback. Others take a developmental perspective, arguing that a formal assessment of homework (beyond just the teacher’s feedback) is acceptable, but that teachers should take a developmental perspective by weighting the assessments more heavily the closer they are to the end of the unit. Some teachers are opposed to assessing homework because it is difficult to determine whether the student worked independently or with the assistance of others.

Teachers sometimes say that colleagues place too much value on print-based assessments (quizzes, tests, essays, reports, etc.) penalizing students who understand the material but have difficulty demonstrating what they know in writing. These teachers supplement written assessment with other demonstrations of understanding (oral presentations, dramatizations, diagrams, use of manipulatives, etc.).

Educational researcher Susan Brookhart and her colleagues reviewed the literature devoted to grading over the past 100 years with a particular emphasis on the meaning and value associated with grades. Among the many important findings of this review were:

·       Teachers primarily assess achievement using tests.

·       Teachers assess many non-academic factors in assigning grades, including effort, improvement, perceptions of student ability, completion of work, and other student behaviour.

·       The relative emphasis teachers assign to academic and non-academic factors differs markedly across teachers.

·       Assessment varies considerably by grade level.

The conclusion that Brookhart and her colleague draw is worth quoting at length:

This review suggests that most teachers’ grades do not yield a pure achievement measure, but rather a multidimensional measure dependent on both what the students learn and how they behave in the classroom. This conclusion, however, does not excuse low quality grading practices or suggest there is no room for improvement. One hundred years of grading research have generally confirmed large variation among teachers in the validity and reliability of grades, both in the meaning of grades and the accuracy of reporting.

Their conclusion is troubling. Inconsistency in the assessment process means that teachers other than the one who made the assessment find it difficult to know how to interpret their colleague’s assessment. It takes a leap of faith to believe that, in the aggregate, teacher assessments can support the high stakes decisions on which they are based.

Some ministries and school boards try to reduce the variability of teacher assessments by encouraging the use of performance standards and separating the appraisal of academic and non-academic factors. While those efforts are to be commended, those practices are also inconsistent across jurisdictions.

Notwithstanding these efforts, much more work is needed to improve classroom assessment, including clarifying the purpose of assessment, distinguishing between academic and non-academic factors, improving teacher preparation in assessment and in communicating results of assessment to students, parents, and others who make use of assessment results.

_______________

Brookhart, S. M., Guskey, T. R., Bowers, A. J., McMillan, J. H., Smith, J. K., Smith, L. F., Stevens, M.T., Welsh, M.E. (2016). A Century of Grading Research: Meaning and Value in the Most Common Educational Measure. Review of Educational Research, 86(4), 803-848.