Some mistakes are too important to ignore

 Charles Ungerleider, Professor Emeritus, The University of British Columbia

[permission to reproduce granted if authorship is acknowledged]  

There are mistakes in Geoff Johnson’s article “FSA not a good way of assessing Indigenous students” in the Times Colonist posted on October 29^th that deserve correction for several reasons. One is that the Times Colonist is widely read. The other is that the designation of “former superintendent” adds authority to the errors he has made.  

As noted in his article, “the Foundations Skills Assessment is administered to all Grade 4 and Grade 7 students, both Indigenous and non-Indigenous.” The mistakes arise from his claim that the Foundation Skills Assessment is not a good way of assessing Indigenous students. One of them is the logical fallacy in his statement. He writes, “turns out that the four ‘lowest performing’ elementary schools on the FSA, according to the Fraser Institute’s one-shot ranking system, have significant populations of First Nations students.”  

In making that statement Johnson is committing the logical fallacy “after this, therefore because of this.” The logical fallacy occurs when someone assumes that because one event or set of conditions preceded another, the first event must have caused the second. In saying that the lowest performing elementary schools have significant populations of First Nations students, he is implying that the poor performance is attributable to the student composition of those schools.  

Another mistake deserving correction is drawing the connection between the performance of those schools and the assertion that “assessing a child in a way that does not seem meaningful or relevant to their life and culture is inauthentic and therefore meaningless, because it does not respect the learning of the whole child.” Indigenous children live in a society in which and literacy in the dominant language and numeracy figure prominently and where both have been used to deny Indigenous peoples their rights. Knowing how well Indigenous students perform on such assessments is essential to ensuring that they are being educated to the same standard as their non-Indigenous peers and from equipping them with the knowledge they need to defend their rights.  

I agree with Johnson that ensuring that the measures used to assess Indigenous youngsters are fair is essential. Johnson strongly asserts that “a central problem with the lack of validity of the FSA, as far as Indigenous students are concerned, is that the tests often contain items expressed in a way not obvious to an Indigenous student who might have a worldview and experiences that differ from the dominant Western culture.”  

This is something about which evidence can be brought to bear. Differential Item Functioning (DIF) is a technique used in assessment to determine if a question is fair to different groups of people. Imagine, for example, a math assessment, and you find out that for some reason, boys are more likely to get Question 5 right than girls, even when both groups are equally good at math. If that's the case, then Question 5 has "differential item functioning." Question 5 is not measuring math skills equally for boys and girls. DIF helps us figure out if a particular question on a test is easier or harder for one group of people compared to another, helping to ensure that tests are fair and unbiased.  

Before strongly asserting that the FSA is inappropriate for Indigenous students, one should consider the evidence. Johnson’s reference to the 2013 article by Jane P. Preston and Tim R. Claypool and the invocation of the imprimatur of The Canadian Council on Learning does not substitute for evidence about the Foundation Skills Assessment.  

Johnson quotes a B.C. government media release saying, “the redesign of curriculum maintains a focus on sound foundations of literacy and numeracy while supporting the development of citizens who are capable thinkers and communicators, and who are personally and socially competent in all areas of their lives.” He follows the quotation with the claim that the government statement ignores research about Indigenous ways of learning and is “dangerously close to being as colonial as you can get.”  

Could it not be argued that Johnson’s assertion is colonial? Johnson seems to imply that Indigenous students are by reason of ancestry or circumstance unable to demonstrate that they are capable thinkers and communicators. I doubt that was his intention. However, it seems very similar to the incorrect inferences drawn about women; namely that they are by constitution incapable of being pilots, surgeons, entrepreneurs, etc.  

There are three crucial omissions from Johnson’s article. One is the fact that the First Nations Leadership Council supports the use of the Foundation Skills Assessment as “. . . one among many tools necessary to address the ‘racism of low expectations’ experienced by First Nations learners as identified by the BC Auditor General in their 2015 report.” The second is that Indigenous educators are involved in item development for the assessment. The third is that the assessments contain First Peoples content written by and about First Peoples, and their development is guided by the First Peoples Principles of Learning.  

It is for these reasons that I hope Johnson will revise his article in the Times Colonist.

Why we need the Foundation Skills Assessment

 Charles Ungerleider. Professor Emeritus, The University of British Columbia

[permission to reproduce granted if authorship is acknowledged]

British Columbia is administering its annual assessment of numeracy and literacy, the Foundation Skills Assessment (FSA), in October and November of this school year. Its administration will generate useful information about how well the system is preparing students in those key areas. But the administration of the FSA will also generate much misinformation that deserves to be corrected.

 Those who argue that the FSA should not be administered during COVID have it wrong. COVID is an event that has had a significant impact on student learning, the magnitude of which is unknown. Based upon evidence from elsewhere, it is likely that the most vulnerable students have suffered disproportionately. We should find out! The FSA is the only tool that will provide a province-wide picture of COVID’s impact on students. It is hard to fix a problem if you don’t know its scope.

 As is the case with most large-scale student assessments, the Foundation Skills Assessment is a tool for decision-making at the provincial, school board, and school levels. The FSA provides useful information about system performance and program effectiveness and about equitable performance and outcomes. Assessments like the FSA make it possible to monitor the performance of groups of students (racialized, Indigenous, boys and girls) to ensure equity of outcomes.

 Although used for different purposes, both classroom and large-scale assessments are valuable. The information from large scale assessments can help to inform plans that schools make for improving student performance that teachers appraise using classroom assessments.

Classroom assessments are specific to classroom conditions and student characteristics. Classroom assessments, while valuable for individual learning, do not provide robust, reliable information at a system-wide level using common benchmarks. Classroom assessment is different from large-scale testing in many ways: purposes, techniques, frequency, timing, consequences.

Large-scale assessments are designed to produce province-wide information based upon standardization of administration, scoring, and reporting. Assessment at the classroom, school district, or provincial level do not use a common measurement scale. The meaning of a C+ assigned by grade 8 teachers means different things in Surrey, Smithers, and Saanich.

The FSA is a ‘no stakes’ assessment. The results have no bearing on any individual student or teacher. The FSA is not used to make decisions about individual students or teachers. The information about student performance is not used to assign marks (grades), determine whether a student is promoted to the next grade or retained in grade, or influence any other aspect of a student’s school experience. The same is true for teachers. The results do not affect their salary, job security, or anything else.  

Nonetheless, like many experiences – making a new friend, flying in an airplane, taking an amusement park ride - the FSA sometimes creates anxiety. Educators and parents play an important role in alleviating assessment anxiety. Teachers can help prevent anxiety about the FSA by explaining the use of the assessments.

The FSA assesses dimensions of student performance in reading, writing, and numeracy that are regarded by teachers, parents, and the broader society as being fundamental or foundational to student learning. It is therefore no surprise that teachers feel they should “teach to the test.” The no-stakes nature of the FSA should eliminate any perceived need to teach to the test, but there is nothing wrong in teachers teaching to a test that assesses important foundational skills. It is especially important for teachers to give students an orientation to the kinds of questions and tasks they will encounter on the assessment so that confusion or misunderstanding aren't the cause of less than representative performance.

Schools teach more than reading, writing, and numeracy. They teach personal and social development, citizenship and social responsibility, science, the arts, languages, etc.  If what counts is measured, I think we should create assessments for all the areas to which we devote our scarce time and resources.