Charles Ungerleider, Professor Emeritus, The University of British Columbia
[permission to reproduce granted if authorship is acknowledged]
Effective feedback supports academic achievement and personal development by helping students to improve. When it is expressed clearly, feedback explains what students are doing well and where they need to focus their efforts. When expressed constructively, feedback encourages students to reflect on their work, understand their mistakes, and learn from them.
Teachers face several obstacles in providing effective feedback. Time constraints are a major issue; providing detailed feedback for numerous students can be overwhelming. Providing feedback to large classes is daunting. It is challenging to provide nuanced feedback aligned with the strengths and weaknesses of each student. These obstacles often reduce the frequency and quality of feedback that teachers provide to students.
A recent study suggests that generative-AI holds promise for providing formative feedback to students.[1] The study compared the quality of formative feedback provided by human evaluators and generative AI, in this case ChatGPT 3.5 (an earlier, public version of ChatGPT). The human evaluators were provided with a 3-hour training session that addressed evaluation criteria, actionable areas for improvement, and effective feedback strategies. They were provided with rubrics and guidelines and had two weeks to provide feedback. On average, the human evaluators spent about 20 minutes per essay.
There were differences in feedback quality between humans and AI. The well-trained human evaluators generally provided higher quality feedback in most categories, such as clarity of directions for improvement, accuracy, prioritization of essential features, and use of a supportive tone. Although the differences between human and AI feedback were statistically significant, they were relatively small.
One of the implications of the study was that generative AI can provide useful feedback to students, which can be especially beneficial in large classes where it may be challenging for teachers to provide individualized feedback to each student. Moreover, generative AI can provide feedback immediately and iteratively, allowing students to receive feedback in a timely manner. This is important because timely feedback has been shown to be more effective in promoting learning and improvement.
Generative AI is low-cost and accessible which makes it a more feasible option for providing formative feedback compared to traditional methods that require extensive training and calibration. This accessibility can potentially increase the availability of feedback to students, especially in contexts where well-prepared educators may be unavailable.
Generative AI can provide feedback that is aligned with the needs of individual students. With proper preparation and analysis that includes examples of students’ writing and comparison to exemplars and performance standards, generative AI can provide specific steps for improvement that are tailored to the student's writing performance. enerative AI can be particularly useful in providing formative feedback on early drafts of student writing. This can help students identify areas for improvement and make revisions before submitting their final work. Generative AI can also provide consistent feedback across different students and essays, ensuring that all students receive the same level of feedback and guidance. This can help maintain fairness and equity in the feedback process.
The fact that this
study used ChatGPT 3.5 suggests that the findings regarding AI feedback quality
may not fully reflect the capabilities of the most recent versions of ChatGPT
or other generative AI models. Since the study, there have been
significant improvements in AI technology, including advancements in the
processing of everyday speech and model training. These improvements may have resulted in even-better
AI-generated feedback, potentially narrowing the gap between human and AI
feedback quality.
[1] Steiss, J., Tate, T., Graham, S., Cruz, J., Hebert, M., Wang, J., Moon, Y., Tseng, W., Warschauer, M., and C. B. Olson, (June 2024) Comparing the quality of human and ChatGPT feedback of students’ writing, Learning and Instruction, 91, https://doi.org/10.1016/j.learninstruc.2024.101894 .