(120b) Program Effectiveness - Evaluating Student Learning of Key Concepts from Year to Year

The {Department blinded} at the {University blinded} undertook a broad curriculum renewal of its core undergraduate programs from 2019 to 2022. During this time, programs it offers were holistically re-evaluated and major changes were put in place [1]. As part of this effort there was a re-evaluation of the methods of measuring program effectiveness. One outcome was the development of a tool for evaluating student learning of key concepts at various year levels [2]. This tool parallels developments at other institutions [3].

Quizzes consisting of conceptual multiple choice questions were posed to student based on concepts from core undergraduate courses. These questions aimed to be at a high-level and did not require sophisticated computation. Students were also given the option to select an “I don’t know” answer. The purpose of the quizzes, namely evaluating one aspect of program effectiveness, was explained to students prior to taking the quizzes. Quiz completion was for a participation mark in one of their courses but the quiz grade itself was not used for evaluation. Students completed two quizzes at the start of each year, one corresponding to the prior year of study and one corresponding to concepts in courses to come (to which they may have not yet been introduced).

Data has been collected since the 2021-2022 academic year. Analyses have been performed evaluating student knowledge changes from year to year, as well as areas of challenge for class cohorts. This work will present a background on the quiz development as well as a highlight of the analysis results.

[1] {blinded for review}

[2] {blinded for review}

[3] P. Farand and J. R. Tavares, “A concept inventory for knowledge base evaluation and continuous curriculum improvement,” Education for Chemical Engineers, vol. 21, pp. 33–39, Oct. 2017, doi: 10.1016/j.ece.2017.07.001.