Abstract:
Introduction: The University of Auckland began to explore the introduction of Progress Testing (PT) in 2011. The MBChB is
six years; Year 1 being a shared foundational health science year. After selection, Year 2 is the start of the medical programme
proper. Years 2-3 are largely campus-based and Years 4-6 are in the clinical workplace. After considering the literature
and discussion with other users, we partnered with PMCD (UK). PT was to become our major tool for assessing applied
knowledge. Implementation: In 2013 we introduced PT to Years 2 and 4, rolling out to all years over the next two years.
We have three tests/year, the first in Year 2 is formative. Each test consists of 125 single best answer questions, with a sixth
“don’t know” response, and uses formula scoring. The Peninsula question bank has been supplemented and adapted. Initially
Peninsula’s data infrastructure was used, but we have developed local tools over time. We now have data on three cohorts
who have used PT from entry to graduation. Data shows similar velocities of progress, no overlap between years within
cohorts and a flattening of peak achievement in the final year.
Aims: Our aims were to reduce a “cram and dump” mentality, provide an assessment for learning, ensure workplace readiness,
allow us to identify and intervene early with struggling students (“frequent look, rapid remediation”), ensure clinical context
from Year 2 and revisit biomedical science in Year 6, reduce stress of high stakes end-of-year examinations.
Developments:
• Dashboard for students that provides feedback for learning and links to the case scenarios that underpin our curriculum
• Systematic moderation process
• Rasch model for item and student performance
• Standard-setting process for final year, utilising classical and Rasch data
• Benchmarking with other Australasian programmes What next?
• Continue with formula scoring?
• Develop an adaptive approach to progress testing?
• International collaboration?
• Revisit standards for Years 2-5?
• Community of practice for psychometricians?
