21st Century Learner Self-Report Feedback Inventories: A review and defence of psychometric approaches

Abstract

<p>Interest in how learners understand, respond to, and value educational feedback has increased notably in the last two decades. Much of the current evidence of the impact or influence of feedback arises from self-report inventories. Self-report respondents are presumed to be well situated to report on their own experiences, attitudes, and behaviours. However, considerable validity threats exist in this method, which may limit the meaningfulness of self-reports. Consequently, many researchers are concerned about the disproportionate role self-reports play in informing our understanding of effective feedback. The validity of claims about learner self-reports on feedback depend on the credibility of the measures used. Inventories developed within the psychometric tradition are expected to provide theoretical and empirical evidence for the validity and reliability of the measures to support subsequent interpretations and decisions. This review provides a defense of the potential value of psychometric self-report data in informing the psychology of effective feedback. A systematic review of 14 feedback self-report inventories published in the 21st century found 25 empirical studies that replicate one or more of these inventories. Thirteen survey and 12 experimental studies were reviewed to provide a thematic analysis of the state of research. Seven studies used latent theory modeling to test the robustness of the MIMIC-based inventories, relying instead on estimates of scale reliability. Eight of 25 studies reported full replication of the inventory. Just four studies had an independent measure (academic achievement), with the vast majority of studies relating feedback inventories to self-reported constructs. Most effects were small and used small samples, which weakens reproducibility. The data showed that feedback perceptions have complex overlap with psychological control and competence beliefs, largely consistent with self-regulation of learning theory. Psychometric methods can improve the quality of research into how students understand, experience, think about, and use feedback that is given to them.</p>