Detecting differential item functioning of polytomous items in small samples: Comparison of MIMIC with a Pure Anchor and MIMIC-Interaction methods

Abstract

Differential item functioning (DIF) may be a result of either item bias or a real difference depending on whether the source of DIF is either construct-irrelevant or construct-relevant. It is relatively more challenging to conduct DIF studies when the small sample is small (i.e., < 200), items follow polytomous scoring (e.g., Likert scales) instead of dichotomous scoring, and psychological grouping variables are used instead of demographic grouping variables (e.g., gender). However, the multiple indicators-multiple causes (MIMIC) approach can be a promising solution to address the aforementioned challenges in DIF studies. This study aims to investigate the performance of two MIMIC methods, namely MIMIC with a pure anchor (MIMIC-PA) and MIMIC-interaction methods, for DIF detection in the Student Conceptions of Assessment inventory based on a psychological grouping variable derived from students’ self-efficacy and subject interest. The results show that MIMIC-PA identified five mathematics and eight reading items with large DIF in the four factors. MIMIC-interaction showed that no items had uniform DIF, while four items had non-uniform DIF. Items with statistically significant DIF were aligned with the known effects of self-efficacy and subject interest on academic achievement, supporting the claim that observed DIF reflects item impact rather than bias. The study's implications for practice and directions for future research with the MIMIC approach are discussed.