Reliability of individual teacher administrator practices during national monitoring assessment interviews in social studies

Abstract

Since 1995 New Zealand’s National Education Monitoring Project (NEMP) has been responsible for the national assessment of students’ achievement in each of the learning areas in the curriculum. One of the assessment approaches used by NEMP is the one-to-one student interview. This paper addresses variation within individual Teacher Administrators (TAs) as they conducted interviews during the 2005 round of monitoring in social studies. An observation schedule was used to gather data from 12 randomly selected TAs, across ten categories, as they carried out three administrations for each of three selected social studies tasks. It is argued that the variations observed within individual TA’s practice were related to elements of the assessment tasks and TAs’ interpretations of these tasks. In addition TAs’ subject knowledge, their understanding of the TA role and their understanding of the ‘standardised’ one-to-one interview process also contributed to variations within individuals. Overall, the nature and levels of variation observed have the potential to pose threats to reliability and have an impact on the consequential validity of information.