The reliability of essay scores: The necessity of rubrics and moderation

Abstract

The use of essays is a well-established means of evaluating student learning in higher education. The accuracy of scoring essays is estimated with three means (i.e., consensus, consistency, and measurement estimates). Evidence is consistent that inter-and intra-marker rating of essays is unreliable. The nature of essay tasks and marker behaviour has been shown to contribute significantly to the error component in essay scores. Scoring guides or rubrics to guide scoring and multiple marking or moderation of essays are necessary requirements of reliable essay scores. While essays may be valid. their usage depends on accuracy of scoring and if this cannot be established, alternative assessment approaches should be considered.