Evaluating the impact of Khan Academy tutorials: A case study of an educational technology being set free

Abstract

This study evaluated an intervention conducted to improve students’ achievement in mathematics by using technology. The study was meant to answer the following question: Does giving students access to IT services and prompting them to use the Khan Academy Tutorial system make any difference in students’ achievement, in-class behaviour, and attitudes towards mathematics?. The school in which this intervention took place was Decile One (i.e., low SES). The 86 Year 9 students who participated were from minority ethnic groups in New Zealand (i.e., Māori, Pasifika, and Indians). Students could borrow from the library one of 50 tablets for up to two days, for use at home or in the school. . They were encouraged to use the table to access Khan Academy tutorials to support their learning in mathematics; however, they were not required to do so. The results showed no statistical significance in students’ mathematics achievement; and a slight improvement in students’ inclass behaviour. Based on focus group responses, it appeared that students used tablets mainly for entertainment rather than for learning from Khan Academy. This could be traced to three reasons (a) using ICT in learning is distractive, (b) the majority of students had low mathematics self-efficacy, so they did not even try to use Khan Academy, and (c) some students were not convinced about the point of using technological devices in learning. In addition, it is suggested this improvement in students’ in-class behaviour might decay overtime when students stop considering the tablets as “Cool”; unless, the school used this improvement to address the real problem which was low achievement. Moreover, this research suggested solutions to make the intervention more productive in the future: (a) integrating using tablets with normal classes, (b) creating an account in Khan Academy to track students’ work, and (c) training students’ on time management and self-discipline. Keywords: Mathematics, low SES, Year 9, New Zealand, Auckland, high school, ethnic minority students, ICT, self-directed learning, tablets, self-efficacy, Khan Academy