Evaluation of an Online English learning program

Abstract

Along with the rapid growth of computer technology, the need for CALL evaluation has become increasingly important; however, its implementation remains problematic for a number of reasons including teachers’ lack of experience and adequate skills (Hubbard, 2006), lack of mutually agreed-upon adequate criteria (Hubbard, 2006), and lack of higher-level evaluative skills to evaluate media online (Oosterhof, Conrad, & Ely, 2008). This study describes the classroom-based evaluation of a CALL program. It was undertaken in a Taiwan technological institute, with participants (N = 39) from second-year night-time junior college. The aims were to evaluate the effectiveness of the English Discoveries Online (EDO) program, to identify changes for more effective future use and sound criteria for CALL evaluation. The program’s key features include self-directed courses, self-access out of class, regularly updated materials delivered through the Internet, Web-literacy development, the Teacher Management System, and support tools. Effectiveness was determined by examining self-access learning out of class, motivation, Web-literacy, English proficiency, and the above key features. Both quantitative and qualitative methods were used. The evaluation followed Alderson’s (1992) framework for planning, Weir and Roberts (1994) for design and implementation, and Ellis’ (1997a & b) three types of evaluation for data collection. Instruments included: questionnaires, checklists, an observation record of student behaviors, a monitoring record of the computer screens, reports, individual student records on self-access, and student e-mails sent to the teacher-researcher. The results showed that students considered the EDO helpful for learning English, for enhancing their computer knowledge and English learning interest out of class. There was no significant change in students’ motivation and Web-literacy. There was a significant improvement in students’ pre- and post-test scores in reading and writing. More positive rather than negative comments were given about the key features of the program in students’ reports. Overall, 91% of the class considered the program effective. The findings indicate that this program has a place in English teaching in this Taiwan context. Implications and limitations, as well as lessons learned as a teacher and an evaluator, are discussed.