An Experiential Learning Approach to Learning Manual Communication Through a Virtual Reality Environment

Abstract

There is a pressing need for effective pedagogical methods of manual languages, as evident in the decline of manual languages, such as New Zealand Sign Language (NZSL). Despite being recognized as one of New Zealand's official languages, recent censuses have shown that fluent NZSL signers have been steadily decreasing. There is a cultural responsibility to preserve such languages, yet the combination of barriers to acquisition and the limited availability of effective teaching methods are standing in the way. In light of this, this article proposes a computer-assisted sign language learning system that incorporates virtual reality and validation-based feedback as tools to implement the experiential learning model. An implementation in the form of virtual reality NZSL is presented, targeting the set of NZSL alphabet. Results show that a vision-based classification method, using the Leap Motion controller, is a scalable, accurate, and usable solution for feedback-assisted manual language learning. A formative usability evaluation of $n=10$ participants showed promising results for engagement, confidence, and memory retention. The results indicate that virtual reality technology is uniquely situated as an innovative medium for the self-directed acquisition of manual languages. It is hoped this article inspires technology researchers to pursue collaborations with the deaf/Deaf community to design and develop pedagogical technology solutions for manual communication.