ThinkInk - Developing and Evaluating a Digital-Ink Based Interactive Data Structures Tutoring tool

Abstract

This research explored the effect of a tutoring system's interaction modality on learning. To better understand this effect, a digital-ink tool running on a stylus supported device was developed and compared against a keyboard & mouse based tool. Digital ink is formed while writing on a stylus supported device like Microsoft Surface. Due to its similarity with pen and paper, digital-ink tools have been preferred over keyboard & mouse based tools for learning topics that involve diagramming as is the case in various Science, Technology, Engineering and Mathematics (STEM) topics. Learning data structures and algorithms is challenging due to its abstract nature and is commonly taught with the help of diagrams. Hence, a digital-ink tutoring tool for learning data structures was developed and evaluated against a keyboard & mouse based data structure tutoring tool. A literature review was carried out to understand the learning process of data structures and to analyse the existing learning tools in the area. Many keyboard & mouse based tools were found, however, just three digital-ink tools were found and hence digital-ink tools became the focus of this project. To understand the interactions and feedback mechanisms of digital-ink tutoring systems, the use of such systems in the STEM area was studied. This research followed an iterative design methodology. Hence, an exploratory prototype of ThinkInk was built and a cognitive dimensions analysis was carried out. The main lessons learnt from the analysis were: the importance of digital-ink recognition; the need for guided instructions of tasks; and an intuitive and interactive user experience. After this, the design and development of ThinkInk - Prototype 1 was commenced. Firstly, a paper prototype was built and evaluated with a Wizard of Oz study. This was followed by the development of Prototype 1 which had tasks for 1D arrays. Next, a usability study was done on this prototype and improvements were made based on the participants' feedback. Following this a technology acceptance analysis for understanding the user experience and learning gain analysis to detect actual learning were conducted. Prototype 1 was found to be liked by the majority of the participants. Relationships between the constructs of Perceived Playfulness, Perceived Ease of Use, Perceived Usefulness, System Interactivity, User Interface Design and Behavioural Intention were explored to get a deeper understanding of the user experience of Prototype 1. The learning gain results also showed that a majority of the participants achieved learning gain. Next, ThinkInk - Prototype 2 was developed which had tasks for tree traversals and Binary Search Trees (BST).This prototype included changes based on the feedback from the analyses of Prototype 1. A between groups A/B experiment was conducted to compare a keyboard & mouse data structure tutoring tool JSAV with Prototype 2. Both user experience and learning gain was measured. The user experience analysis showed that users found some aspects of Prototype 2 for user interface design, system interactivity and ease of use significantly better than JSAV. However, overall there was no significant difference in the user experience for the two tools. Majority of the participants achieved learning gain in both the learning environments. There was no significant learning difference between the tools for most tasks. However, learning gain for ThinkInk was significantly better for in order traversal and BST insertion. For most aspects the two tools did not have significant difference in user experience or learning. Nonetheless, the consistently positive results for both the prototypes of ThinkInk across two different experiments does show that a digital-ink tool can provide a good user experience along with learning gain. The factors of guided instructions, constrained task design, an easy to use interface and interactive and engaging visualizations played an important role in the success of the two prototypes of ThinkInk. The iterative development process of ThinkInk combining concepts of education theory and human computer interaction design and development procedures helped in the success of ThinkInk. This research can be used as a case study for building educational tools especially those in the area of computer science. Studies with a larger sample size, more tasks and in different educational areas could be carried out to establish the learning effectiveness and preference of different interface modalities. The effect of tool modality on learning could not be clearly understood and future studies should probe this area further. However, the findings do establish digital-ink tools as a successful alternative to keyboard & mouse based learning tools for data structures and algorithms.