Learning Analytics Information Systems for Higher Education: Design, Development and Implementation

Abstract

Information Systems (IS) and technology has long transformed the landscape of education. Recently, learning analytics (LA) has emerged as a new educational technology and drawn the attention of academics, researchers, and administrators. LA can offer educational stakeholders useful feedback and insights about the process of learning and teaching. LA is rapidly being applied in different educational settings including higher education, often as an ad-hoc analysis without the guidance of a research. Although the literature has revealed a substantial base of LA techniques for analysing discourse, social interactions, and descriptive and predictive models, few institutions have been able to provide evidence around successful implementation of learning analytics information systems (LAIS). Despite the currently limited understanding of underlying information systems that support learning analytics, existing reports on LAIS implementations indicate their significant potential for improving learning and teaching. This research aims to examine the development and implementation of LAIS in the context of higher education. Notably, it attempts to address two overarching research questions: 1) What are the fundamentals of learning analytics and their applications? And 2) How to design underlying information systems that support learning analytics? First, the study investigates and contributes to better understanding of the concepts around data analytics in higher education, i.e., learning analytics (LA), academic analytics (AA), and educational data mining (EDM). This study is then focused on LA which centres around learning and teaching. The key dimensions of LA are determined to establish a multi-layered taxonomy of LA applications and a domain-specific ontology. Furthermore, the study proposes conceptual frameworks and roadmaps for applying LA in different learning environments. The study establishes a foundation for the development and implementation of LAIS by creating a set of design principles. Moreover, LAIS architecture is proposed for managing processes and technologies. An operational LAIS is developed and implemented at a university for demonstration and evaluation. The research offers not only several conceptual and system artefacts, but also empirical evidence of the opportunities and challenges facing the implementation of LAIS in higher education. This study was guided by the ontology-based design science research (ODSR) approach. The research proposes ODSR as an approach combining the forces of design science research (DSR) and ontology studies for knowledge accumulation and evolution. ODSR offers a semantic view of the research domain, and the use of ontologies provides considerable help throughout the process of creating and evaluating research artefacts in DSR. The ontologies are also considered as research artefacts created and evaluated through DSR cycles. As a DSR methodology, ODSR is multi-methodological and involves multiple iterations of observation, theory building, artefact design, development, demonstration, and evaluation. This thesis follows the "Ph.D. with publications" approach and contains six original research articles. The first article describes the research approach that was used in this study. The second article reviews the literature on learning analytics and related concepts in the context of higher education to provide a holistic view of the domain. The third article examines the critical dimensions of learning analytics and constructs a multi-layered taxonomy of learning analytics applications. The fourth article builds a domain-specific ontology of learning analytics to offer a semantic view. The fifth article derives the proposed conceptual framework for developing learning analytics and demonstrates its application in a specific context, i.e., educational games for learners with intellectual disabilities. The sixth paper develops a set of design principles for LAIS then demonstrates and evaluates them in a case study.