A Framework for Long-Term Learning Systems

Abstract

Machine learning comprises the ability of computer systems to learn from data. The adoption of machine learning technologies has increased substantially during the past decade. The common setting in machine learning is to use a set of data examples to learn a representation that describes these examples and/or can be used to predict unseen data. Learning tasks are typically executed in isolation, and the performance during these tasks is evaluated using pre-defined metrics. Learning a sequence of those tasks is a long-standing challenge in machine learning. This setting applies to learning systems that, rather than observing all tasks at once, observe examples of these tasks at different points in time. A learning system should become more knowledgeable as more related tasks are learned. Although the problem of learning sequentially was first acknowledged a long time ago, the research in this area has been rather limited. Research in transfer learning, multitask learning, meta learning and deep learning has studied some of the challenges of these kinds of systems. Recent research in lifelong machine learning and continual learning has revived interest in this problem. In this thesis we propose a framework as an approximation to systems that learn a sequence of supervised tasks. Our proposed framework relies on knowledge transferred between hypotheses learned with Support Vector Machines. The first component of the framework is focused on transferring forward selectively from a set of existing hypotheses or functions representing knowledge acquired during previous tasks to a new target task. We demonstrate that transferring selectively from previous hypotheses helps to achieve better convergence rates on target tasks, whilst maintaining similar accuracy levels than counterpart methods. A second connected part of this framework is focused on transferring backward, a novel ability of long-term learning systems which uses knowledge derived from recent tasks to encourage refinement of existing knowledge. We first propose a method for transferring backward selectively in systems composed of a small number of tasks. Then, we extend this idea to systems that observe any number of tasks sequentially. We demonstrate, both theoretically and experimentally, that knowledge refinement by transferring backward selectively can be achieved whilst encouraging retention of knowledge acquired during previous tasks. Finally, we conceptualise and present Proficiente, a full framework for long-term learning systems. We accompany our framework with a novel and simple metric that can be used to determine if a long-term learning system is becoming more knowledgeable as more tasks are observed. We evaluate our proposed methods in a range of real-world datasets, and on synthetic datasets proposed for supervised lifelong machine learning.