New neural network models based on unsupervised competitive learning

Abstract

This thesis examines the development, evaluation and some possible applications of new neural network models based on unsupervlsed competitive learning. Unsupervised learning methods can exploit the statistical regularities of unlabelled input examples and form meaningful representations. Competitive learning is a suitable mechanism for this learning paradigm. The neural network systems which are based on competitive learning are frequently used for solving certain pattern recognition tasks such as pattern classification and 7f-means clustering. Basic aspects of pattern recognition and cluster analysis are outlined. Principles of operation of some important neural network models are described. A review of competitive learning and some associated problems are presented. A new learning algorithm called Dynamic Competitive Learning (DCL) is developed to overcome the deficiencies of simple competitive learning methods which rely on the Winner-Take-All (WTA) type of competition. Unlike the WTA method, the proposed algorithm takes into account all the win-loss information of the competing neurons. Based on the DCL algorithm and the neurobiologically inspired learning processes, two versions (binary and analogue) of a neural network model, termed a differentiator, are presented. Several features of the differentiator (e.g., the use of control neurons and one-step lateral inhibition) distinguish it from other unsupervised learning models. The performance of the proposed models is evaluated for cluster detection and centroid estimation problems using computer simulations. Comparative studies show the superiority of the differentiator over other competitive learning models in terms of efficient use of weight vectors, insensitivity to the initial weight values, speed of training and clustering distortion. By combining the unsupervised differentiator and a supervised associative network, an alternative network for performing mapping operations is achieved. Two versions (binary and analogue) of the mapping network, called the Extended Differentiator Network (EDN) are presented. The speed of training in the EDN is substantially lower than that in the supervised networks which use gradient descent techniques. The operation of the EDN in different Iearning paradigms; namely auto-association, pattern association, pattern classification and regularity detection are investigated. It is shown that the EDN's approach for solving various computational problems, highlighted by its fairly simple structure and ease of training, makes it a viable network for many applications. Three application areas which require the classification, mapping and adaptive learning capabilities of neural networks are outlined and the feasibility of utilising the EDN is investigated. These applications are vowel recognition, voiced/unvoicecbsilence classification and adaptive congestion control for broadband ATM communications networks. Feature extraction techniques from time-varying signals are introduced for the first two applications. With respect to the third application, neural control structures using forward and inverse modeling of the dynamics of the communications network are suggested for the control of the input traffic subject to the bounds on the transmission delay and loss rate.