Mixture Modeling for Multivariate Observations

Abstract

In this thesis, nonparametric multivariate density estimation is studied. The kernel density estimation predominately used in the literature is known to be less than ideal in both computation and estimation efficiency, especially for multivariate observations. Mixtures, particularly nonparametric and semiparametric mixtures, have the potential to outperform the kernel-based methods, as has been shown in previous research in the univariate case. It is the main goal of this research to extend the application of these mixture models to estimate nonparametrically a multivariate distribution. The major difficulty with the likelihood approach that is associated with the estimation of these mixtures in the multivariate case is that the likelihood function cannot be maximized directly with respect to the whole component covariance matrix, because the covariance matrix will become singular, and that it is also computationally infeasible to use a model selection method to select a large part of the covariance matrix. To overcome it, the new method uses a volume parameter h to put a minimal restriction on the covariance matrix and hence has the likelihood function bounded with h fixed. The scalar h plays the same role as a bandwidth parameter in the univariate case and its value can be determined by a model selection method. New algorithms are also developed for fitting nonparametric and semiparametric mixtures in the multivariate case. Empirical studies using simulated and real-world data show that the new method performs significantly better than the kernel-based methods. The availability of the new density estimator makes it possible to provide new solutions to other statistical problems. Two problems are especially addressed in the thesis: classification and the fitting of generalized linear mixed models with multiple random effects. For classification, a new classification method is proposed that makes a direct use of the new density estimator. It firstly estimates the probability density function for the observations in each class and then classifies a new observation according to the maximum a posterior criterion. Some issues specific to classification are also studied and discussed. The new classification method is able to produce adaptively smooth and complicated decision boundaries in a high-dimensional space. Empirical studies indicate that it outperforms the commonly used methods. For generalized linear mixed models, the situation with multiple random effects is investigated, where their joint distribution is completely unspecified. This naturally leads to a semiparametric multivariate mixture formulation and its fitting is to find the nonparametric maximum likelihood estimate of the joint distribution of random effects and the maximum likelihood estimate of the fixed effects. Efficient algorithms are also developed for fitting these generalized linear mixed models. Numerical studies with simulation and real-world data show that the new method performs favorably well as compared to the parametric method under the normality assumption.