Author: Jakob Jozef VerbeekPublisher:
ISBN: 9789057761256
Category :
Languages : en
Pages : 162
Book Description
Mixture Models for Clustering and Dimension Reduction
Author: Jakob Jozef VerbeekPublisher:
ISBN: 9789057761256
Category :
Languages : en
Pages : 162
Book Description
Dimension Reduction for Model-based Clustering Via Mixtures of Multivariate T-Distributions
Author: Katherine MorrisFinite Mixture Models
Author: Geoffrey McLachlanPublisher: John Wiley & Sons
ISBN: 047165406X
Category : Mathematics
Languages : en
Pages : 419
Book Description
An up-to-date, comprehensive account of major issues in finitemixture modeling This volume provides an up-to-date account of the theory andapplications of modeling via finite mixture distributions. With anemphasis on the applications of mixture models in both mainstreamanalysis and other areas such as unsupervised pattern recognition,speech recognition, and medical imaging, the book describes theformulations of the finite mixture approach, details itsmethodology, discusses aspects of its implementation, andillustrates its application in many common statisticalcontexts. Major issues discussed in this book include identifiabilityproblems, actual fitting of finite mixtures through use of the EMalgorithm, properties of the maximum likelihood estimators soobtained, assessment of the number of components to be used in themixture, and the applicability of asymptotic theory in providing abasis for the solutions to some of these problems. The author alsoconsiders how the EM algorithm can be scaled to handle the fittingof mixture models to very large databases, as in data miningapplications. This comprehensive, practical guide: * Provides more than 800 references-40% published since 1995 * Includes an appendix listing available mixture software * Links statistical literature with machine learning and patternrecognition literature * Contains more than 100 helpful graphs, charts, and tables Finite Mixture Models is an important resource for both applied andtheoretical statisticians as well as for researchers in the manyareas in which finite mixture models can be used to analyze data.
Topics on Mixture Models and Discriminant Analysis
Author: Kai DengPublisher:
ISBN:
Category : Statistics
Languages : en
Pages : 0
Book Description
Mixture models for clustering and regressions and discriminant analysis are the cornerstones of multivariate statistics and supervised/unsupervised learning research. The structure of data has become increasingly complex in many modern applications including but not limited to computational biology, recommendation systems and text/image analysis. Therefore, it is of great interest to develop methodologies and algorithms for mixture models and discriminant analysis that target the challenges arising from such complex data. In this dissertation, I address three types of challenging supervised and unsupervised topics with novel methodologies and algorithms: (1) tensor data simultaneous clustering and multiway dimension reduction; (2) high-dimensional heterogeneous data in mixture linear regression; (3) multivariate and multi-label response classification in high dimensions. The three chapters are elaborated as follows. In the form of multi-dimensional arrays, tensor data have become increasingly prevalent in modern scientific studies and biomedical applications such as computational biology, brain imaging analysis, and process monitoring system. These data are intrinsically heterogeneous with complex dependencies and structure. Therefore, ad-hoc dimension reduction methods on tensor data may lack statistical efficiency and can obscure essential findings. Model-based clustering is a cornerstone of multivariate statistics and unsupervised learning; however, existing methods and algorithms are not designed for tensor-variate samples. In the first chapter, we propose a Tensor Envelope Mixture Model (TEMM) for simultaneous clustering and multiway dimension reduction of tensor data. TEMM incorporates tensor-structure-preserving dimension reduction into mixture modeling and drastically reduces the number of free parameters and estimative variability. An EM-type algorithm is developed to obtain likelihood-based estimators of the cluster means and covariances, which are jointly parameterized and constrained onto a series of lower-dimensional subspaces known as the tensor envelopes. We demonstrate the encouraging empirical performance of the proposed method in extensive simulation studies and a real data application in comparison with existing vector and tensor clustering methods. In the second chapter, we consider the problem of finite mixture of linear regressions (MLR) for high-dimensional heterogeneous data where the sample size is much smaller than the number of random variables, which is widely used in many modern applications such as biological science, genetics and engineering. In order to capture the common sparse structure in large heterogeneous data, traditional high-dimensional EM algorithm can be computational intractable thus fail to produce meaningful estimation results. We propose a fast group-penalized EM algorithm (FGEM) for high-dimensional MLR that estimates the regression coefficients from a group sparsity perspective and is computationally efficient and less sensitive to initialization. The statistical property of the proposed algorithm is established without requiring sample-splitting that allows the predictor dimension grows exponentially with the sample size. We demonstrate the encouraging performance of FGEM in numerical studies in comparison with traditional high-dimensional EM algorithms. The problem of classifying multiple categorical responses is pervasive in modern machine learning and statistics, with diverse applications in fields such as bioinformatics and image classification. The third chapter investigates linear discriminant analysis (LDA) with high-dimensional predictors and multiple multi-class responses. Specifically, we examine two different classification scenarios under the bivariate LDA model: joint classification of the two responses and conditional classification of one response while observing the other. To achieve optimal classification rules for both scenarios, we introduce two novel tensor formulations of the discriminant coefficients and corresponding penalties. For joint classification, we propose an overlapping group lasso penalty and a blockwise coordinate descent algorithm to efficiently compute joint tensor discriminant coefficients. For conditional classification, we utilize an alternating direction method of multipliers (ADMM) algorithm to compute tensor discriminant coefficients under new constraints. We extend our method and algorithms to general multivariate responses. Finally, we validate the effectiveness of our approach through simulation studies and real data examples.
Mixture Model-Based Classification
Author: Paul D. McNicholasPublisher: CRC Press
ISBN: 1315356112
Category : Mathematics
Languages : en
Pages : 244
Book Description
"This is a great overview of the field of model-based clustering and classification by one of its leading developers. McNicholas provides a resource that I am certain will be used by researchers in statistics and related disciplines for quite some time. The discussion of mixtures with heavy tails and asymmetric distributions will place this text as the authoritative, modern reference in the mixture modeling literature." (Douglas Steinley, University of Missouri) Mixture Model-Based Classification is the first monograph devoted to mixture model-based approaches to clustering and classification. This is both a book for established researchers and newcomers to the field. A history of mixture models as a tool for classification is provided and Gaussian mixtures are considered extensively, including mixtures of factor analyzers and other approaches for high-dimensional data. Non-Gaussian mixtures are considered, from mixtures with components that parameterize skewness and/or concentration, right up to mixtures of multiple scaled distributions. Several other important topics are considered, including mixture approaches for clustering and classification of longitudinal data as well as discussion about how to define a cluster Paul D. McNicholas is the Canada Research Chair in Computational Statistics at McMaster University, where he is a Professor in the Department of Mathematics and Statistics. His research focuses on the use of mixture model-based approaches for classification, with particular attention to clustering applications, and he has published extensively within the field. He is an associate editor for several journals and has served as a guest editor for a number of special issues on mixture models.
Mixture Model-Based Classification
Author: Paul D. McNicholasPublisher: CRC Press
ISBN: 1482225670
Category : Mathematics
Languages : en
Pages : 212
Book Description
"This is a great overview of the field of model-based clustering and classification by one of its leading developers. McNicholas provides a resource that I am certain will be used by researchers in statistics and related disciplines for quite some time. The discussion of mixtures with heavy tails and asymmetric distributions will place this text as the authoritative, modern reference in the mixture modeling literature." (Douglas Steinley, University of Missouri) Mixture Model-Based Classification is the first monograph devoted to mixture model-based approaches to clustering and classification. This is both a book for established researchers and newcomers to the field. A history of mixture models as a tool for classification is provided and Gaussian mixtures are considered extensively, including mixtures of factor analyzers and other approaches for high-dimensional data. Non-Gaussian mixtures are considered, from mixtures with components that parameterize skewness and/or concentration, right up to mixtures of multiple scaled distributions. Several other important topics are considered, including mixture approaches for clustering and classification of longitudinal data as well as discussion about how to define a cluster Paul D. McNicholas is the Canada Research Chair in Computational Statistics at McMaster University, where he is a Professor in the Department of Mathematics and Statistics. His research focuses on the use of mixture model-based approaches for classification, with particular attention to clustering applications, and he has published extensively within the field. He is an associate editor for several journals and has served as a guest editor for a number of special issues on mixture models.
Efficient Methods for Unsupervised Learning
Author: Sida LiuPublisher:
ISBN:
Category : Statistics
Languages : en
Pages : 0
Book Description
Unsupervised Learning is a critical topic in Machine Learning. It studies how a system can learn a particular representation without explicit outputs (i.e labels in Supervised Learning). In this thesis, we introduce two novel and efficient methods in Unsupervised Learning, in Clustering and Dimensionality Reduction. Firstly, we propose a novel clustering algorithm for a variant of classic Gaussian Mixture Model (GMM), where the data is corrupted by outliers sampled uniformly in the space, which we call GMM with a uniform background. Robust loss minimization is the backbone of the proposed algorithm and it performs well in clustering GMM with a uniform background. We also prove theoretical guarantees that the algorithm obtains good clustering with high probability. We support the efficiency and effectiveness of our algorithm with experiments on synthetic and real datasets. The investigation on high dimensional data of the first clustering algorithm mentioned above motivates us to study ways to combine together Dimensionality Reduction and Clustering. In this respect we propose a generic framework for Dimensionality Reduction and Clustering based on Manifold Optimization, which can learn the dimension reduction and clustering parameters simultaneously. The clustering framework studied in this work is a Gaussian Mixture Model and the projection functions are Linear Projection and a simple Neural Network.
Model-Based Clustering and Classification for Data Science
Author: Charles BouveyronPublisher: Cambridge University Press
ISBN: 1108640591
Category : Mathematics
Languages : en
Pages : 447
Book Description
Cluster analysis finds groups in data automatically. Most methods have been heuristic and leave open such central questions as: how many clusters are there? Which method should I use? How should I handle outliers? Classification assigns new observations to groups given previously classified observations, and also has open questions about parameter tuning, robustness and uncertainty assessment. This book frames cluster analysis and classification in terms of statistical models, thus yielding principled estimation, testing and prediction methods, and sound answers to the central questions. It builds the basic ideas in an accessible but rigorous way, with extensive data examples and R code; describes modern approaches to high-dimensional data and networks; and explains such recent advances as Bayesian regularization, non-Gaussian model-based clustering, cluster merging, variable selection, semi-supervised and robust classification, clustering of functional data, text and images, and co-clustering. Written for advanced undergraduates in data science, as well as researchers and practitioners, it assumes basic knowledge of multivariate calculus, linear algebra, probability and statistics.
Mixture Models
Author: Weixin YaoPublisher: CRC Press
ISBN: 1040009875
Category : Mathematics
Languages : en
Pages : 398
Book Description
Mixture models are a powerful tool for analyzing complex and heterogeneous datasets across many scientific fields, from finance to genomics. Mixture Models: Parametric, Semiparametric, and New Directions provides an up-to-date introduction to these models, their recent developments, and their implementation using R. It fills a gap in the literature by covering not only the basics of finite mixture models, but also recent developments such as semiparametric extensions, robust modeling, label switching, and high-dimensional modeling. Features Comprehensive overview of the methods and applications of mixture models Key topics include hypothesis testing, model selection, estimation methods, and Bayesian approaches Recent developments, such as semiparametric extensions, robust modeling, label switching, and high-dimensional modeling Examples and case studies from such fields as astronomy, biology, genomics, economics, finance, medicine, engineering, and sociology Integrated R code for many of the models, with code and data available in the R Package MixSemiRob Mixture Models: Parametric, Semiparametric, and New Directions is a valuable resource for researchers and postgraduate students from statistics, biostatistics, and other fields. It could be used as a textbook for a course on model-based clustering methods, and as a supplementary text for courses on data mining, semiparametric modeling, and high-dimensional data analysis.
Python Data Science Handbook
Author: Jake VanderPlasPublisher: "O'Reilly Media, Inc."
ISBN: 1491912138
Category : Computers
Languages : en
Pages : 743
Book Description
For many researchers, Python is a first-class tool mainly because of its libraries for storing, manipulating, and gaining insight from data. Several resources exist for individual pieces of this data science stack, but only with the Python Data Science Handbook do you get them all—IPython, NumPy, Pandas, Matplotlib, Scikit-Learn, and other related tools. Working scientists and data crunchers familiar with reading and writing Python code will find this comprehensive desk reference ideal for tackling day-to-day issues: manipulating, transforming, and cleaning data; visualizing different types of data; and using data to build statistical or machine learning models. Quite simply, this is the must-have reference for scientific computing in Python. With this handbook, you’ll learn how to use: IPython and Jupyter: provide computational environments for data scientists using Python NumPy: includes the ndarray for efficient storage and manipulation of dense data arrays in Python Pandas: features the DataFrame for efficient storage and manipulation of labeled/columnar data in Python Matplotlib: includes capabilities for a flexible range of data visualizations in Python Scikit-Learn: for efficient and clean Python implementations of the most important and established machine learning algorithms