Shrinkage Parameter Selection in Generalized Linear and Mixed Models PDF Download

Author: Erin K. Melcon
Publisher:
ISBN: 9781321363388
Category :
Languages : en
Pages :

View

Book Description
Penalized likelihood methods such as lasso, adaptive lasso, and SCAD have been highly utilized in linear models. Selection of the penalty parameter is an important step in modeling with penalized techniques. Traditionally, information criteria or cross validation are used to select the penalty parameter. Although methods of selecting this have been evaluated in linear models, general linear models and linear mixed models have not been so thoroughly explored.This dissertation will introduce a data-driven bootstrap (Empirical Optimal Selection, or EOS) approach for selecting the penalty parameter with a focus on model selection. We implement EOS on selecting the penalty parameter in the case of lasso and adaptive lasso. In generalized linear models we will introduce the method, show simulations comparing EOS to information criteria and cross validation, and give theoretical justification for this approach. We also consider a practical upper bound for the penalty parameter, with theoretical justification. In linear mixed models, we use EOS with two different objective functions; the traditional log-likelihood approach (which requires an EM algorithm), and a predictive approach. In both of these cases, we compare selecting the penalty parameter with EOS to selection with information criteria. Theoretical justification for both objective functions and a practical upper bound for the penalty parameter in the log-likelihood case are given. We also applied our technique to two datasets; the South African heart data (logistic regression) and the Yale infant data (a linear mixed model). For the South African data, we compare the final models using EOS and information criteria via the mean squared prediction error (MSPE). For the Yale infant data, we compare our results to those obtained by Ibrahim et al. (2011).

Author:
Publisher:
ISBN:
Category :
Languages : en
Pages :

View

Book Description
Model selection is important for longitudinal data analysis. But up to date little work has been done on variable selection for generalized linear mixed models (GLMM). In this paper we propose and study a class of variable selection methods. Full likelihood (FL) approach is proposed for simultaneous model selection and parameter estimation. Due to the intensive computation involved in FL approach, Penalized Quasi-Likelihood (PQL) procedure is developed so that model selection in GLMMs can proceed in the framework of linear mixed models. Since the PQL approach will produce biased parameter estimates for sparse binary longitudinal data, Two-stage Penalized Quasi-Likelihood approach (TPQL) is proposed to bias correct PQL in terms of estimation: use PQL to do model selection at the first stage and existing software to do parameter estimation at the second stage. Marginal approach for some special types of data is also developed. A robust estimator of standard error for the fitted parameters is derived based on a sandwich formula. A bias correction is proposed to improve the estimation accuracy of PQL for binary data. The sampling performance of four proposed procedures is evaluated through extensive simulations and their application to real data analysis. In terms of model selection, all of them perform closely. As for parameter estimation, FL, AML and TPQL yield similar results. Compared with FL, the other procedures greatly reduce computational load. The proposed procedures can be extended to longitudinal data analysis involving missing data, and the shrinkage penalty based approach allows them to work even when the number of observations n is less than the number of parameters d.

Author: Ludwig Fahrmeir
Publisher: Springer Science & Business Media
ISBN: 1489900101
Category : Mathematics
Languages : en
Pages : 440

View

Book Description
Concerned with the use of generalised linear models for univariate and multivariate regression analysis, this is a detailed introductory survey of the subject, based on the analysis of real data drawn from a variety of subjects such as the biological sciences, economics, and the social sciences. Where possible, technical details and proofs are deferred to an appendix in order to provide an accessible account for non-experts. Topics covered include: models for multi-categorical responses, model checking, time series and longitudinal data, random effects models, and state-space models. Throughout, the authors have taken great pains to discuss the underlying theoretical ideas in ways that relate well to the data at hand. As a result, numerous researchers whose work relies on the use of these models will find this an invaluable account.

Author: Jiming Jiang
Publisher: Springer Nature
ISBN: 1071612824
Category : Medical
Languages : en
Pages : 343

View

Book Description
This book covers two major classes of mixed effects models, linear mixed models and generalized linear mixed models. It presents an up-to-date account of theory and methods in analysis of these models as well as their applications in various fields. The book offers a systematic approach to inference about non-Gaussian linear mixed models. Furthermore, it includes recently developed methods, such as mixed model diagnostics, mixed model selection, and jackknife method in the context of mixed models. The book is aimed at students, researchers and other practitioners who are interested in using mixed models for statistical data analysis.

Author: Charles E. McCulloch
Publisher: IMS
ISBN: 9780940600546
Category : Mathematics
Languages : en
Pages : 100

View

Book Description
Wiley Series in Probability and Statistics A modern perspective on mixed models The availability of powerful computing methods in recent decades has thrust linear and nonlinear mixed models into the mainstream of statistical application. This volume offers a modern perspective on generalized, linear, and mixed models, presenting a unified and accessible treatment of the newest statistical methods for analyzing correlated, nonnormally distributed data. As a follow-up to Searle's classic, Linear Models, and Variance Components by Searle, Casella, and McCulloch, this new work progresses from the basic one-way classification to generalized linear mixed models. A variety of statistical methods are explained and illustrated, with an emphasis on maximum likelihood and restricted maximum likelihood. An invaluable resource for applied statisticians and industrial practitioners, as well as students interested in the latest results, Generalized, Linear, and Mixed Models features: * A review of the basics of linear models and linear mixed models * Descriptions of models for nonnormal data, including generalized linear and nonlinear models * Analysis and illustration of techniques for a variety of real data sets * Information on the accommodation of longitudinal data using these models * Coverage of the prediction of realized values of random effects * A discussion of the impact of computing issues on mixed models

Author: Ludwig Fahrmeir
Publisher: OUP Oxford
ISBN: 9780199533022
Category : Mathematics
Languages : en
Pages : 0

View

Book Description
Bringing together recent advances in smoothing and semiparametric regression from a Bayesian perspective, this book demonstrates, with worked examples, the application of these statistical methods to a variety of fields including forestry, development economics, medicine and marketing.

Author: Olivia Abena Atutey
Publisher:
ISBN:
Category : Linear models (Statistics)
Languages : en
Pages : 110

View

Book Description
The analyses of correlated, repeated measures, or multilevel data with a Gaussian response are often based on models known as the linear mixed models (LMMs). LMMs are modeled using both fixed effects and random effects. The random intercepts (RI) and random intercepts and slopes (RIS) models are two exceptional cases from the linear mixed models that are taken into consideration. Our primary focus in this dissertation is to propose an approach for simultaneous selection and estimation of fixed effects only in LMMs. This dissertation, inspired by recent research of methods and criteria for model selection, aims to extend a variable selection procedure referred to as minimum approximated information criterion (MIC) of Su et al. (2018). Our contribution presents further use of the MIC for variable selection and sparse estimation in LMMs. Thus, we design a penalized log-likelihood procedure referred to as the minimum approximated information criterion for LMMs (lmmMAIC), which is used to find a parsimonious model that better generalizes data with a group structure. Our proposed lmmMAIC method enforces variable selection and sparse estimation simultaneously by adding a penalty term to the negative log-likelihood of the linear mixed model. The method differs from existing regularized methods mainly due to the penalty parameter and the penalty function.With regards to the penalty function, the lmmMAIC mimics the traditional Bayesian information criterion (BIC)-based best subset selection (BSS) method but requires a continuous or smooth approximation to the L0 norm penalty of BSS. In this context, lmmMAIC performs sparse estimation by optimizing an approximated information criterion, which substantially requires approximating that L0 norm penalty of BSS with a continuous unit dent function. A unit dent function, motivated by bump functions called mollifiers (Friedrichs, 1944), is an even continuous function with a [0, 1] range. Among several unit dent functions, incorporating a hyperbolic tangent function is most preferred. The approximation changes the discrete nature of the L0 norm penalty of BSS to a continuous or smooth one making our method less computationally expensive. Besides, the hyperbolic tangent function has a simple form and it is much easier to compute its derivatives. This shrinkage-based method fits a linear mixed model containing all p predictors instead of comparing and selecting a correct sub-model across 2p candidate models. On this account, the lmmMAIC is feasible for high-dimensional data. The replacement, however, does not enforce sparsity since the hyperbolic tangent function is not singular at its origin. To better handle this issue, a reparameterization trick of the regression coefficients is needed to achieve sparsity.For a finite number of parameters, numerical investigations demonstrated by Shi and Tsai (2002) prove that traditional information criterion (IC)-based procedure like BIC can consistently identify a model. Following these suggestions of consistent variable selection and computational efficiency, we maintain the BIC fixed penalty parameter. Thus, our newly proposed procedure is free of using the frequently applied practices such as generalized cross validation (GCV) in selecting an optimal penalty parameter for our penalized likelihood framework. The lmmMAIC enjoys less computational time compared to other regularization methods.We formulate the lmmMAIC procedure as a smooth optimization problem and seek to solve for the fixed effects parameters by minimizing the penalized log-likelihood function. The implementation of the lmmMAIC involves an initial step of using the simulated annealing algorithm to obtain estimates. We proceed using these estimates as starting values by applying the modified Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm until convergence. After this step, we plug estimates obtained from the modified BFGS into the reparameterized hyperbolic tangent function to obtain our fixed effects estimates. Alternatively, the optimization of the penalized log-likelihood can be solved using generalized simulation annealing.Our research explores the performance and asymptotic properties of the lmmMAIC method by conducting extensive simulation studies using different model settings. The numerical results of our simulations for our proposed variable selection and estimation method are compared to other standard LMMs shrinkage-based methods such as Lasso, ridge, and elastic net. The results provide evidence that lmmMAIC is more consistent and efficient than the existing shrinkage-based methods under study. Furthermore, two applications with real-life examples are illustrated to evaluate the effectiveness of the lmmMAIC method.

Author: Eugene Demidenko
Publisher: John Wiley & Sons
ISBN: 1118091574
Category : Mathematics
Languages : en
Pages : 768

View

Book Description
Praise for the First Edition “This book will serve to greatly complement the growing number of texts dealing with mixed models, and I highly recommend including it in one’s personal library.” —Journal of the American Statistical Association Mixed modeling is a crucial area of statistics, enabling the analysis of clustered and longitudinal data. Mixed Models: Theory and Applications with R, Second Edition fills a gap in existing literature between mathematical and applied statistical books by presenting a powerful examination of mixed model theory and application with special attention given to the implementation in R. The new edition provides in-depth mathematical coverage of mixed models’ statistical properties and numerical algorithms, as well as nontraditional applications, such as regrowth curves, shapes, and images. The book features the latest topics in statistics including modeling of complex clustered or longitudinal data, modeling data with multiple sources of variation, modeling biological variety and heterogeneity, Healthy Akaike Information Criterion (HAIC), parameter multidimensionality, and statistics of image processing. Mixed Models: Theory and Applications with R, Second Edition features unique applications of mixed model methodology, as well as: Comprehensive theoretical discussions illustrated by examples and figures Over 300 exercises, end-of-section problems, updated data sets, and R subroutines Problems and extended projects requiring simulations in R intended to reinforce material Summaries of major results and general points of discussion at the end of each chapter Open problems in mixed modeling methodology, which can be used as the basis for research or PhD dissertations Ideal for graduate-level courses in mixed statistical modeling, the book is also an excellent reference for professionals in a range of fields, including cancer research, computer science, and engineering.

Author: Xian Liu
Publisher: Elsevier
ISBN: 0128014822
Category : Mathematics
Languages : en
Pages : 531

View

Book Description
Methods and Applications of Longitudinal Data Analysis describes methods for the analysis of longitudinal data in the medical, biological and behavioral sciences. It introduces basic concepts and functions including a variety of regression models, and their practical applications across many areas of research. Statistical procedures featured within the text include: descriptive methods for delineating trends over time linear mixed regression models with both fixed and random effects covariance pattern models on correlated errors generalized estimating equations nonlinear regression models for categorical repeated measurements techniques for analyzing longitudinal data with non-ignorable missing observations Emphasis is given to applications of these methods, using substantial empirical illustrations, designed to help users of statistics better analyze and understand longitudinal data. Methods and Applications of Longitudinal Data Analysis equips both graduate students and professionals to confidently apply longitudinal data analysis to their particular discipline. It also provides a valuable reference source for applied statisticians, demographers and other quantitative methodologists. From novice to professional: this book starts with the introduction of basic models and ends with the description of some of the most advanced models in longitudinal data analysis Enables students to select the correct statistical methods to apply to their longitudinal data and avoid the pitfalls associated with incorrect selection Identifies the limitations of classical repeated measures models and describes newly developed techniques, along with real-world examples.

Author: Jiming Jiang
Publisher: CRC Press
ISBN: 1498700462
Category : Mathematics
Languages : en
Pages : 252

View

Book Description
Large sample techniques are fundamental to all fields of statistics. Mixed effects models, including linear mixed models, generalized linear mixed models, non-linear mixed effects models, and non-parametric mixed effects models are complex models, yet, these models are extensively used in practice. This monograph provides a comprehensive account of asymptotic analysis of mixed effects models. The monograph is suitable for researchers and graduate students who wish to learn about asymptotic tools and research problems in mixed effects models. It may also be used as a reference book for a graduate-level course on mixed effects models, or asymptotic analysis.