glmmPen: High Dimensional Penalized Generalized Linear Mixed Models

University of North Carolina Chapel Hill

hmheiling@gmail.com

University of North Carolina Chapel Hill

nur2@email.unc.edu

University of North Carolina Chapel Hill

quefeng@email.unc.edu

University of North Carolina Chapel Hill

ibrahim@bios.unc.edu

Abstract

Generalized linear mixed models (GLMMs) are widely used in research for their ability to model correlated outcomes with non-Gaussian conditional distributions. The proper selection of fixed and random effects is a critical part of the modeling process since model misspecification may lead to significant bias. However, the joint selection of fixed and random effects has historically been limited to lower-dimensional GLMMs, largely due to the use of criterion-based model selection strategies. Here we present the R package glmmPen, one of the first to select fixed and random effects in higher dimension using a penalized GLMM modeling framework. Model parameters are estimated using a Monte Carlo Expectation Conditional Minimization (MCECM) algorithm, which leverages Stan and RcppArmadillo for increased computational efficiency. Our package supports the Binomial, Gaussian, and Poisson families and multiple penalty functions. In this manuscript we discuss the modeling procedure, estimation scheme, and software implementation through application to a pancreatic cancer subtyping study. Simulation results show our method has good performance in selecting both the fixed and random effects in high dimensional GLMMs.

Archila, Felipe Acosta. 2020. : Maximum Likelihood Estimation for Generalized Linear Mixed Models. https://CRAN.R-project.org/package=mcemGLM.

Bates, Douglas, Martin Mächler, Ben Bolker, and Steve Walker. 2015. “Fitting Linear Mixed-Effects Models Using Lme4.” Journal of Statistical Software 67 (1): 1–48. https://doi.org/10.18637/jss.v067.i01.

Bolker, Benjamin M, Mollie E Brooks, Connie J Clark, Shane W Geange, John R Poulsen, M Henry H Stevens, and Jada-Simone S White. 2009. “Generalized Linear Mixed Models: A Practical Guide for Ecology and Evolution.” Trends in Ecology & Evolution 24 (3): 127–35. https://doi.org/10.1016/j.tree.2008.10.008.

Bondell, Howard D, Arun Krishna, and Sujit K Ghosh. 2010. “Joint Variable Selection for Fixed and Random Effects in Linear Mixed-Effects Models.” Biometrics 66 (4): 1069–77. https://doi.org/10.1111/j.1541-0420.2010.01391.x.

Booth, James G, and James P Hobert. 1999. “Maximizing Generalized Linear Mixed Model Likelihoods with an Automated Monte Carlo EM Algorithm.” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 61 (1): 265–85. https://doi.org/10.1111/1467-9868.00176.

Breheny, Patrick, and Jian Huang. 2011. “Coordinate Descent Algorithms for Nonconvex Penalized Regression, with Applications to Biological Feature Selection.” Annals of Applied Statistics 5 (1): 232–53. https://doi.org/10.1214/10-AOAS388.

———. 2015. “Group Descent Algorithms for Nonconvex Penalized Linear and Logistic Regression Models with Grouped Predictors.” Statistics and Computing 25 (2): 173–87. https://doi.org/10.1007/s11222-013-9424-2.

Carpenter, Bob, Andrew Gelman, Matthew D Hoffman, Daniel Lee, Ben Goodrich, Michael Betancourt, Marcus Brubaker, Jiqiang Guo, Peter Li, and Allen Riddell. 2017. “Stan: A Probabilistic Programming Language.” Journal of Statistical Software 76 (1). https://doi.org/10.18637/jss.v076.i01.

Chen, Zhen, and David B Dunson. 2003. “Random Effects Selection in Linear Mixed Models.” Biometrics 59 (4): 762–69. https://doi.org/10.1111/j.0006-341X.2003.00089.x.

Dean, CB, and Jason D Nielsen. 2007. “Generalized Linear Mixed Models: A Review and Some Extensions.” Lifetime Data Analysis 13: 497–512. https://doi.org/10.1007/s10985-007-9065-x.

Delattre, Maud, Marc Lavielle, Marie-Anne Poursat, et al. 2014. “A Note on BIC in Mixed-Effects Models.” Electronic Journal of Statistics 8 (1): 456–75. https://doi.org/10.1214/14-EJS890.

Donohue, MC, R Overholser, R Xu, and F Vaida. 2011. “Conditional Akaike Information Under Generalized Linear and Proportional Hazards Mixed Models.” Biometrika 98 (3): 685–700. https://doi.org/10.1093/biomet/asr023.

Eddelbuettel, Dirk, and Romain François. 2011. “: Seamless r and c++ Integration.” Journal of Statistical Software 40 (8): 1–18. http://www.jstatsoft.org/v40/i08/.

Eddelbuettel, Dirk, and Conrad Sanderson. 2014. “: Accelerating r with High-Performance c++ Linear Algebra.” Computational Statistics and Data Analysis 71: 1054–63. https://doi.org/10.1016/j.csda.2013.02.005.

Fan, Yingying, and Runze Li. 2012. “Variable Selection in Linear Mixed Effects Models.” Annals of Statistics 40 (4): 2043. https://doi.org/10.1214/12-AOS1028.

Feaster, Daniel J, Susan Mikulich-Gilbertson, and Ahnalee M Brincks. 2011. “Modeling Site Effects in the Design and Analysis of Multi-Site Trials.” The American Journal of Drug and Alcohol Abuse 37 (5): 383–91. https://doi.org/10.3109/00952990.2011.600386.

Fitzmaurice, Garrett M, Nan M Laird, and James H Ware. 2012. Applied Longitudinal Analysis. 2nd ed. Vol. 998. John Wiley & Sons. https://doi.org/10.1002/9781119513469.

Friedman, Jerome, Trevor Hastie, and Rob Tibshirani. 2010. “Regularization Paths for Generalized Linear Models via Coordinate Descent.” Journal of Statistical Software 33 (1): 1–22. https://www.jstatsoft.org/v33/i01/.

Garcia, Ramon I, Joseph G Ibrahim, and Hongtu Zhu. 2010. “Variable Selection for Regression Models with Missing Data.” Statistica Sinica 20 (1): 149. https://pubmed.ncbi.nlm.nih.gov/20336190/.

Givens, Geof H, and Jennifer A Hoeting. 2012. “Computational Statistics.” In, 2nd ed. Vol. 703. John Wiley & Sons. https://doi.org/10.1111/j.1467-985X.2006.00430_5.x.

Groll, Andreas. 2017. glmmLasso: Variable Selection for Generalized Linear Mixed Models by L1-Penalized Estimation. https://CRAN.R-project.org/package=glmmLasso.

Gurka, Matthew J, Lloyd J Edwards, and Keith E Muller. 2011. “Avoiding Bias in Mixed Model Inference for Fixed Effects.” Statistics in Medicine 30 (22): 2696–2707. https://doi.org/10.1002/sim.4293.

Hadfield, Jarrod D. 2010. “MCMC Methods for Multi-Response Generalized Linear Mixed Models: The r Package.” Journal of Statistical Software 33 (2): 1–22. https://www.jstatsoft.org/v33/i02/.

Hoffman, Matthew D, and Andrew Gelman. 2014. “The No-u-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo.” Journal of Machine Learning Research 15 (1): 1593–623. https://dl.acm.org/doi/abs/10.5555/2627435.2638586.

Ibrahim, Joseph G, Hongtu Zhu, Ramon I Garcia, and Ruixin Guo. 2011. “Fixed and Random Effects Selection in Mixed Effects Models.” Biometrics 67 (2): 495–503. https://doi.org/10.1111/j.1541-0420.2010.01463.x.

Kane, Michael J., John Emerson, and Stephen Weston. 2013. “Scalable Strategies for Computing with Massive Data.” Journal of Statistical Software 55 (14): 1–19. http://www.jstatsoft.org/v55/i14/.

Kleinman, Ken, Ross Lazarus, and Richard Platt. 2004. “A Generalized Linear Mixed Models Approach for Detecting Incident Clusters of Disease in Small Areas, with an Application to Biological Terrorism.” American Journal of Epidemiology 159 (3): 217–24. https://doi.org/10.1093/aje/kwh029.

Langford, Ian H. 1994. “Using a Generalized Linear Mixed Model to Analyze Dichotomous Choice Contingent Valuation Data.” Land Economics, 507–14. https://doi.org/10.2307/3146644.

Lorah, Julie, and Andrew Womack. 2019. “Value of Sample Size for Computation of the Bayesian Information Criterion (BIC) in Multilevel Modeling.” Behavior Research Methods 51 (1): 440–50. https://doi.org/10.3758/s13428-018-1188-3.

Ma, Siyuan, Shuji Ogino, Princy Parsana, Reiko Nishihara, Zhirong Qian, Jeanne Shen, Kosuke Mima, et al. 2018. “Continuity of Transcriptomes Among Colorectal Cancer Subtypes Based on Meta-Analysis.” Genome Biology 19 (1): 142. https://doi.org/10.1186/s13059-018-1511-4.

Misztal, IJJOaB. 2008. “Reliable Computing in Estimation of Variance Components.” Journal of Animal Breeding and Genetics 125 (6): 363–70. https://doi.org/10.1111/j.1439-0388.2008.00774.x.

Moffitt, Richard A, Raoud Marayati, Elizabeth L Flate, Keith E Volmar, S Gabriela Herrera Loeza, Katherine A Hoadley, Naim U Rashid, et al. 2015. “Virtual Microdissection Identifies Distinct Tumor- and Stroma-Specific Subtypes of Pancreatic Ductal Adenocarcinoma.” Nature Genetics 47 (10): 1168. https://doi.org/10.1038/ng.3398.

Pajor, Anna. 2017. “Estimating the Marginal Likelihood Using the Arithmetic Mean Identity.” Bayesian Analysis 12 (1): 261–87. https://doi.org/10.1214/16-BA1001.

Patil, Prasad, and Giovanni Parmigiani. 2018. “Training Replicable Predictors in Multiple Studies.” Proceedings of the National Academy of Sciences 115 (11): 2578–83. https://doi.org/10.1073/pnas.1708283115.

Pinheiro, Jose, Douglas Bates, Saikat DebRoy, Deepayan Sarkar, and R Core Team. 2021. : Linear and Nonlinear Mixed Effects Models. https://CRAN.R-project.org/package=nlme.

Rashid, Naim U, Quefeng Li, Jen Jen Yeh, and Joseph G Ibrahim. 2020. “Modeling Between-Study Heterogeneity for Improved Replicability in Gene Signature Selection and Clinical Prediction.” Journal of the American Statistical Association 115 (531): 1125–38. https://doi.org/10.1080/01621459.2019.1671197.

Riester, Markus, Wei Wei, Levi Waldron, Aedin C Culhane, Lorenzo Trippa, Esther Oliva, Sung-hoon Kim, et al. 2014. “Risk Prediction for Late-Stage Ovarian Cancer by Meta-Analysis of 1525 Patient Samples.” JNCI: Journal of the National Cancer Institute 106 (5). https://doi.org/10.1093/jnci/dju048.

Roberts, Gareth O, and Jeffrey S Rosenthal. 2009. “Examples of Adaptive MCMC.” Journal of Computational and Graphical Statistics 18 (2): 349–67. https://doi.org/10.1002/wics.1307.

SAS Institute Inc. 2008. SAS/STAT Software, Version 9.2. Cary, NC. http://www.sas.com/.

Schmidt-Catran, Alexander W, and Malcolm Fairbrother. 2016. “The Random Effects in Multilevel Models: Getting Them Wrong and Getting Them Right.” European Sociological Review 32 (1): 23–38. https://doi.org/10.1093/esr/jcv090.

Stan Development Team. 2020. “: The r Interface to Stan.” http://mc-stan.org/.

Szyszkowicz, MIECZYSŁAW. 2006. “Use of Generalized Linear Mixed Models to Examine the Association Between Air Pollution and Health Outcomes.” International Journal of Occupational Medicine and Environmental Health 19 (4): 224–27. https://doi.org/10.2478/v10001-006-0032-7.

Thompson, Jennifer A, Katherine L Fielding, Calum Davey, Alexander M Aiken, James R Hargreaves, and Richard J Hayes. 2017. “Bias and Inference from Misspecified Mixed-Effect Models in Stepped Wedge Trial Analysis.” Statistics in Medicine 36 (23): 3670–82. https://doi.org/10.1002/sim.7348.

Weinstein, John N, Eric A Collisson, Gordon B Mills, Kenna R Shaw, Brad A Ozenberger, Kyle Ellrott, Ilya Shmulevich, Chris Sander, and Joshua M Stuart. 2013. “The Cancer Genome Atlas Pan-Cancer Analysis Project.” Nature Genetics 45 (10): 1113–20. https://doi.org/10.1038/ng.2764.

Wickham, Hadley. 2016. Ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York. https://ggplot2.tidyverse.org.