SIMEXBoost: An R package for Analysis of High-Dimensional Error-Prone Data Based on Boosting Method

Department of Statistics, National Chengchi University

lchen723@nccu.edu.tw

Department of Statistics, National Chengchi University

1135427976@qq.com

Abstract

Boosting is a powerful statistical learning method. Its key feature is the ability to derive a strong learner from simple yet weak learners by iteratively updating the learning results. Moreover, boosting algorithms have been employed to do variable selection and estimation for regression models. However, measurement error usually appears in covariates. Ignoring measurement error can lead to biased estimates and wrong inferences. To the best of our knowledge, few packages have been developed to address measurement error and variable selection simultaneously by using boosting algorithms. In this paper, we introduce an R package SIMEXBoost, which covers some widely used regression models and applies the simulation and extrapolation method to deal with measurement error effects. Moreover, the package SIMEXBoost enables us to do variable selection and estimation for high-dimensional data under various regression models. To assess the performance and illustrate the features of the package, we conduct numerical studies.

Agresti, A. 2012. Categorical Data Analysis. New York: Wiley.

Alfaro, E., M. Gamez, L. Garcia N. Guo, A. Albano, M. Sciandra, and A. Plaia. 2023. Adabag: Applies Multiclass AdaBoost.M1, SAMME and Bagging. https://cran.r-project.org/package=adabag.

Bartoszek, Krzysztof. 2019. GLSME: Generalized Least Squares with Measurement Error. https://cran.r-project.org/package=GLSME.

Boyd, S., and L. Vandenberghe. 2004. Convex Optimization. New York: Cambridge University Press.

Brown, B., C. J. Miller, and J. Wolfson. 2017. “ThrEEBoost: Thresholded Boosting for Variable Selection and Prediction via Estimating Equations.” Journal of Computational and Graphical Statistics 26: 579–88.

Carroll, R. J., D. Ruppert, L. A. Stefanski, and C. M. Crainiceanu. 2006. Measurement Error in Nonlinear Model. New York: CRC Press.

Chen, L.-P. 2021. “Feature Screening Based on Distance Correlation for Ultrahigh-Dimensional Censored Data with Covariate Measurement Error.” Computational Statistics 36: 857–84.

———. 2023a. “A Note of Feature Screening via Rank-Based Coefficient of Correlation.” Biometrical Journal 65: 2100373.

———. 2023b. “BOOME: A Python Package for Handling Misclassified Disease and Ultrahigh-Dimensional Error-Prone Gene Expression Data.” PLOS ONE 17: e0276664.

———. 2023c. “De-Noising Boosting Methods for Variable Selection and Estimation Subject to Error-Prone Variables.” Statistics and Computing 33:38: 1–13.

———. 2023d. “Variable Selection and Estimation for Misclassified Binary Responses and Multivariate Error-Prone Predictors.” Journal of Computational and Graphical Statistics. https://doi.org/10.1080/10618600.2023.2218428.

Chen, L.-P., and B. Qiu. 2023. “Analysis of Length-Biased and Partly Interval-Censored Survival Data with Mismeasured Covariates.” Biometrics 79: 3929–40.

Chen, L.-P., and G. Y. Yi. 2021. “Analysis of Noisy Survival Data with Graphical Proportional Hazards Measurement Error Models.” Biometrics 77: 956–69.

Chen, Tianqi, Tong He, Michael Benesty, Vadim Khotilovich, Yuan Tang, Hyunsu Cho, Kailong Chen, et al. 2023. Xgboost: Extreme Gradient Boosting. https://cran.r-project.org/package=xgboost.

Feng, Yang, Jianqing Fan, Diego Franco Saldana, Yichao Wu, and Richard Samworth. 2020. SIS: Sure Independence Screening. https://cran.r-project.org/package=SIS.

Friedman, Jerome, Trevor Hastie, Rob Tibshirani, Balasubramanian Narasimhan, Kenneth Tay, Noah Simon, Junyang Qian, and James Yang. 2023. Glmnet: Lasso and Elastic-Net Regularized Generalized Linear Models. https://cran.r-project.org/package=glmnet.

Greenwell, Brandon, Bradley Boehmke, Jay Cunningham, and GBM Developers. 2022. Gbm: Generalized Boosted Regression Models. https://cran.r-project.org/package=gbm.

Groll, Andreas. 2020. GMMBoost: Likelihood-Based Boosting for Generalized Mixed Models. https://cran.r-project.org/package=GMMBoost.

Hastie, T., R. Tibshirani, and J. Friedman. 2008. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. New York: Springer.

Hofner, Benjamin, Andreas Mayr, Nora Fenske, Janek Thomas, and Matthias Schmid. 2023. gamboostLSS: Boosting Methods for ’GAMLSS’. https://cran.r-project.org/package=glmnet.

Lawless, J. F. 2003. Statistical Models and Methods for Lifetime Data. New York: Wiley.

Lederer, Wolfgang, Heidi Seibold, Helmut Küchenhoff, Chris Lawrence, and Rasmus Froberg Brøndum. 2019. Simex: SIMEX- and MCSIMEX-Algorithm for Measurement Error Models. https://cran.r-project.org/package=simex.

Nab, Linda. 2021. Mecor: Measurement Error Correction in Linear Models with a Continuous Outcome. https://cran.r-project.org/package=mecor.

Qiu, Bangxu, and Li-Pang Chen. 2023. SIMEXBoost: Boosting Method for High-Dimensional Error-Prone Data. https://cran.r-project.org/package=SIMEXBoost.

Shi, Yu, Guolin Ke, Damien Soukhavong, James Lamb, Qi Meng, Thomas Finley, Taifeng Wang, et al. 2023. Lightgbm: Light Gradient Boosting Machine. https://cran.r-project.org/package=lightgbm.

Tibshirani, R. 1996. “Regression Shrinkage and Selection via the LASSO.” Journal of Royal Statistical Society, Series B 58: 267–88.

Wang, Zhu, and Torsten Hothorn. 2023. Bst: Gradient Boosting. https://cran.r-project.org/package=bst.

Wolfson, J. 2011. “EEBOOST: A General Method for Prediction and Variable Selection Based on Estimating Equation.” Journal of the American Statistical Association 106: 296–305.

Xiong, Juan, Wenqing He, and Grace Y. Yi. 2019. Simexaft: Simexaft. https://cran.r-project.org/package=simexaft.

Yi, G. Y. 2017. Statistical Analysis with Measurement Error and Misclassication: Strategy, Method and Application. New York: Springer.

Zhang, Qihuang, and Grace Y. Yi. 2020. augSIMEX: Analysis of Data with Mixed Measurement Error and Misclassification in Covariates. https://cran.r-project.org/package=augSIMEX.

Zou, H. 2006. “The Adaptive Lasso and Its Oracle Properties.” Journal of the American Statistical Association 101: 1418–29.

Zou, H., and T. Hastie. 2005. “Regularization and Variable Selection via the Elastic Net.” Journal of the Royal Statistical Society: Series B 67: 301–20.