TwoSampleTest.HD: An R Package for the Two-Sample Problem with High-Dimensional Data

Instituto Español de Oceanografía (IEO, CSIC), Centro Oceanográfico de Vigo

marta.cousido@ieo.csic.es

CINBIO, Universidade de Vigo, SiDOR Research Group

jacobo@uvigo.es

Abstract

The two-sample problem refers to the comparison of two probability distributions via two independent samples. With high-dimensional data, such comparison is performed along a large number \(p\) of possibly correlated variables or outcomes. In genomics, for instance, the variables may represent gene expression levels for \(p\) locations, recorded for two (usually small) groups of individuals. In this paper we introduce TwoSampleTest.HD, a new R package to test for the equal distribution of the \(p\) outcomes. Specifically, TwoSampleTest.HD implements the tests recently proposed by (Cousido-Rocha, Uña-Álvarez, and Hart 2019) for the low sample size, large dimensional setting. These tests take the possible dependence among the \(p\) variables into account, and work for sample sizes as small as two. The tests are based on the distance between the empirical characteristic functions of the two samples, when averaged along the \(p\) locations. Different options to estimate the variance of the test statistic under dependence are allowed. The package TwoSampleTest.HD provides the user with individual permutation \(p\)-values too, so feature discovery is possible when the null hypothesis of equal distribution is rejected. We illustrate the usage of the package through the analysis of simulated and real data, where results provided by alternative approaches are considered for comparison purposes. In particular, benefits of the implemented tests relative to ordinary multiple comparison procedures are highlighted. Practical recommendations are given.

Benjamini, Y, and Y Hochberg. 1995. “Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing.” Journal of the Royal Statistical Society B 57: 289–300.

Biswas, Moo, and Aoo Koo Gosh. 2014. “A Nonparametric Two-Sample Test Applicable to High Dimensional Data.” Journal of Multivariate Analysis 123: 160–71.

Biswas, Moo, Moo Mukhopadhyay, and Aooo Kooo Ghosh. 2014. “A Distribution-Free Two-Sample Run Tests Applicable to High-Dimensional Data.” Biometrika 101: 913–26.

Bosq, D. 1998. Nonparametric Statistics for Stochastic Processes: Estimation and Prediction. Second Edition. Springer-Verlag, New York.

Carlstein, Eoo. 1996. “The Use of Subseries Values for Estimating the Variance of a General Statistic from a Stationary Sequence.” Annals of Statistics 4: 1171–79.

Cousido-Rocha, Marta, and Jac de Uña-Álvarez. 2022. “Equalden.HD: An R Package for Testing the Equality of a High Dimensional Set of Densities.” Computer Methods and Programs in Biomedicine 217: 106694. https://doi.org/https://doi.org/10.1016/j.cmpb.2022.106694.

Cousido-Rocha, Marta, Jac de Uña-Álvarez, and Je Hart. 2019. “A Two-Sample Test for the Equality of Distributions for High-Dimensional Data.” Journal of Multivariate Analysis 174: 104537. https://doi.org/https://doi.org/10.1016/j.jmva.2019.104537.

Cousido-Rocha, Marta, Jacobo de Uña-Álvarez, and Sebastian Döhler. 2021. “Multiple Comparison Procedures for Discrete Uniform and Homogeneous Tests.” Journal of the Royal Statistical Society: Series C (Applied Statistics) 71: 219–43. https://doi.org/https://doi.org/10.1111/rssc.12529.

Dehling, Hoo, Roo Fried, Ioo Garcia, and Moo Wendler. 2015. “Change-point Detection Under Dependence Based on Two-Sample U-Statistics. In: Dawson D., R. Kulik, M. Ould Haye, B. Szyszkowicz, Y. Zhao (eds) Asymptotic Laws and Methods in Stochastics. Fields Institute Communications, vol 76. Springer, New York, NY.”

Doukhan, P. 1995. Mixing: Properties and Examples. Springer-Verlag, New York.

Dudoit, Sandrine, and Mark J van der Laan. 2007. Multiple Testing Procedures and Applications to Genomics. Spinger, New York.

Fisher, Roo Aooo. 1934. Statistical Methods for Research Workers. Fourth Edition. Oliver; Boyd, Edinburgh.

Gibbons, Joo D, and S Chakraborti. 1992. Nonparametric Statistical Inference. Third Edition. Marcel Dekker, Inc, New York.

Hall, Peter, and Jiashun Jin. 2008. “Properties of Higher Criticism Under Strong Dependence.” The Annals of Statistics 36: 381–402.

Hedenfalk, I, D Duggan, Y Chen, M Radmacher, M Bittner, R Simon, P Meltzer, et al. 2001. “Gene-Expression Profiles in Hereditary Breast Cancer.” New England Journal of Medicine 344: 539–48.

Levene, Hooo. 1960. “Robust Tests for Equality of Variances.” In Contributions to Probability and Statistics, edited by Iooo Olkin, 278–92. Palo Alto, Calif.: Stanford University Press.

Liu, Zoo, Xooo Xia, and Woo Zhou. 2015. “A Test for Equality of Two Distributions via Jackknife Empirical Likelihood and Characteristic Functions.” Computational Statistics and Data Analysis 92: 97–114.

Martínez-Camblor, Pablo, and Jacobo de Uña-Álvarez. 2009. “Nonparametric k-Sample Tests: Density Functions Vs Distribution Functions.” Computational Statistics and Data Analysis 53: 3344–57.

Mondal, Poo Kooo, Mooo Biswas, and Aoo Kooo Ghosh. 2015. “On High Dimensional Two-Sample Tests Based on Nearest Neighbors.” Journal of Multivariate Analysis 141: 168–78.

Neumann, Moo Hooo, and Eoo Paparoditis. 2000.“ On bootstrapping \(L_2\)-type statistics in density testing.” Statistics \(\&\) Probability Letters 50: 137–47.

Politis, Do No, and Ho White. 2004. “Automatic Block-Length Selection for the Dependent Bootstrap.” American Economic Review 23: 53–70.

Stouffer, Soo Aoo, Eoo Aoo Suchman, Loo Coo DeVinney, Soo Aoo Star, and Roo Moo Williams. 1949. The American Soldier. Adjustment During Army Life. Princeton University Press, England.

Wei, Soo, Coo Lee, Looo Wichers, and Joo Soo Marron. 2016. “Direction-Projection-Permutation for High-Dimensional Hypothesis Tests.” Journal of Computational and Graphical Statistics 25: 549–69.

Zhang, Hong, Jiashun Jin, and Zheyang Wu. 2020. “Distributions and Power of Optimal Signal-Detection Statistics in Finite Case.” IEEE Transactions on Signal Processing 68: 1021–33. https://doi.org/https://doi.org/10.1109/TSP.2020.2967179.