Using Synthetic Data and Dimensionality Reduction in High-Dimensional Classi cation via Logistic Regression

Traditional logistic regression is plugged with degenerates and violent behavior in high-dimensional classi cation, because of the problem of non-invertible matrices in estimating model parameters. In this paper, to overcome the high-dimensionality of data, we introduce two new algorithms. First, we improve the e ciency of nite population Bayesian bootstrapping logistic regression classi er by the rule of majority vote. Second, using simple random sampling without replacement and selecting a smaller numberof covariates rather than the sample size and applying traditional logistic regression, we introduce the other new algorithm for high-dimensional binary classi cation. We compare the proposed algorithms with the regularized logistic regression models using simulated data.

Keywords:

High-dimensional classi cation , Logistic regression classi er , Dimensionality reduction , Fi-nite population Bayesian bootstrapping.