A MODIFIED CORRELATION BASED REGULARIZATION TECHNIQUE FOR REGRESSION ESTIMATION AND FEATURE SELECTION

Abstract

Variable selection is important for making sense with (ultra) high-dimensional data. Penalized least squares such as the LASSO, elastic-net and the correlation based elastic-net (L1CP) are popular methods for carrying out variable selection and estimation simultaneously. This study proposes a modified version of the L1CP motivated by reasons similar to that given by Zou and Hastie (2005) where the naïve elastic net was rescaled to give the elastic net. The scaling transformation is derived such that the double shrinkage caused by applying two penalties is undone thereby reducing bias. The derived scaling transformations are found to depend on the correlations among the predictors. A robust worst-case quadratic solver is used to obtain estimates. An evaluation of the proposed method which is referred to as CL1CP alongside the L1CP, LASSO and elastic-net through simulation studies illustrate the advantages of the CL1CP compared to the other alternatives considered especially in correct selection of sparse models. In terms of variable selection, estimation and prediction accuracy the proposed CL1CP performs favourably compared to the L1CP, LASSO and elastic-net especially for “grouped-variables” selection. Results from applications to two real life datasets corroborate the findings from simulation studies.