Duy Nhat Phan, Hoai An Le Thi, Tao Pham Dinh, Efficient Bi-level Variable Selection and Application to Estimation of Multiple Covariance Matrices

Abstract: Variable selection plays an important role in analyzing high dimensional data. When the data possesses certain group structures in which individual variables are also meaningful scientifically, we are naturally interested in selecting important groups as well as important variables. We introduce a new regularization by combining the l_{p,0}-norm and l_0-norm for bi-level variable selection. Using an appropriate DC (Difference of Convex functions) approximation, the resulting problem can be solved by DC Algorithm. As an application, we implement the proposed algorithm for estimating multiple covariance matrices sharing some common structures such as the locations or weights of non-zero elements. The experimental results on both simulated and real datasets demonstrate the efficiency of our algorithm.

 

Keywords: Sparse group, Variable selection, Covariance matrix, DC programming, DCA 

 

Citation: Phan D.N., Le Thi H.A., Pham D.T. (2017) Efficient Bi-level Variable Selection and Application to Estimation of Multiple Covariance Matrices. In: Kim J., Shim K., Cao L., Lee JG., Lin X., Moon YS. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2017. Lecture Notes in Computer Science, vol 10234, pp. 304-316, Springer, Cham.

 

Download link

Variable selection plays an important role in analyzing high dimensional data. When the data possesses certain group structures in which individual variables are also meaningful scientifically, we are naturally interested in selecting important groups as well as important variables. We introduce a new regularization by combining the p,0-norm and 0-norm for bi-level variable selection. Using an appropriate DC (Difference of Convex functions) approximation, the resulting problem can be solved by DC Algorithm. As an application, we implement the proposed algorithm for estimating multiple covariance matrices sharing some common structures such as the locations or weights of non-zero elements. The experimental results on both simulated and real datasets demonstrate the efficiency of our algorithm.