Tree-structured feature extraction using mutual information.

One of the most informative measures for feature extraction (FE) is mutual information (MI). In terms of MI, the optimal FE creates new features that jointly have the largest dependency on the target class. However, obtaining an accurate estimate of a high-dimensional MI as well as optimizing with respect to it is not always easy, especially when only small training sets are available. In this paper, we propose an efficient tree-based method for FE in which at each step a new feature is created by selecting and linearly combining two features such that the MI between the new feature and the class is maximized. Both the selection of the features to be combined and the estimation of the coefficients of the linear transform rely on estimating 2-D MIs. The estimation of the latter is computationally very efficient and robust. The effectiveness of our method is evaluated on several real-world data sets. The results show that the classification accuracy obtained by the proposed method is higher than that achieved by other FE methods.

A tree-structure mutual information-based feature extraction and its application to EEG-based brain-computer interfacing.

This paper presents a novel algorithm for efficient feature extraction using mutual information (MI). In terms of mutual information, the optimal feature extraction is creating a new feature set from the data which jointly have largest dependency on the target class. However, it is not always easy to get an accurate estimation for high-dimensional MI. In this paper, we propose an efficient method for feature extraction using two-dimensional MI estimates. A new feature is created such that the MI between the new feature and the target class is maximized and the redundancy is minimized. The effectiveness of the proposed algorithm is evaluated by using the classification of EEG signals. The tasks to be discriminated are the imaginative hand movement and the resting state. The results demonstrate that the proposed mutual information-based feature extraction (MIFX) algorithm performed well in several experiments on different subjects and can improve the classification accuracy of the EEG patterns. The results show that the classification accuracy obtained by MIFX is higher than that achieved by full feature set.