Nonparametric data reduction.

A nonparametric data reduction technique is proposed. Its goal is to select samples that are ``representative'' of the entire data set. The technique is iterative and is based on the use of a criterion function and nearest neighbor density estimates. Experiments are presented to demonstrate the algorithm.

Nonparametric discriminant analysis.

A nonparametric method of discriminant analysis is proposed. It is based on nonparametric extensions of commonly used scatter matrices. Two advantages result from the use of the proposed nonparametric scatter matrices. First, they are generally of full rank. This provides the ability to specify the number of extracted features desired. This is in contrast to parametric discriminant analysis, which for an L class problem typically can determine at most L 1 features. Second, the nonparametric nature of the scatter matrices allows the procedure to work well even for non-Gaussian data sets. Using the same basic framework, a procedure is proposed to test the structural similarity of two distributions. The procedure works in high-dimensional space. It specifies a linear decomposition of the original data space in which a relative indication of dissimilarity along each new basis vector is provided. The nonparametric scatter matrices are also used to derive a clustering procedure, which is recognized as a k-nearest neighbor version of the nonparametric valley seeking algorithm. The form which results provides a unified view of the parametric nearest mean reclassification algorithm and the nonparametric valley seeking algorithm.

Systematic feature extraction.

A systematic feature extraction procedure is proposed. It is based on successive extractions of features. At each stage a dimensionality reduction is made and a new feature is extracted. A specific example is given using the Gaussian minus-log-likelihood ratio as a basis for the extracted features. This form has the advantage that if both classes are Gaussianly distributed, only a single feature, the sufficient statistic, is extracted. If the classes are not Gaussianly distributed, additional features are extracted in an effort to improve the classification performance. Two examples are presented to demonstrate the performance of the procedure.

A nonparametric two-dimensional display for classification.

A two-dimensional display whose coordinates are related to the distance to the kth-nearest neighbor of each class is presented. Applications of the display to minimum error, minimum cost, minimax, and Neyman-Pearson type classifier designs are given. The display is shown to present risk information in a manner that easily allows the specification of reject regions. Two methods of error estimation using the display, an error counting technique and a risk averaging method, are detailed. It is shown that the classifiers that result are generalizations of the standard k-NN majority vote classifier. As a result of the properties of the display, classifiers can be readily evaluated and modified. In addition, a condensing algorithm that preserves the nearest neighbor error count of any preclassified data set is described. The display is used to graphically illustrate the distance relationships that are central to the algorithm.