Deep Discriminative Feature Models (DDFMs) for Set Based Face Recognition and Distance Metric Learning.

This article introduces two methods that find compact deep feature models for approximating images in set based face recognition problems. The proposed method treats each image set as a nonlinear face manifold that is composed of linear components. To find linear components of the face manifold, we first split image sets into subsets containing face images which share similar appearances. Then, our first proposed method approximates each subset by using the center of the deep feature representations of images in those subsets. Centers modeling the subsets are learned by using distance metric learning. The second proposed method uses discriminative common vectors to represent image features in the subsets, and entire subset is approximated with an affine hull in this approach. Discriminative common vectors are subset centers that are projected onto a new feature space where the combined within-class variances coming from all subsets are removed. Our proposed methods can also be considered as distance metric learning methods using triplet loss function where the learned subcluster centers are the selected anchors. This procedure yields to applying distance metric learning to quantized data and brings many advantages over using classical distance metric learning methods. We tested proposed methods on various face recognition problems using image sets and some visual object classification problems. Experimental results show that the proposed methods achieve the state-of-the-art accuracies on the most of the tested image datasets.

Polyhedral Conic Classifiers for Computer Vision Applications and Open Set Recognition.

This paper introduces a family of quasi-linear discriminants that outperform current large-margin methods in sliding window visual object detection and open set recognition tasks. In these applications, the classification problems are both numerically imbalanced - positive (object class) training and test windows are much rarer than negative (non-class) ones - and geometrically asymmetric - the positive samples typically form compact, visually-coherent groups while negatives are much more diverse, including anything at all that is not a well-centered sample from the target class. For such tasks, there is a need for discriminants whose decision regions focus on tightly circumscribing the positive class, while still taking account of negatives in zones where the two classes overlap. To this end, we propose a family of quasi-linear "polyhedral conic" discriminants whose positive regions are distorted L1 or L2 balls. In addition, we also integrated the proposed classification loss into deep neural networks so that both the features and classifier can be learned simultaneously end-to-end fashion to improve the classification accuracies. The methods have properties and run-time complexities comparable to linear Support Vector Machines (SVMs), and they can be trained from either binary or positive-only samples using constrained quadratic programs related to SVMs. Our experiments show that they significantly outperform linear SVMs, deep neural networks using softmax loss function and existing one-class discriminants on a wide range of object detection, face verification, open set recognition and conventional closed-set classification tasks.

Best Fitting Hyperplanes for Classification.

In this paper, we propose novel methods that are more suitable than classical large-margin classifiers for open set recognition and object detection tasks. The proposed methods use the best fitting hyperplanes approach, and the main idea is to find the best fitting hyperplanes such that each hyperplane is close to the samples of one of the classes and is as far as possible from the other class samples. To this end, we propose two different classifiers: The first classifier solves a convex quadratic optimization problem, but negative samples can lie on one side of the best fitting hyperplane. The second classifier, however, allows the negative samples to lie on both sides of the fitting hyperplane by using concave-convex procedure. Both methods are extended to the nonlinear case by using the kernel trick. In contrast to the existing hyperplane fitting classifiers in the literature, our proposed methods are suitable for large-scale problems, and they return sparse solutions. The experiments on several databases show that the proposed methods typically outperform other hyperplane fitting classifiers, and they work as good as the SVM classifier in classical recognition tasks. However, the proposed methods significantly outperform SVM in open set recognition and object detection tasks.

Local classifier weighting by quadratic programming.

It has been widely accepted that the classification accuracy can be improved by combining outputs of multiple classifiers. However, how to combine multiple classifiers with various (potentially conflicting) decisions is still an open problem. A rich collection of classifier combination procedures -- many of which are heuristic in nature -- have been developed for this goal. In this brief, we describe a dynamic approach to combine classifiers that have expertise in different regions of the input space. To this end, we use local classifier accuracy estimates to weight classifier outputs. Specifically, we estimate local recognition accuracies of classifiers near a query sample by utilizing its nearest neighbors, and then use these estimates to find the best weights of classifiers to label the query. The problem is formulated as a convex quadratic optimization problem, which returns optimal nonnegative classifier weights with respect to the chosen objective function, and the weights ensure that locally most accurate classifiers are weighted more heavily for labeling the query sample. Experimental results on several data sets indicate that the proposed weighting scheme outperforms other popular classifier combination schemes, particularly on problems with complex decision boundaries. Hence, the results indicate that local classification-accuracy-based combination techniques are well suited for decision making when the classifiers are trained by focusing on different regions of the input space.

The kernel common vector method: a novel nonlinear subspace classifier for pattern recognition.

The common vector (CV) method is a linear subspace classifier method which allows one to discriminate between classes of data sets, such as those arising in image and word recognition. This method utilizes subspaces that represent classes during classification. Each subspace is modeled such that common features of all samples in the corresponding class are extracted. To accomplish this goal, the method eliminates features that are in the direction of the eigenvectors corresponding to the nonzero eigenvalues of the covariance matrix of each class. In this paper, we introduce a variation of the CV method, which will be referred to as the modified CV (MCV) method. Then, a novel approach is proposed to apply the MCV method in a nonlinearly mapped higher dimensional feature space. In this approach, all samples are mapped into a higher dimensional feature space using a kernel mapping function, and then, the MCV method is applied in the mapped space. Under certain conditions, each class gives rise to a unique CV, and the method guarantees a 100% recognition rate with respect to the training set data. Moreover, experiments with several test cases also show that the generalization performance of the proposed kernel method is comparable to the generalization performances of other linear subspace classifier methods as well as the kernel-based nonlinear subspace method. While both the MCV method and its kernel counterpart did not outperform the support vector machine (SVM) classifier in most of the reported experiments, the application of our proposed methods is simpler than that of the multiclass SVM classifier. In addition, it is not necessary to adjust any parameters in our approach.

Discriminative common vector method with kernels.

In some pattern recognition tasks, the dimension of the sample space is larger than the number of samples in the training set. This is known as the "small sample size problem". Linear discriminant analysis (LDA) techniques cannot be applied directly to the small sample size case. The small sample size problem is also encountered when kernel approaches are used for recognition. In this paper, we attempt to answer the question of "How should one choose the optimal projection vectors for feature extraction in the small sample size case?" Based on our findings, we propose a new method called the kernel discriminative common vector method. In this method, we first nonlinearly map the original input space to an implicit higher dimensional feature space, in which the data are hoped to be linearly separable. Then, the optimal projection vectors are computed in this transformed space. The proposed method yields an optimal solution for maximizing a modified Fisher's linear discriminant criterion, discussed in the paper. Thus, under certain conditions, a 100% recognition rate is guaranteed for the training set samples. Experiments on test data also show that, in many situations, the generalization performance of the proposed method compares favorably with other kernel approaches.

Discriminative common vectors for face recognition.

In face recognition tasks, the dimension of the sample space is typically larger than the number of the samples in the training set. As a consequence, the within-class scatter matrix is singular and the Linear Discriminant Analysis (LDA) method cannot be applied directly. This problem is known as the "small sample size" problem. In this paper, we propose a new face recognition method called the Discriminative Common Vector method based on a variation of Fisher's Linear Discriminant Analysis for the small sample size case. Two different algorithms are given to extract the discriminative common vectors representing each person in the training set of the face database. One algorithm uses the within-class scatter matrix of the samples in the training set while the other uses the subspace methods and the Gram-Schmidt orthogonalization procedure to obtain the discriminative common vectors. Then, the discriminative common vectors are used for classification of new faces. The proposed method yields an optimal solution for maximizing the modified Fisher's Linear Discriminant criterion given in the paper. Our test results show that the Discriminative Common Vector method is superior to other methods in terms of recognition accuracy, efficiency, and numerical stability.

Phantom validation of coregistration of PET and CT for image-guided radiotherapy.

Radiotherapy treatment planning integrating positron emission tomography (PET) and computerized tomography (CT) is rapidly gaining acceptance in the clinical setting. Although hybrid systems are available, often the planning CT is acquired on a dedicated system separate from the PET scanner. A limiting factor to using PET data becomes the accuracy of the CT/PET registration. In this work, we use phantom and patient validation to demonstrate a general method for assessing the accuracy of CT/PET image registration and apply it to two multi-modality image registration programs. An IAEA (International Atomic Energy Association) brain phantom and an anthropomorphic head phantom were used. Internal volumes and externally mounted fiducial markers were filled with CT contrast and 18F-fluorodeoxyglucose (FDG). CT, PET emission, and PET transmission images were acquired and registered using two different image registration algorithms. CT/PET Fusion (GE Medical Systems, Milwaukee, WI) is commercially available and uses a semi-automated initial step followed by manual adjustment. Automatic Mutual Information-based Registration (AMIR), developed at our institution, is fully automated and exhibits no variation between repeated registrations. Registration was performed using distinct phantom structures; assessment of accuracy was determined from registration of the calculated centroids of a set of fiducial markers. By comparing structure-based registration with fiducial-based registration, target registration error (TRE) was computed at each point in a three-dimensional (3D) grid that spans the image volume. Identical methods were also applied to patient data to assess CT/PET registration accuracy. Accuracy was calculated as the mean with standard deviation of the TRE for every point in the 3D grid. Overall TRE values for the IAEA brain phantom are: CT/PET Fusion = 1.71 +/- 0.62 mm, AMIR = 1.13 +/- 0.53 mm; overall TRE values for the anthropomorphic head phantom are: CT/PET Fusion = 1.66 +/- 0.53 mm, AMIR = 1.15 +/- 0.48 mm. Precision (repeatability by a single user) measured for CT/PET Fusion: IAEA phantom = 1.59 +/- 0.67 mm and anthropomorphic head phantom = 1.63 +/- 0.52 mm. (AMIR has exact precision and so no measurements are necessary.) One sample patient demonstrated the following accuracy results: CT/PET Fusion = 3.89 +/- 1.61 mm, AMIR = 2.86 +/- 0.60 mm. Semi-automatic and automatic image registration methods may be used to facilitate incorporation of PET data into radiotherapy treatment planning in relatively rigid anatomic sites, such as head and neck. The overall accuracies in phantom and patient images are < 2 mm and < 4 mm, respectively, using either registration algorithm. Registration accuracy may decrease, however, as distance from the initial registration points (CT/PET fusion) or center of the image (AMIR) increases. Additional information provided by PET may improve dose coverage to active tumor subregions and hence tumor control. This study shows that the accuracy obtained by image registration with these two methods is well suited for image-guided radiotherapy.