Multiple Flat Projections for Cross-Manifold Clustering.

Cross-manifold clustering is an extreme challenge learning problem. Since the low-density hypothesis is not satisfied in cross-manifold problems, many traditional clustering methods failed to discover the cross-manifold structures. In this article, we propose multiple flat projections clustering (MFPC) for cross-manifold clustering. In our MFPC, the given samples are projected into multiple localized flats to discover the global structures of implicit manifolds. Thus, the intersected clusters are distinguished in various projection flats. In MFPC, a series of nonconvex matrix optimization problems is solved by a proposed recursive algorithm. Furthermore, a nonlinear version of MFPC is extended via kernel tricks to deal with a more complex cross-manifold learning situation. The synthetic tests show that our MFPC works on the cross-manifold structures well. Moreover, experimental results on the benchmark datasets and object tracking videos show excellent performance of our MFPC compared with some state-of-the-art manifold clustering methods.

Two-dimensional Bhattacharyya bound linear discriminant analysis with its applications.

The recently proposed L2-norm linear discriminant analysis criterion based on Bhattacharyya error bound estimation (L2BLDA) was an effective improvement over linear discriminant analysis (LDA) and was used to handle vector input samples. When faced with two-dimensional (2D) inputs, such as images, converting two-dimensional data to vectors, regardless of the inherent structure of the image, may result in some loss of useful information. In this paper, we propose a novel two-dimensional Bhattacharyya bound linear discriminant analysis (2DBLDA). 2DBLDA maximizes the matrix-based between-class distance, which is measured by the weighted pairwise distances of class means and minimizes the matrix-based within-class distance. The criterion of 2DBLDA is equivalent to optimizing the upper bound of the Bhattacharyya error. The weighting constant between the between-class and within-class terms is determined by the involved data that make the proposed 2DBLDA adaptive. The construction of 2DBLDA avoids the small sample size (SSS) problem, is robust, and can be solved through a simple standard eigenvalue decomposition problem. The experimental results on image recognition and face image reconstruction demonstrate the effectiveness of 2DBLDA.

Generalized two-dimensional linear discriminant analysis with regularization.

Recent advances show that two-dimensional linear discriminant analysis (2DLDA) is a successful matrix based dimensionality reduction method. However, 2DLDA may encounter the singularity issue theoretically, and also is sensitive to outliers. In this paper, a generalized Lp-norm 2DLDA framework with regularization for an arbitrary p>0 is proposed, named G2DLDA. There are mainly two contributions of G2DLDA: one is G2DLDA model uses an arbitrary Lp-norm to measure the between-class and within-class scatter, and hence a proper p can be selected to achieve robustness. The other one is that the introduced regularization term makes G2DLDA enjoy better generalization performance and avoid singularity. In addition, an effective learning algorithm is designed for G2LDA, which can be solved through a series of convex problems with closed-form solutions. Its convergence can be guaranteed theoretically when 1≤p≤2. Preliminary experimental results on three contaminated human face databases show the effectiveness of the proposed G2DLDA.

General Plane-Based Clustering With Distribution Loss.

In this article, we propose a general model for plane-based clustering. The general model reveals the relationship between cluster assignment and cluster updating during clustering implementation, and it contains many existing plane-based clustering methods, e.g., k-plane clustering, proximal plane clustering, twin support vector clustering, and their extensions. Under this general model, one may obtain an appropriate clustering method for a specific purpose. The general model is a procedure corresponding to an optimization problem, which minimizes the total loss of the samples. Thereinto, the loss of a sample derives from both within-cluster and between-cluster information. We discuss the theoretical termination conditions and prove that the general model terminates in a finite number of steps at a local or weak local solution. Furthermore, we propose a distribution loss function that fluctuates with the input data and introduce it into the general model to obtain a plane-based clustering method (DPC). DPC can capture the data distribution precisely because of its statistical characteristics, and its termination that finitely terminates at a weak local solution is given immediately based on the general model. The experimental results show that our DPC outperforms the state-of-the-art plane-based clustering methods on many synthetic and benchmark data sets.

Robust and Sparse Linear Discriminant Analysis via an Alternating Direction Method of Multipliers.

In this paper, we propose a robust linear discriminant analysis (RLDA) through Bhattacharyya error bound optimization. RLDA considers a nonconvex problem with the L1 -norm operation that makes it less sensitive to outliers and noise than the L2 -norm linear discriminant analysis (LDA). In addition, we extend our RLDA to a sparse model (RSLDA). Both RLDA and RSLDA can extract unbounded numbers of features and avoid the small sample size (SSS) problem, and an alternating direction method of multipliers (ADMM) is used to cope with the nonconvexity in the proposed formulations. Compared with the traditional LDA, our RLDA and RSLDA are more robust to outliers and noise, and RSLDA can obtain sparse discriminant directions. These findings are supported by experiments on artificial data sets as well as human face databases.

Robust recursive absolute value inequalities discriminant analysis with sparseness.

In this paper, we propose a novel absolute value inequalities discriminant analysis (AVIDA) criterion for supervised dimensionality reduction. Compared with the conventional linear discriminant analysis (LDA), the main characteristics of our AVIDA are robustness and sparseness. By reformulating the generalized eigenvalue problem in LDA to a related SVM-type "concave-convex" problem based on absolute value inequalities loss, our AVIDA is not only more robust to outliers and noises, but also avoids the SSS problem. Moreover, the additional L1-norm regularization term in the objective makes sure sparse discriminant vectors are obtained. A successive linear algorithm is employed to solve the proposed optimization problem, where a series of linear programs are solved. The superiority of our AVIDA is supported by experimental results on artificial examples as well as benchmark image databases.

A Learning Framework of Nonparallel Hyperplanes Classifier.

A novel learning framework of nonparallel hyperplanes support vector machines (NPSVMs) is proposed for binary classification and multiclass classification. This framework not only includes twin SVM (TWSVM) and its many deformation versions but also extends them into multiclass classification problem when different parameters or loss functions are chosen. Concretely, we discuss the linear and nonlinear cases of the framework, in which we select the hinge loss function as example. Moreover, we also give the primal problems of several extension versions of TWSVM's deformation versions. It is worth mentioning that, in the decision function, the Euclidean distance is replaced by the absolute value |w (T) x + b|, which keeps the consistency between the decision function and the optimization problem and reduces the computational cost particularly when the kernel function is introduced. The numerical experiments on several artificial and benchmark datasets indicate that our framework is not only fast but also shows good generalization.

Robust L1-norm two-dimensional linear discriminant analysis.

In this paper, we propose an L1-norm two-dimensional linear discriminant analysis (L1-2DLDA) with robust performance. Different from the conventional two-dimensional linear discriminant analysis with L2-norm (L2-2DLDA), where the optimization problem is transferred to a generalized eigenvalue problem, the optimization problem in our L1-2DLDA is solved by a simple justifiable iterative technique, and its convergence is guaranteed. Compared with L2-2DLDA, our L1-2DLDA is more robust to outliers and noises since the L1-norm is used. This is supported by our preliminary experiments on toy example and face datasets, which show the improvement of our L1-2DLDA over L2-2DLDA.

Twin support vector machine for clustering.

The twin support vector machine (TWSVM) is one of the powerful classification methods. In this brief, a TWSVM-type clustering method, called twin support vector clustering (TWSVC), is proposed. Our TWSVC includes both linear and nonlinear versions. It determines k cluster center planes by solving a series of quadratic programming problems. To make TWSVC more efficient and stable, an initialization algorithm based on the nearest neighbor graph is also suggested. The experimental results on several benchmark data sets have shown a comparable performance of our TWSVC.

A coordinate descent margin based-twin support vector machine for classification.

Twin support vector machines (TWSVMs) obtain faster learning speed by solving a pair of smaller SVM-type problems. In order to increase its efficiency further, this paper presents a coordinate descent margin based twin vector machine (CDMTSVM) compared with the original TWSVM. The major advantages of CDMTSVM lie in two aspects: (1) The primal and dual problems are reformulated and improved by adding a regularization term in the primal problems which implies maximizing the "margin" between the proximal hyperplane and bounding hyperplane, yielding the dual problems to be stable positive definite quadratic programming problems. (2) A novel coordinate descent method is proposed for our dual problems which leads to very fast training. As our coordinate descent method handles one data point at a time, it can process very large datasets that need not reside in memory. Our experiments on publicly available datasets indicate that our CDMTSVM is not only fast, but also shows good generalization performance.

Improvements on twin support vector machines.

For classification problems, the generalized eigenvalue proximal support vector machine (GEPSVM) and twin support vector machine (TWSVM) are regarded as milestones in the development of the powerful SVMs, as they use the nonparallel hyperplane classifiers. In this brief, we propose an improved version, named twin bounded support vector machines (TBSVM), based on TWSVM. The significant advantage of our TBSVM over TWSVM is that the structural risk minimization principle is implemented by introducing the regularization term. This embodies the marrow of statistical learning theory, so this modification can improve the performance of classification. In addition, the successive overrelaxation technique is used to solve the optimization problems to speed up the training procedure. Experimental results show the effectiveness of our method in both computation time and classification accuracy, and therefore confirm the above conclusion further.

An efficient support vector machine approach for identifying protein S-nitrosylation sites.

Protein S-nitrosylation plays a key and specific role in many cellular processes. Detecting possible S-nitrosylated substrates and their corresponding exact sites is crucial for studying the mechanisms of these biological processes. Comparing with the expensive and time-consuming biochemical experiments, the computational methods are attracting considerable attention due to their convenience and fast speed. Although some computational models have been developed to predict S-nitrosylation sites, their accuracy is still low. In this work,we incorporate support vector machine to predict protein S-nitrosylation sites. After a careful evaluation of six encoding schemes, we propose a new efficient predictor, CPR-SNO, using the coupling patterns based encoding scheme. The performance of our CPR-SNO is measured with the area under the ROC curve (AUC) of 0.8289 in 10-fold cross validation experiments, which is significantly better than the existing best method GPS-SNO 1.0's 0.685 performance. In further annotating large-scale potential S-nitrosylated substrates, CPR-SNO also presents an encouraging predictive performance. These results indicate that CPR-SNO can be used as a competitive protein S-nitrosylation sites predictor to the biological community. Our CPR-SNO has been implemented as a web server and is available at http://math.cau.edu.cn/CPR -SNO/CPR-SNO.html.

Improved prediction of palmitoylation sites using PWMs and SVM.

Protein palmitoylation is an important and common post-translational lipid modification of proteins and plays a critical role in various cellular processes. Identification of Palmitoylation sites is fundamental to decipher the mechanisms of these biological processes. However, experimental determination of palmitoylation residues without prior knowledge is laborious and costly. Thus computational approaches for prediction of palmitoylation sites in proteins have become highly desirable. Here, we propose PPWMs, a computational predictor using Position Weight Matrices (PWMs) encoding scheme and support vector machine (SVM) for identifying protein palmitoylation sites. Our PPWMs shows a nice predictive performance with the area under the ROC curve (AUC) of 0.9472 for the S-palmitoylation sites prediction and 0.9964 for the N-palmitoylation sites prediction on the newly proposed dataset. Comparison results show the superiority of PPWMs over two existing widely known palmitoylation site predictors CSS-Palm 2.0 and CKSAAP-Palm in many cases. Moreover, an attempt of incorporating structure information such as accessible surface area (ASA) and secondary structure (SS) into prediction is made and the structure characteristics are analyzed roughly. The corresponding software can be freely downloaded from http://math.cau.edu.cn/PPWMs.html.