Multi-Instance Nonparallel Tube Learning.

In multi-instance nonparallel plane learning (NPL), the training set is comprised of bags of instances and the nonparallel planes are trained to classify the bags. Most of the existing multi-instance NPL methods are proposed based on a twin support vector machine (TWSVM). Similar to TWSVM, they use only a single plane to generalize the data occurrence of one class and do not sufficiently consider the boundary information, which may lead to the limitation of their classification accuracy. In this article, we propose a multi-instance nonparallel tube learning (MINTL) method. Distinguished from the existing multi-instance NPL methods, MINTL embeds the boundary information into the classifier by learning a large-margin-based ϵ -tube for each class, such that the boundary information can be incorporated into refining the classifier and further improving the performance. Specifically, given a K -class multi-instance dataset, MINTL seeks K ϵ -tubes, one for each class. In multi-instance learning, each positive bag contains at least one positive instance. To build up the ϵk -tube of class k , we require that each bag of class k should have at least one instance included in the ϵk -tube. Moreover, except for one instance included in the ϵk -tube, the remaining instances in the positive bag may include positive instances or irrelevant instances, and their labels are unavailable. A large margin constraint is presented to assign the remaining instances either inside the ϵk -tube or outside the ϵk -tube with a large margin. Substantial experiments on real-world datasets have shown that MINTL obtains significantly better classification accuracy than the existing multi-instance NPL methods.

Ordinal Regression With Pinball Loss.

Ordinal regression (OR) aims to solve multiclass classification problems with ordinal classes. Support vector OR (SVOR) is a typical OR algorithm and has been extensively used in OR problems. In this article, based on the characteristics of OR problems, we propose a novel pinball loss function and present an SVOR method with pinball loss (pin-SVOR). Pin-SVOR is fundamentally different from traditional SVOR with hinge loss. Traditional SVOR employs the hinge loss function, and the classifier is determined by only a few data points near the class boundary, called support vectors, which may lead to a noise sensitive and re-sampling unstable classifier. Distinctively, pin-SVOR employs the pinball loss function. It attaches an extra penalty to correctly classified data that lies inside the class, such that all the training data is involved in deciding the classifier. The data near the middle of each class has a small penalty, and that near the class boundary has a large penalty. Thus, the training data tend to lie near the middle of each class instead of on the class boundary, which leads to scatter minimization in the middle of each class and noise insensitivity. The experimental results show that pin-SVOR has better classification performance than state-of-the-art OR methods.

A Multitask Latent Feature Augmentation Method for Few-Shot Learning.

Few-shot learning (FSL) aims to learn novel concepts quickly from a few novel labeled samples with the transferable knowledge learned from base dataset. The existing FSL methods usually treat each sample as a single feature point in embedding space and classify through one single comparison task. However, the few-shot single feature points on the novel meta-testing episode are still vulnerable to noise easily although with the good transferable knowledge, because the novel categories are never seen on base dataset. Besides, the existing FSL models are trained by only one single comparison task and ignore that different semantic feature maps have different weights on different comparison objects and tasks, which cannot take full advantage of the valuable information from different multiple comparison tasks and objects to make the latent features (LFs) more robust based on only few-shot samples. In this article, we propose a novel multitask LF augmentation (MTLFA) framework to learn the meta-knowledge of generalizing key intraclass and distinguishable interclass sample features from only few-shot samples through an LF augmentation (LFA) module and a multitask (MT) framework. Our MTLFA treats the support features as sampling from the class-specific LF distribution, enhancing the diversity of support features and reducing the impact of noise based on few-shot support samples. Furthermore, an MT framework is introduced to obtain more valuable comparison-task-related and episode-related comparison information from multiple different comparison tasks in which different semantic feature maps have different weights, adjusting the prior LFs and generating the more robust and effective episode-related classifier. Besides, we analyze the feasibility and effectiveness of MTLFA from theoretical views based on the Hoeffding's inequality and the Chernoff's bounding method. Extensive experiments conducted on three benchmark datasets demonstrate that the MTLFA achieves the state-of-the-art performance in FSL. The experimental results verify our theoretical analysis and the effectiveness and robustness of MTLFA framework in FSL.

An Efficient Image Categorization Method With Insufficient Training Samples.

Image classification is an important part of pattern recognition. With the development of convolutional neural networks (CNNs), many CNN methods are proposed, which have a large number of samples for training, which can have high performance. However, there may exist limited samples in some real-world applications. In order to improve the performance of CNN learning with insufficient samples, this article proposes a new method called the classifier method based on a variational autoencoder (CFVAE), which is comprised of two parts: 1) a standard CNN as a prior classifier and 2) a CNN based on variational autoencoder (VAE) as a posterior classifier. First, the prior classifier is utilized to generate the prior label and information about distributions of latent variables; and the posterior classifier is trained to augment some latent variables from regularized distributions to improve the performance. Second, we also present the uniform objective function of CFVAE and put forward an optimization method based on the stochastic gradient variational Bayes method to solve the objective model. Third, we analyze the feasibility of CFVAE based on Hoeffding's inequality and Chernoff's bounding method. This analysis indicates that the latent variables augmentation method based on regularized latent variables distributions can generate samples fitting well with the distribution of data such that the proposed method can improve the performance of CNN with insufficient samples. Finally, the experiments manifest that our proposed CFVAE can provide more accurate performance than state-of-the-art methods.

A Transfer Learning-Based Multi-Instance Learning Method With Weak Labels.

In multi-instance learning (MIL), labels are associated with bags rather than the instances in the bag. Most of the previous MIL methods assume that each bag has the actual label in the training set. However, from the process of labeling work, the label of a bag is always evaluated by the calculation of the labels obtained from a number of labelers. In the calculation, the weight of each labeler is always unknown and people always assign the weight for each labeler by random or equally, and this may result in the ambiguous labels for the bags, which is called weak labels here. In addition, we always meet the problem of knowledge transfer from the source task to the target task, and this leads to the study of multiple instance transfer learning. In this article, we propose a new framework called transfer learning-based multiple instance learning (TMIL) framework to address the problem of multiple instance transfer learning in which both the source task and the target task contain the weak labels. We first construct a TMIL model with weak labels, which can transfer knowledge from the source task to the target task where both source and target tasks contain weak labels. We then put forward an iterative framework to solve the transfer learning model with weak labels so that we can update the label of the bag to improve the performance of multiple instance learning. We then present the convergence analysis of the proposed method. The experiments show that the proposed method outperforms the existing multiple instance learning methods and can correct the initial labels to obtain the actual labels for the bags.

A Convex Model for Support Vector Distance Metric Learning.

Distance metric learning (DML) aims to learn a distance metric to process the data distribution. However, most of the existing methods are k NN DML methods and employ the k NN model to classify the test instances. The drawback of k NN DML is that all training instances need to be accessed and stored to classify the test instances, and the classification performance is influenced by the setting of the nearest neighbor number k . To solve these problems, there are several DML methods that employ the SVM model to classify the test instances. However, all of them are nonconvex and the convex support vector DML method has not been explicitly proposed. In this article, we propose a convex model for support vector DML (CSV-DML), which is capable of replacing the k NN model of DML with the SVM model. To make CSV-DML can use the most kernel functions of the existing SVM methods, a nonlinear mapping is used to map the original instances into a feature space. Since the explicit form of nonlinear mapped instances is unknown, the original instances are further transformed into the kernel form, which can be calculated explicitly. CSV-DML is constructed to work directly on the kernel-transformed instances. Specifically, we learn a specific Mahalanobis distance metric from the kernel-transformed training instances and train a DML-based separating hyperplane based on it. An iterated approach is formulated to optimize CSV-DML, which is based on generalized block coordinate descent and can converge to the global optimum. In CSV-DML, since the dimension of kernel-transformed instances is only related to the number of original training instances, we develop a novel parameter reduction scheme for reducing the feature dimension. Extensive experiments show that the proposed CSV-DML method outperforms the previous methods.

Multiple-Instance Ordinal Regression.

Ordinal regression (OR) is a paradigm in supervised learning, which aims at learning a prediction model for ordered classes. The existing studies mainly focus on single-instance OR, and the multi-instance OR problem has not been explicitly addressed. In many real-world applications, considering the OR problem from a multiple-instance aspect can yield better classification performance than from a single-instance aspect. For example, in image retrieval, an image may contain multiple and possibly heterogeneous objects. The user is usually interested in only a small part of the objects. If we represent the whole image as a global feature vector, the useful information from the targeted objects that the user is of interest may be overridden by the noisy information from irrelevant objects. However, this problem fits in the multiple-instance setting well. Each image is considered as a bag, and each object region is treated as an instance. The image is considered as of the user interest if it contains at least one targeted object region. In this paper, we address the multi-instance OR where the OR classifier is learned on multiple-instance data, instead of single-instance data. To solve this problem, we present a novel multiple-instance ordinal regression (MIOR) method. In MIOR, a set of parallel hyperplanes is used to separate the classes, and the label ordering information is incorporated into learning the classifier by imputing the parallel hyperplanes with an order. Moreover, considering that a bag may contain instances not belonging to its class, for each bag, the instance which is nearest to the middle of the corresponding class is selected to learn the classifier. Compared with the existing single-instance OR work, MIOR is able to learn a more accurate OR classifier on multiple-instance data where only the bag label is available and the instance label is unknown. Extensive experiments show that MIOR outperforms the existing single-instance OR methods.

A Sphere-Description-Based Approach for Multiple-Instance Learning.

Multiple-instance learning (MIL) is a generalization of supervised learning which addresses the classification of bags. Similar to traditional supervised learning, most of the existing MIL work is proposed based on the assumption that a representative training set is available for a proper learning of the classifier. That is to say, the training data can appropriately describe the distribution of positive and negative data in the testing set. However, this assumption may not be always satisfied. In real-world MIL applications, the negative data in the training set may not sufficiently represent the distribution of negative data in the testing set. Hence, how to learn an appropriate MIL classifier when a representative training set is not available becomes a key challenge for real-world MIL applications. To deal with this problem, we propose a novel Sphere-Description-Based approach for Multiple-Instance Learning (SDB-MIL). SDB-MIL learns an optimal sphere by determining a large margin among the instances, and meanwhile ensuring that each positive bag has at least one instance inside the sphere and all negative bags are outside the sphere. Enclosing at least one instance from each positive bag in the sphere enables a more desirable MIL classifier when the negative data in the training set cannot sufficiently represent the distribution of negative data in the testing set. Substantial experiments on the benchmark and real-world MIL datasets show that SDB-MIL obtains statistically better classification performance than the MIL methods compared.

A Maximum Margin Approach for Semisupervised Ordinal Regression Clustering.

Ordinal regression (OR) is generally defined as the task where the input samples are ranked on an ordinal scale. OR has found a wide variety of applications, and a great deal of work has been done on it. However, most of the existing work focuses on supervised/semisupervised OR classification, and the semisupervised OR clustering problems have not been explicitly addressed. In real-world OR applications, labeling a large number of training samples is usually time-consuming and costly, and instead, a set of unlabeled samples can be utilized to set up the OR model. Moreover, although the sample labels are unavailable, we can sometimes get the relative ranking information of the unlabeled samples. This sample ranking information can be utilized to refine the OR model. Hence, how to build an OR model on the unlabeled samples and incorporate the sample ranking information into the process of improving the clustering accuracy remains a key challenge for OR applications. In this paper, we consider the semisupervised OR clustering problems with sample-ranking constraints, which give the relative ranking information of the unlabeled samples, and put forward a maximum margin approach for semisupervised OR clustering ( [Formula: see text]SORC). On one hand, [Formula: see text]SORC seeks a set of parallel hyperplanes to partition the unlabeled samples into clusters. On the other hand, a loss function is put forward to incorporate the sample ranking information into the clustering process. As a result, the optimization function of [Formula: see text]SORC is formulated to maximize the margins of the closest neighboring clusters and meanwhile minimize the loss associated with the sample-ranking constraints. Extensive experiments on OR data sets show that the proposed [Formula: see text]SORC method outperforms the traditional semisupervised clustering methods considered.

A similarity-based classification framework for multiple-instance learning.

Multiple-instance learning (MIL) is a generalization of supervised learning that attempts to learn useful information from bags of instances. In MIL, the true labels of instances in positive bags are not available for training. This leads to a critical challenge, namely, handling the instances of which the labels are ambiguous (ambiguous instances). To deal with these ambiguous instances, we propose a novel MIL approach, called similarity-based multiple-instance learning (SMILE). Instead of eliminating a number of ambiguous instances in positive bags from training the classifier, as done in some previous MIL works, SMILE explicitly deals with the ambiguous instances by considering their similarity to the positive class and the negative class. Specifically, a subset of instances is selected from positive bags as the positive candidates and the remaining ambiguous instances are associated with two similarity weights, representing the similarity to the positive class and the negative class, respectively. The ambiguous instances, together with their similarity weights, are thereafter incorporated into the learning phase to build an extended SVM-based predictive classifier. A heuristic framework is employed to update the positive candidates and the similarity weights for refining the classification boundary. Experiments on real-world datasets show that SMILE demonstrates highly competitive classification accuracy and shows less sensitivity to labeling noise than the existing MIL methods.

Improved algorithm for phase-to-height mapping in phase measuring profilometry.

An improved algorithm for phase-to-height mapping in phase-measuring profilometry (PMP) is proposed, in which the phase-to-height mapping relationship is no longer restricted to the condition that the optical axes of the imaging system must be orthogonal to the reference plane in the basic PMP. Only seven coefficients independent of the coordinate system need to be calibrated, and the system calibration can be accomplished using only two different gauge blocks, instead of more than three different standard planes. With the proposed algorithm, both the phase measurement and system calibration can be completed simultaneously, which makes the three-dimensional (3-D) measurement faster and more flexible. Experiments have verified its feasibility and validity.