Universum-Inspired Supervised Contrastive Learning.

As an effective data augmentation method, Mixup synthesizes an extra amount of samples through linear interpolations. Despite its theoretical dependency on data properties, Mixup reportedly performs well as a regularizer and calibrator contributing reliable robustness and generalization to deep model training. In this paper, inspired by Universum Learning which uses out-of-class samples to assist the target tasks, we investigate Mixup from a largely under-explored perspective - the potential to generate in-domain samples that belong to none of the target classes, that is, universum. We find that in the framework of supervised contrastive learning, Mixup-induced universum can serve as surprisingly high-quality hard negatives, greatly relieving the need for large batch sizes in contrastive learning. With these findings, we propose Universum-inspired supervised Contrastive learning (UniCon), which incorporates Mixup strategy to generate Mixup-induced universum as universum negatives and pushes them apart from anchor samples of the target classes. We extend our method to the unsupervised setting, proposing Unsupervised Universum-inspired contrastive model (Un-Uni). Our approach not only improves Mixup with hard labels, but also innovates a novel measure to generate universum data. With a linear classifier on the learned representations, UniCon shows state-of-the-art performance on various datasets. Specially, UniCon achieves 81.7% top-1 accuracy on CIFAR-100, surpassing the state of art by a significant margin of 5.2% with a much smaller batch size, typically, 256 in UniCon vs. 1024 in SupCon (Khosla et al., 2020) using ResNet-50. Un-Uni also outperforms SOTA methods on CIFAR-100. The code of this paper is released on https://github.com/hannaiiyanggit/UniCon.

Multiplane Convex Proximal Support Vector Machine.

As an effective method for XOR problems, generalized eigenvalue proximal support vector machine (GEPSVM) recently has gained widespread attention accompanied with many variants proposed. Although these variants strengthen the classification performance to different extents, the number of fitting hyperplanes, similar to GEPSVM, for each class is still limited to just one. Intuitively, using single hyperplane seems not enough, especially for the datasets with complex feature structures. Therefore, this article mainly focuses on extending the fitting hyperplanes for each class from single one to multiple ones. However, such an extension from the original GEPSVM is not trivial even though, if possible, the elegant solution via generalized eigenvalues will also not be guaranteed. To address this issue, we first make a simple yet crucial transformation for the optimization problem of GEPSVM and then propose a novel multiplane convex proximal support vector machine (MCPSVM), where a set of hyperplanes determined by the features of the data are learned for each class. We adopt a strictly (geodesically) convex objective to characterize this optimization problem; thus, a more elegant closed-form solution is obtained, which only needs a few lines of MATLAB codes. Besides, MCPSVM is more flexible in form and can be naturally and seamlessly extended to the feature weighting learning, whereas GEPSVM and its variants can hardly straightforwardly work like this. Extensive experiments on benchmark and large-scale image datasets indicate the advantages of our MCPSVM.

Recent Advances in Open Set Recognition: A Survey.

In real-world recognition/classification tasks, limited by various objective factors, it is usually difficult to collect training samples to exhaust all classes when training a recognizer or classifier. A more realistic scenario is open set recognition (OSR), where incomplete knowledge of the world exists at training time, and unknown classes can be submitted to an algorithm during testing, requiring the classifiers to not only accurately classify the seen classes, but also effectively deal with unseen ones. This paper provides a comprehensive survey of existing open set recognition techniques covering various aspects ranging from related definitions, representations of models, datasets, evaluation criteria, and algorithm comparisons. Furthermore, we briefly analyze the relationships between OSR and its related tasks including zero-shot, one-shot (few-shot) recognition/learning techniques, classification with reject option, and so forth. Additionally, we also review the open world recognition which can be seen as a natural extension of OSR. Importantly, we highlight the limitations of existing approaches and point out some promising subsequent research directions in this field.

Metric Learning-Guided Least Squares Classifier Learning.

For a multicategory classification problem, discriminative least squares regression (DLSR) explicitly introduces an -dragging technique to enlarge the margin between the categories, yielding superior classification performance from a margin perspective. In this brief, we reconsider this classification problem from a metric learning perspective and propose a framework of metric learning-guided least squares classifier (MLG-LSC) learning. The core idea is to learn a unified metric matrix for the error of LSR, such that such a metric matrix can yield small distances for the same category, while large ones for the different categories. As opposed to the -dragging in DLSR, we call this the error-dragging (e-dragging). Different from DLSR and its related variants, our MLG-LSC implicitly carries out the e-dragging and can naturally reflect the roughly relative distance relationships among the categories from a metric learning perspective. Furthermore, our optimization objective functions are strictly (geodesically) convex and thus can obtain their corresponding closed-form solutions, resulting in higher computational performance. Experimental results on a set of benchmark data sets indicate the validity of our learning framework.