A Simple Label Switching Algorithm for Semisupervised Structural SVMs.

In structured output learning, obtaining labeled data for real-world applications is usually costly, while unlabeled examples are available in abundance. Semisupervised structured classification deals with a small number of labeled examples and a large number of unlabeled structured data. In this work, we consider semisupervised structural support vector machines with domain constraints. The optimization problem, which in general is not convex, contains the loss terms associated with the labeled and unlabeled examples, along with the domain constraints. We propose a simple optimization approach that alternates between solving a supervised learning problem and a constraint matching problem. Solving the constraint matching problem is difficult for structured prediction, and we propose an efficient and effective label switching method to solve it. The alternating optimization is carried out within a deterministic annealing framework, which helps in effective constraint matching and avoiding poor local minima, which are not very useful. The algorithm is simple and easy to implement. Further, it is suitable for any structured output learning problem where exact inference is available. Experiments on benchmark sequence labeling data sets and a natural language parsing data set show that the proposed approach, though simple, achieves comparable generalization performance.

Validation-based sparse Gaussian process classifier design.

Gaussian processes (GPs) are promising Bayesian methods for classification and regression problems. Design of a GP classifier and making predictions using it is, however, computationally demanding, especially when the training set size is large. Sparse GP classifiers are known to overcome this limitation. In this letter, we propose and study a validation-based method for sparse GP classifier design. The proposed method uses a negative log predictive (NLP) loss measure, which is easy to compute for GP models. We use this measure for both basis vector selection and hyperparameter adaptation. The experimental results on several real-world benchmark data sets show better or comparable generalization performance over existing methods.

Fast generalized cross-validation algorithm for sparse model learning.

We propose a fast, incremental algorithm for designing linear regression models. The proposed algorithm generates a sparse model by optimizing multiple smoothing parameters using the generalized cross-validation approach. The performances on synthetic and real-world data sets are compared with other incremental algorithms such as Tipping and Faul's fast relevance vector machine, Chen et al.'s orthogonal least squares, and Orr's regularized forward selection. The results demonstrate that the proposed algorithm is competitive.

Molecular changes from dysplastic nodule to hepatocellular carcinoma through gene expression profiling.

Progression of hepatocellular carcinoma (HCC) is a stepwise process that proceeds from pre-neoplastic lesions--including low-grade dysplastic nodules (LGDNs) and high-grade dysplastic nodules (HGDNs)--to advanced HCC. The molecular changes associated with this progression are unclear, however, and the morphological cues thought to distinguish pre-neoplastic lesions from well-differentiated HCC are not universally accepted. To understand the multistep process of hepato-carcinogenesis at the molecular level, we used oligo-nucleotide microarrays to investigate the transcription profiles of 50 hepatocellular nodular lesions ranging from LGDNs to primary HCC (Edmondson grades 1-3). We demonstrated that gene expression profiles can discriminate not only between dysplastic nodules and overt carcinoma but also between different histological grades of HCC via unsupervised hierarchical clustering with 10,376 genes. We identified 3,084 grade-associated genes, correlated with tumor progression, using one-way ANOVA and a one-versus-all unpooled t test. Functional assignment of these genes revealed discrete expression clusters representing grade-dependent biological properties of HCC. Using both diagonal linear discriminant analysis and support vector machines, we identified 240 genes that could accurately classify tumors according to histological grade, especially when attempting to discriminate LGDNs, HGDNs, and grade 1 HCC. In conclusion, a clear molecular demarcation between dysplastic nodules and overt HCC exists. The progression from grade 1 through grade 3 HCC is associated with changes in gene expression consistent with plausible functional consequences.