Generalized zero-shot domain adaptation via coupled conditional variational autoencoders.

Domain adaptation aims to exploit useful information from the source domain where annotated training data are easier to obtain to address a learning problem in the target domain where only limited or even no annotated data are available. In classification problems, domain adaptation has been studied under the assumption all classes are available in the target domain regardless of the annotations. However, a common situation where only a subset of classes in the target domain are available has not attracted much attention. In this paper, we formulate this particular domain adaptation problem within a generalized zero-shot learning framework by treating the labelled source-domain samples as semantic representations for zero-shot learning. For this novel problem, neither conventional domain adaptation approaches nor zero-shot learning algorithms directly apply. To solve this problem, we present a novel Coupled Conditional Variational Autoencoder (CCVAE) which can generate synthetic target-domain image features for unseen classes from real images in the source domain. Extensive experiments have been conducted on three domain adaptation datasets including a bespoke X-ray security checkpoint dataset to simulate a real-world application in aviation security. The results demonstrate the effectiveness of our proposed approach both against established benchmarks and in terms of real-world applicability.

Data augmentation with norm-AE and selective pseudo-labelling for unsupervised domain adaptation.

We address the Unsupervised Domain Adaptation (UDA) problem in image classification from a new perspective. In contrast to most existing works which either align the data distributions or learn domain-invariant features, we directly learn a unified classifier for both the source and target domains in the high-dimensional homogeneous feature space without explicit domain alignment. To this end, we employ the effective Selective Pseudo-Labelling (SPL) technique to take advantage of the unlabelled samples in the target domain. Surprisingly, data distribution discrepancy across the source and target domains can be well handled by a computationally simple classifier (e.g., a shallow Multi-Layer Perceptron) trained in the original feature space. Besides, we propose a novel generative model norm-AE to generate synthetic features for the target domain as a data augmentation strategy to enhance the classifier training. Experimental results on several benchmark datasets demonstrate the pseudo-labelling strategy itself can lead to comparable performance to many state-of-the-art methods whilst the use of norm-AE for feature augmentation can further improve the performance in most cases. As a result, our proposed methods (i.e. naive-SPL and norm-AE-SPL) can achieve comparable performance with state-of-the-art methods with the average accuracy of 93.4% and 90.4% on Office-Caltech and ImageCLEF-DA datasets, and achieve competitive performance on Digits, Office31 and Office-Home datasets with the average accuracy of 97.2%, 87.6% and 68.6% respectively.

A reference architecture for plausible Threat Image Projection (TIP) within 3D X-ray computed tomography volumes.

BACKGROUND: Threat Image Projection (TIP) is a technique used in X-ray security baggage screening systems that superimposes a threat object signature onto a benign X-ray baggage image in a plausible and realistic manner. It has been shown to be highly effective in evaluating the ongoing performance of human operators, improving their vigilance and performance on threat detection. OBJECTIVE: With the increasing use of 3D Computed Tomography (CT) in aviation security for both hold and cabin baggage screening a significant challenge arises in extending TIP to 3D CT volumes due to the difficulty in 3D CT volume segmentation and the proper insertion location determination. In this paper, we present an approach for 3D TIP in CT volumes targeting realistic and plausible threat object insertion within 3D CT baggage images. METHOD: The proposed approach consists of dual threat (source) and baggage (target) volume segmentation, particle swarm optimisation based insertion determination and metal artefact generation. In our experiments, real baggage data collected from airports are used to generate TIP volumes for evaluation. We also propose a TIP quality score metric to automatically estimate the quality of generated TIP volumes. RESULT: In our experiments with real baggage CT volumes and varying threat items, 90.25% of the generated TIP volumes are graded as good by human evaluation, 7% of them are of medium quality with minor flaws and 2.75% of them are bad. CONCLUSION: Qualitative evaluations on real 3D CT baggage imagery show that our approach is able to generate realistic and plausible TIP which are indiscernible from real CT volumes and the TIP quality scores are consistent with human evaluations.

An approach for adaptive automatic threat recognition within 3D computed tomography images for baggage security screening.

BACKGROUND: The screening of baggage using X-ray scanners is now routine in aviation security with automatic threat detection approaches, based on 3D X-ray computed tomography (CT) images, known as Automatic Threat Recognition (ATR) within the aviation security industry. These current strategies use pre-defined threat material signatures in contrast to adaptability towards new and emerging threat signatures. To address this issue, the concept of adaptive automatic threat recognition (AATR) was proposed in previous work. OBJECTIVE: In this paper, we present a solution to AATR based on such X-ray CT baggage scan imagery. This aims to address the issues of rapidly evolving threat signatures within the screening requirements. Ideally, the detection algorithms deployed within the security scanners should be readily adaptable to different situations with varying requirements of threat characteristics (e.g., threat material, physical properties of objects). METHODS: We tackle this issue using a novel adaptive machine learning methodology with our solution consisting of a multi-scale 3D CT image segmentation algorithm, a multi-class support vector machine (SVM) classifier for object material recognition and a strategy to enable the adaptability of our approach. Experiments are conducted on both open and sequestered 3D CT baggage image datasets specifically collected for the AATR study. RESULTS: Our proposed approach performs well on both recognition and adaptation. Overall our approach can achieve the probability of detection around 90% with a probability of false alarm below 20%. CONCLUSIONS: Our AATR shows the capabilities of adapting to varying types of materials, even the unknown materials which are not available in the training data, adapting to varying required probability of detection and adapting to varying scales of the threat object.

On the Relative Contribution of Deep Convolutional Neural Networks for SSVEP-Based Bio-Signal Decoding in BCI Speller Applications.

Brain-computer interfaces (BCI) harnessing steady state visual evoked potentials (SSVEPs) manipulate the frequency and phase of visual stimuli to generate predictable oscillations in neural activity. For BCI spellers, oscillations are matched with alphanumeric characters allowing users to select target numbers and letters. Advances in BCI spellers can, in part, be accredited to subject-specific optimization, including; 1) custom electrode arrangements; 2) filter sub-band assessments; and 3) stimulus parameter tuning. Here, we apply deep convolutional neural networks (DCNNs) demonstrating cross-subject functionality for the classification of frequency and phase encoded SSVEP. Electroencephalogram (EEG) data are collected and classified using the same parameters across subjects. Subjects fixate forty randomly cued flickering characters ( 5 ×8 keyboard array) during concurrent wet-EEG acquisition. These data are provided by an open source SSVEP dataset. Our proposed DCNN, PodNet, achieves 86% and 77% offline accuracy of classification across-subjects for two data capture periods, respectively, 6-seconds (information transfer rate = 40 bpm) and 2-seconds (information transfer rate = 101 bpm). Subjects demonstrating sub-optimal (<70%) performance are classified to similar levels after a short subject-specific training period. PodNet outperforms filter-bank canonical correlation analysis for a low volume (3-channel) clinically feasible occipital electrode configuration. The networks defined in this study achieve functional performance for the largest number of SSVEP classes decoded via DCNN to date. Our results demonstrate PodNet achieves cross-subject, calibrationless classification and adaptability to sub-optimal subject data, and low-volume EEG electrode arrangements.

On the relevance of denoising and artefact reduction in 3D segmentation and classification within complex computed tomography imagery.

We evaluate the impact of denoising and Metal Artefact Reduction (MAR) on 3D object segmentation and classification in low-resolution, cluttered dual-energy Computed Tomography (CT). To this end, we present a novel 3D materials-based segmentation technique based on the Dual-Energy Index (DEI) to automatically generate subvolumes for classification. Subvolume classification is performed using an extension of Extremely Randomised Clustering (ERC) forest codebooks, constructed using dense feature-point sampling and multiscale Density Histogram (DH) descriptors. Within this experimental framework, we evaluate the impact on classification accuracy and computational expense of pre-processing by intensity thresholding, Non-Local Means (NLM) filtering, Linear Interpolation-based MAR (LIMar) and Distance-Driven MAR (DDMar) in the domain of 3D baggage security screening. We demonstrate that basic NLM filtering, although removing fewer artefacts, produces state-of-the-art classification results comparable to the more complex DDMar but at a significant reduction in computational cost - bringing into question the importance (in terms of automated CT analysis) of computationally expensive artefact reduction techniques. Overall, it was found that the use of MAR pre-processing approaches produced only a marginal improvement in classification performance (< 1%) at considerable additional computational cost (> 10×) when compared to NLM pre-processing.

Quantitative Evaluation of Stereo Visual Odometry for Autonomous Vessel Localisation in Inland Waterway Sensing Applications.

Autonomous survey vessels can increase the efficiency and availability of wide-area river environment surveying as a tool for environment protection and conservation. A key challenge is the accurate localisation of the vessel, where bank-side vegetation or urban settlement preclude the conventional use of line-of-sight global navigation satellite systems (GNSS). In this paper, we evaluate unaided visual odometry, via an on-board stereo camera rig attached to the survey vessel, as a novel, low-cost localisation strategy. Feature-based and appearance-based visual odometry algorithms are implemented on a six degrees of freedom platform operating under guided motion, but stochastic variation in yaw, pitch and roll. Evaluation is based on a 663 m-long trajectory (>15,000 image frames) and statistical error analysis against ground truth position from a target tracking tachymeter integrating electronic distance and angular measurements. The position error of the feature-based technique (mean of ±0.067 m) is three times smaller than that of the appearance-based algorithm. From multi-variable statistical regression, we are able to attribute this error to the depth of tracked features from the camera in the scene and variations in platform yaw. Our findings inform effective strategies to enhance stereo visual localisation for the specific application of river monitoring.

A review of automated image understanding within 3D baggage computed tomography security screening.

Baggage inspection is the principal safeguard against the transportation of prohibited and potentially dangerous materials at airport security checkpoints. Although traditionally performed by 2D X-ray based scanning, increasingly stringent security regulations have led to a growing demand for more advanced imaging technologies. The role of X-ray Computed Tomography is thus rapidly expanding beyond the traditional materials-based detection of explosives. The development of computer vision and image processing techniques for the automated understanding of 3D baggage-CT imagery is however, complicated by poor image resolutions, image clutter and high levels of noise and artefacts. We discuss the recent and most pertinent advancements and identify topics for future research within the challenging domain of automated image understanding for baggage security screening CT.

An experimental survey of metal artefact reduction in computed tomography.

We present a survey of techniques for the reduction of streaking artefacts caused by metallic objects in X-ray Computed Tomography (CT) images. A comprehensive review of the existing state-of-the-art Metal Artefact Reduction (MAR) techniques, drawn predominantly from the medical CT literature, is supported by an experimental comparison of twelve MAR techniques. The experimentation is grounded in an evaluation based on a standard scientific comparison protocol for MAR methods, using a software generated medical phantom image as well as a clinical CT scan. The experimentation is extended by considering novel applications of CT imagery consisting of metal objects in non-tissue surroundings acquired from the aviation security screening domain. We address the shortage of thorough performance analyses in the existing MAR literature by conducting a qualitative as well as quantitative comparative evaluation of the selected techniques. We find that the difficulty in generating accurate priors to be the predominant factor limiting the effectiveness of the state-of-the-art medical MAR techniques when applied to non-medical CT imagery. This study thus extends previous works by: comparing several state-of-the-art MAR techniques; considering both medical and non-medical applications and performing a thorough performance analysis, considering both image quality as well as computational demands.

Morphologic characterization of cell neighborhoods in neoplastic and preneoplastic epithelium.

OBJECTIVE: To explore tissue organization based on the geometry of cell neighborhoods in histologic preparations. STUDY DESIGN: Local complexity of solid tissues was measured in images of discrete tissue compartments. Exclusive areas associated with cell nuclei (v-cells) were computed using a watershed transform of the nuclear staining intensity. Mathematical morphology was used to define neighborhood membership, distances and identify complete nested neighborhoods. Neighborhood complexity was estimated as the scaling of the number of neighbors relative to reference v-cells. RESULTS: The methodology applied to hematoxylin-eosin-stained sections from normal, dysplastic and neoplastic oral epithelium revealed that the scaling exponent, over a finite range of neighborhood levels, is nonunique and fractional. While scaling values overlapped across classes, the average was marginally higher in neoplastic than in dysplastic and normal epithelia. The best classificatory power of the exponent was 58% correct classification into 3 diagnostic classes (11 levels) and 83% between dysplastic and neoplastic classes (13 levels). CONCLUSION: V-cell architecture retains features of the original tissue classes and demonstrates an increase in tissue disorder in neoplasia. This methodology seems suitable for extracting information from tissues where identification of cell boundaries (and therefore segmentation into individual cells) is unfeasible.

Three-dimensional surface relief completion via nonparametric techniques.

Common 3D acquisition techniques, such as laser scanning and stereo capture, are realistically only 2.5D in nature. Here we consider the automated completion of hidden or missing portions in 3D scenes originally acquired from 2.5D (or 3D) capture. We propose an approach based on the non-parametric propagation of available scene knowledge from the known (visible) scene areas to these unknown (invisible) 3D regions in conjunction with an initial underlying geometric surface completion.