Prostate cancer segmentation from MRI by a multistream fusion encoder.

BACKGROUND: Targeted prostate biopsy guided by multiparametric magnetic resonance imaging (mpMRI) detects more clinically significant lesions than conventional systemic biopsy. Lesion segmentation is required for planning MRI-targeted biopsies. The requirement for integrating image features available in T2-weighted and diffusion-weighted images poses a challenge in prostate lesion segmentation from mpMRI. PURPOSE: A flexible and efficient multistream fusion encoder is proposed in this work to facilitate the multiscale fusion of features from multiple imaging streams. A patch-based loss function is introduced to improve the accuracy in segmenting small lesions. METHODS: The proposed multistream encoder fuses features extracted in the three imaging streams at each layer of the network, thereby allowing improved feature maps to propagate downstream and benefit segmentation performance. The fusion is achieved through a spatial attention map generated by optimally weighting the contribution of the convolution outputs from each stream. This design provides flexibility for the network to highlight image modalities according to their relative influence on the segmentation performance. The encoder also performs multiscale integration by highlighting the input feature maps (low-level features) with the spatial attention maps generated from convolution outputs (high-level features). The Dice similarity coefficient (DSC), serving as a cost function, is less sensitive to incorrect segmentation for small lesions. We address this issue by introducing a patch-based loss function that provides an average of the DSCs obtained from local image patches. This local average DSC is equally sensitive to large and small lesions, as the patch-based DSCs associated with small and large lesions have equal weights in this average DSC. RESULTS: The framework was evaluated in 931 sets of images acquired in several clinical studies at two centers in Hong Kong and the United Kingdom. In particular, the training, validation, and test sets contain 615, 144, and 172 sets of images, respectively. The proposed framework outperformed single-stream networks and three recently proposed multistream networks, attaining F1 scores of 82.2 and 87.6% in the lesion and patient levels, respectively. The average inference time for an axial image was 11.8 ms. CONCLUSION: The accuracy and efficiency afforded by the proposed framework would accelerate the MRI interpretation workflow of MRI-targeted biopsy and focal therapies.

Multimodal Image Classification by Multiview Latent Pattern Extraction, Selection, and Correlation.

The large amount of data available in the modern big data era opens new opportunities to expand our knowledge by integrating information from heterogeneous sources. Multiview learning has recently achieved tremendous success in deriving complementary information from multiple data modalities. This article proposes a framework called multiview latent space projection (MVLSP) to integrate features extracted from multiple sources in a discriminative way to facilitate binary and multiclass classifications. Our approach is associated with three innovations. First, most existing multiview learning algorithms promote pairwise consistency between two views and do not have a natural extension to applications with more than two views. MVLSP finds optimum mappings from a common latent space to match the feature space in each of the views. As the matching is performed on a view-by-view basis, the framework can be readily extended to multiview applications. Second, feature selection in the common latent space can be readily achieved by adding a class view, which matches the latent space representations of training samples with their corresponding labels. Then, high-order view correlations are extracted by considering feature-label correlations. Third, a technique is proposed to optimize the integration of different latent patterns based on their correlations. The experimental results on the prostate image dataset demonstrate the effectiveness of the proposed method.

Sleep stage classification based on multi-level feature learning and recurrent neural networks via wearable device.

BACKGROUND: Automatic sleep stage classification is essential for long-term sleep monitoring. Wearable devices show more advantages than polysomnography for home use. In this paper, we propose a novel method for sleep staging using heart rate and wrist actigraphy derived from a wearable device. METHODS: The proposed method consists of two phases: multi-level feature learning and recurrent neural networks-based (RNNs) classification. The feature learning phase is designed to extract low- and mid-level features. Low-level features are extracted from raw signals, capturing temporal and frequency domain properties. Mid-level features are explored based on low-level ones to learn compositions and structural information of signals. Sleep staging is a sequential problem with long-term dependencies. RNNs with bidirectional long short-term memory architectures are employed to learn temporally sequential patterns. RESULTS: To better simulate the use of wearable devices in the daily scene, experiments were conducted with a resting group in which sleep was recorded in the resting state, and a comprehensive group in which both resting sleep and non-resting sleep were included. The proposed algorithm classified five sleep stages (wake, non-rapid eye movement 1-3, and rapid eye movement) and achieved weighted precision, recall, and F1 score of 66.6%, 67.7%, and 64.0% in the resting group and 64.5%, 65.0%, and 60.5% in the comprehensive group using leave-one-out cross-validation. Various comparison experiments demonstrated the effectiveness of the algorithm. CONCLUSIONS: Our method is efficient and effective in scoring sleep stages. It is suitable to be applied to wearable devices for monitoring sleep at home.